Jul 15 11:56:55.879529 ip-10-0-135-243 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 15 11:56:55.879540 ip-10-0-135-243 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 15 11:56:55.879548 ip-10-0-135-243 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 15 11:56:55.879800 ip-10-0-135-243 systemd[1]: Failed to start Kubernetes Kubelet. Jul 15 11:57:04.967964 ip-10-0-135-243 systemd[1]: kubelet.service: Failed to schedule restart job: Unit openvswitch.service not found. Jul 15 11:57:04.967981 ip-10-0-135-243 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot fb5bc21a554a4ae5a80d5009987f4f4a -- Jul 15 11:58:48.429065 ip-10-0-135-243 systemd[1]: Starting Kubernetes Kubelet... Jul 15 11:58:48.907023 ip-10-0-135-243 kubenswrapper[2587]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 11:58:48.907023 ip-10-0-135-243 kubenswrapper[2587]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 15 11:58:48.907023 ip-10-0-135-243 kubenswrapper[2587]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 11:58:48.907023 ip-10-0-135-243 kubenswrapper[2587]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 15 11:58:48.907023 ip-10-0-135-243 kubenswrapper[2587]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 11:58:48.907023 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.855756 2587 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 15 11:58:48.907023 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.861940 2587 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 11:58:48.907023 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.861955 2587 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 11:58:48.907023 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.861959 2587 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 11:58:48.907023 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.861963 2587 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 11:58:48.907023 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.861966 2587 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 11:58:48.907023 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.861970 2587 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 11:58:48.907023 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.861974 2587 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 11:58:48.907023 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.861983 2587 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 11:58:48.907846 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.861987 2587 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 11:58:48.907846 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.861991 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 11:58:48.907846 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.861994 2587 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 11:58:48.907846 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.861998 2587 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 11:58:48.907846 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862001 2587 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 11:58:48.907846 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862004 2587 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 11:58:48.907846 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862008 2587 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 11:58:48.907846 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862011 2587 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 11:58:48.907846 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862014 2587 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 11:58:48.907846 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862017 2587 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 11:58:48.907846 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862021 2587 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 11:58:48.907846 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862023 2587 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 11:58:48.907846 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862026 2587 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 11:58:48.907846 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862029 2587 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 11:58:48.907846 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862033 2587 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 11:58:48.907846 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862036 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 11:58:48.907846 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862038 2587 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 11:58:48.907846 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862042 2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 11:58:48.907846 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862046 2587 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 11:58:48.907846 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862048 2587 feature_gate.go:328] unrecognized feature gate: Example Jul 15 11:58:48.908403 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862052 2587 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 11:58:48.908403 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862056 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 11:58:48.908403 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862059 2587 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 11:58:48.908403 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862063 2587 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 11:58:48.908403 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862066 2587 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 11:58:48.908403 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862068 2587 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 11:58:48.908403 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862071 2587 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 11:58:48.908403 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862073 2587 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 11:58:48.908403 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862076 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 11:58:48.908403 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862078 2587 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 11:58:48.908403 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862083 2587 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 11:58:48.908403 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862086 2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 11:58:48.908403 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862089 2587 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 11:58:48.908403 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862091 2587 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 11:58:48.908403 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862094 2587 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 11:58:48.908403 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862097 2587 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 11:58:48.908403 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862100 2587 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 11:58:48.908403 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862102 2587 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 11:58:48.908403 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862105 2587 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 11:58:48.908403 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862108 2587 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 11:58:48.909036 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862111 2587 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 11:58:48.909036 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862114 2587 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 11:58:48.909036 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862116 2587 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 11:58:48.909036 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862121 2587 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 11:58:48.909036 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862125 2587 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 11:58:48.909036 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862127 2587 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 11:58:48.909036 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862131 2587 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 11:58:48.909036 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862135 2587 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 11:58:48.909036 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862139 2587 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 11:58:48.909036 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862144 2587 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 11:58:48.909036 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862148 2587 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 11:58:48.909036 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862151 2587 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 11:58:48.909036 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862155 2587 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 11:58:48.909036 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862159 2587 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 11:58:48.909036 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862162 2587 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 11:58:48.909036 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862167 2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 11:58:48.909036 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862170 2587 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 11:58:48.909036 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862173 2587 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 11:58:48.909036 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862176 2587 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 11:58:48.909834 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862180 2587 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 11:58:48.909834 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862182 2587 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 11:58:48.909834 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862185 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 11:58:48.909834 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862187 2587 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 11:58:48.909834 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862192 2587 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 11:58:48.909834 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862195 2587 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 11:58:48.909834 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862197 2587 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 11:58:48.909834 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862200 2587 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 11:58:48.909834 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862203 2587 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 11:58:48.909834 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862207 2587 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 11:58:48.909834 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862209 2587 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 11:58:48.909834 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862212 2587 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 11:58:48.909834 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862216 2587 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 11:58:48.909834 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862220 2587 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 11:58:48.909834 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862224 2587 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 11:58:48.909834 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862228 2587 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 11:58:48.909834 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862231 2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 11:58:48.909834 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862234 2587 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 11:58:48.909834 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862237 2587 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 11:58:48.910491 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862241 2587 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 11:58:48.910491 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862244 2587 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 11:58:48.910491 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862247 2587 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 11:58:48.910491 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862250 2587 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 11:58:48.910491 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862254 2587 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 11:58:48.910491 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862257 2587 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 11:58:48.910491 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862260 2587 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 11:58:48.910491 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862263 2587 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 11:58:48.910491 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862266 2587 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 11:58:48.910491 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862268 2587 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 11:58:48.910491 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862272 2587 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 11:58:48.910491 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862275 2587 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 11:58:48.910491 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.862277 2587 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 11:58:48.910491 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863067 2587 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 11:58:48.910491 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863076 2587 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 11:58:48.910491 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863081 2587 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 11:58:48.910491 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863085 2587 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 11:58:48.910491 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863090 2587 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 11:58:48.910491 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863100 2587 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 11:58:48.911098 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863104 2587 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 11:58:48.911098 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863107 2587 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 11:58:48.911098 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863111 2587 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 11:58:48.911098 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863114 2587 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 11:58:48.911098 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863117 2587 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 11:58:48.911098 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863120 2587 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 11:58:48.911098 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863122 2587 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 11:58:48.911098 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863125 2587 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 11:58:48.911098 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863128 2587 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 11:58:48.911098 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863130 2587 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 11:58:48.911098 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863133 2587 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 11:58:48.911098 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863137 2587 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 11:58:48.911098 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863141 2587 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 11:58:48.911098 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863144 2587 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 11:58:48.911098 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863148 2587 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 11:58:48.911098 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863150 2587 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 11:58:48.911098 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863153 2587 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 11:58:48.911098 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863156 2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 11:58:48.911098 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863161 2587 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 11:58:48.911098 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863164 2587 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 11:58:48.911098 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863166 2587 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 11:58:48.911757 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863171 2587 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 11:58:48.911757 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863174 2587 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 11:58:48.911757 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863176 2587 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 11:58:48.911757 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863179 2587 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 11:58:48.911757 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863182 2587 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 11:58:48.911757 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863186 2587 feature_gate.go:328] unrecognized feature gate: Example Jul 15 11:58:48.911757 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863188 2587 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 11:58:48.911757 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863191 2587 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 11:58:48.911757 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863194 2587 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 11:58:48.911757 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863198 2587 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 11:58:48.911757 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863200 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 11:58:48.911757 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863203 2587 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 11:58:48.911757 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863207 2587 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 11:58:48.911757 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863210 2587 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 11:58:48.911757 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863212 2587 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 11:58:48.911757 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863215 2587 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 11:58:48.911757 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863218 2587 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 11:58:48.911757 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863222 2587 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 11:58:48.911757 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863225 2587 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 11:58:48.911757 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863227 2587 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 11:58:48.912540 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863231 2587 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 11:58:48.912540 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863234 2587 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 11:58:48.912540 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863237 2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 11:58:48.912540 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863239 2587 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 11:58:48.912540 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863243 2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 11:58:48.912540 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863246 2587 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 11:58:48.912540 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863249 2587 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 11:58:48.912540 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863251 2587 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 11:58:48.912540 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863255 2587 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 11:58:48.912540 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863258 2587 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 11:58:48.912540 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863260 2587 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 11:58:48.912540 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863263 2587 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 11:58:48.912540 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863266 2587 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 11:58:48.912540 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863270 2587 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 11:58:48.912540 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863273 2587 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 11:58:48.912540 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863275 2587 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 11:58:48.912540 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863278 2587 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 11:58:48.912540 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863281 2587 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 11:58:48.912540 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863284 2587 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 11:58:48.913067 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863286 2587 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 11:58:48.913067 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863290 2587 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 11:58:48.913067 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863293 2587 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 11:58:48.913067 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863296 2587 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 11:58:48.913067 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863298 2587 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 11:58:48.913067 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863302 2587 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 11:58:48.913067 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863305 2587 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 11:58:48.913067 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863308 2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 11:58:48.913067 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863310 2587 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 11:58:48.913067 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863314 2587 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 11:58:48.913067 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863317 2587 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 11:58:48.913067 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863322 2587 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 11:58:48.913067 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863328 2587 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 11:58:48.913067 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863341 2587 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 11:58:48.913067 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863344 2587 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 11:58:48.913067 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863347 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 11:58:48.913067 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863350 2587 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 11:58:48.913067 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863355 2587 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 11:58:48.913067 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863358 2587 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 11:58:48.913666 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863360 2587 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 11:58:48.913666 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863364 2587 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 11:58:48.913666 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863367 2587 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 11:58:48.913666 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863370 2587 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 11:58:48.913666 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863372 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 11:58:48.913666 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863376 2587 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 11:58:48.913666 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863379 2587 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 11:58:48.913666 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863381 2587 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 11:58:48.913666 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863384 2587 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 11:58:48.913666 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863387 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 11:58:48.913666 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863390 2587 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 11:58:48.913666 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863393 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 11:58:48.913666 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863397 2587 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 11:58:48.913666 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863401 2587 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 11:58:48.913666 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863828 2587 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 11:58:48.913666 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863834 2587 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 11:58:48.913666 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863837 2587 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 11:58:48.913666 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863841 2587 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 11:58:48.913666 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863846 2587 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 11:58:48.914296 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863849 2587 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 11:58:48.914296 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863851 2587 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 11:58:48.914296 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863855 2587 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 11:58:48.914296 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863858 2587 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 11:58:48.914296 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863861 2587 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 11:58:48.914296 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863867 2587 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 11:58:48.914296 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863871 2587 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 11:58:48.914296 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863873 2587 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 11:58:48.914296 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863876 2587 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 11:58:48.914296 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863879 2587 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 11:58:48.914296 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863881 2587 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 11:58:48.914296 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863884 2587 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 11:58:48.914296 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863887 2587 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 11:58:48.914296 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863889 2587 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 11:58:48.914296 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863892 2587 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 11:58:48.914296 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863894 2587 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 11:58:48.914296 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863898 2587 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 11:58:48.914296 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863902 2587 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 11:58:48.914296 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863904 2587 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 11:58:48.914296 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863907 2587 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 11:58:48.914880 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863911 2587 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 11:58:48.914880 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863915 2587 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 11:58:48.914880 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863919 2587 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 11:58:48.914880 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863923 2587 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 11:58:48.914880 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863926 2587 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 11:58:48.914880 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863929 2587 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 11:58:48.914880 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863932 2587 feature_gate.go:328] unrecognized feature gate: Example Jul 15 11:58:48.914880 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863936 2587 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 11:58:48.914880 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863939 2587 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 11:58:48.914880 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863941 2587 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 11:58:48.914880 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863944 2587 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 11:58:48.914880 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863948 2587 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 11:58:48.914880 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863953 2587 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 11:58:48.914880 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863956 2587 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 11:58:48.914880 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863958 2587 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 11:58:48.914880 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863962 2587 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 11:58:48.914880 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863965 2587 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 11:58:48.914880 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863969 2587 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 11:58:48.914880 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863972 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 11:58:48.914880 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863976 2587 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 11:58:48.915642 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863979 2587 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 11:58:48.915642 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863981 2587 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 11:58:48.915642 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863984 2587 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 11:58:48.915642 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863986 2587 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 11:58:48.915642 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863989 2587 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 11:58:48.915642 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863991 2587 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 11:58:48.915642 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863994 2587 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 11:58:48.915642 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.863998 2587 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 11:58:48.915642 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.864000 2587 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 11:58:48.915642 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.864003 2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 11:58:48.915642 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.864006 2587 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 11:58:48.915642 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.864010 2587 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 11:58:48.915642 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.864013 2587 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 11:58:48.915642 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.864015 2587 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 11:58:48.915642 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.864018 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 11:58:48.915642 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.864022 2587 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 11:58:48.915642 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.864025 2587 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 11:58:48.915642 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.864028 2587 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 11:58:48.915642 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.864031 2587 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 11:58:48.915642 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.864034 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 11:58:48.916269 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.864036 2587 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 11:58:48.916269 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.864040 2587 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 11:58:48.916269 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.864042 2587 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 11:58:48.916269 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.864046 2587 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 11:58:48.916269 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.864049 2587 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 11:58:48.916269 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.864052 2587 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 11:58:48.916269 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.864055 2587 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 11:58:48.916269 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.864058 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 11:58:48.916269 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.864063 2587 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 11:58:48.916269 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.864067 2587 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 11:58:48.916269 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.864071 2587 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 11:58:48.916269 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.864074 2587 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 11:58:48.916269 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.864077 2587 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 11:58:48.916269 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.864080 2587 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 11:58:48.916269 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.864083 2587 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 11:58:48.916269 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.864086 2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 11:58:48.916269 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.864089 2587 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 11:58:48.916269 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.864091 2587 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 11:58:48.916269 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.864095 2587 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 11:58:48.916893 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.864098 2587 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 11:58:48.916893 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.864100 2587 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 11:58:48.916893 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.864103 2587 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 11:58:48.916893 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.864107 2587 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 11:58:48.916893 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.864110 2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 11:58:48.916893 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.864113 2587 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 11:58:48.916893 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.864116 2587 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 11:58:48.916893 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.864120 2587 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 11:58:48.916893 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.864123 2587 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 11:58:48.916893 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.864126 2587 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 11:58:48.916893 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.864131 2587 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 11:58:48.916893 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.864133 2587 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 11:58:48.916893 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.864136 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 11:58:48.916893 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.864139 2587 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 11:58:48.916893 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.864142 2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 11:58:48.916893 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864223 2587 flags.go:64] FLAG: --address="0.0.0.0" Jul 15 11:58:48.916893 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864238 2587 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 15 11:58:48.916893 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864247 2587 flags.go:64] FLAG: --anonymous-auth="true" Jul 15 11:58:48.916893 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864254 2587 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 15 11:58:48.916893 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864260 2587 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 15 11:58:48.916893 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864265 2587 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 15 11:58:48.917522 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864272 2587 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 15 11:58:48.917522 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864277 2587 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 15 11:58:48.917522 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864282 2587 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 15 11:58:48.917522 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864286 2587 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 15 11:58:48.917522 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864289 2587 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 15 11:58:48.917522 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864293 2587 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 15 11:58:48.917522 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864299 2587 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 15 11:58:48.917522 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864302 2587 flags.go:64] FLAG: --cgroup-root="" Jul 15 11:58:48.917522 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864306 2587 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 15 11:58:48.917522 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864309 2587 flags.go:64] FLAG: --client-ca-file="" Jul 15 11:58:48.917522 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864312 2587 flags.go:64] FLAG: --cloud-provider="external" Jul 15 11:58:48.917522 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864316 2587 flags.go:64] FLAG: --cluster-dns="[]" Jul 15 11:58:48.917522 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864322 2587 flags.go:64] FLAG: --cluster-domain="" Jul 15 11:58:48.917522 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864325 2587 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 15 11:58:48.917522 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864329 2587 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 15 11:58:48.917522 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864333 2587 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 15 11:58:48.917522 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864336 2587 flags.go:64] FLAG: --container-log-max-files="5" Jul 15 11:58:48.917522 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864341 2587 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 15 11:58:48.917522 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864345 2587 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 15 11:58:48.917522 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864348 2587 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 15 11:58:48.917522 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864352 2587 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 15 11:58:48.917522 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864356 2587 flags.go:64] FLAG: --contention-profiling="false" Jul 15 11:58:48.917522 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864359 2587 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 15 11:58:48.917522 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864362 2587 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 15 11:58:48.918143 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864366 2587 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 15 11:58:48.918143 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864369 2587 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 15 11:58:48.918143 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864374 2587 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 15 11:58:48.918143 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864378 2587 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 15 11:58:48.918143 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864381 2587 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 15 11:58:48.918143 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864384 2587 flags.go:64] FLAG: --enable-load-reader="false" Jul 15 11:58:48.918143 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864387 2587 flags.go:64] FLAG: --enable-server="true" Jul 15 11:58:48.918143 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864392 2587 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 15 11:58:48.918143 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864396 2587 flags.go:64] FLAG: --event-burst="100" Jul 15 11:58:48.918143 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864401 2587 flags.go:64] FLAG: --event-qps="50" Jul 15 11:58:48.918143 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864405 2587 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 15 11:58:48.918143 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864408 2587 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 15 11:58:48.918143 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864426 2587 flags.go:64] FLAG: --eviction-hard="" Jul 15 11:58:48.918143 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864430 2587 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 15 11:58:48.918143 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864436 2587 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 15 11:58:48.918143 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864440 2587 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 15 11:58:48.918143 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864443 2587 flags.go:64] FLAG: --eviction-soft="" Jul 15 11:58:48.918143 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864447 2587 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 15 11:58:48.918143 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864451 2587 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 15 11:58:48.918143 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864454 2587 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 15 11:58:48.918143 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864457 2587 flags.go:64] FLAG: --experimental-mounter-path="" Jul 15 11:58:48.918143 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864461 2587 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 15 11:58:48.918143 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864465 2587 flags.go:64] FLAG: --fail-swap-on="true" Jul 15 11:58:48.918143 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864468 2587 flags.go:64] FLAG: --feature-gates="" Jul 15 11:58:48.918143 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864473 2587 flags.go:64] FLAG: --file-check-frequency="20s" Jul 15 11:58:48.918965 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864477 2587 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 15 11:58:48.918965 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864481 2587 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 15 11:58:48.918965 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864485 2587 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 15 11:58:48.918965 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864488 2587 flags.go:64] FLAG: --healthz-port="10248" Jul 15 11:58:48.918965 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864492 2587 flags.go:64] FLAG: --help="false" Jul 15 11:58:48.918965 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864495 2587 flags.go:64] FLAG: --hostname-override="ip-10-0-135-243.ec2.internal" Jul 15 11:58:48.918965 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864498 2587 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 15 11:58:48.918965 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864501 2587 flags.go:64] FLAG: --http-check-frequency="20s" Jul 15 11:58:48.918965 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864505 2587 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 15 11:58:48.918965 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864510 2587 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 15 11:58:48.918965 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864513 2587 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 15 11:58:48.918965 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864517 2587 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 15 11:58:48.918965 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864521 2587 flags.go:64] FLAG: --image-service-endpoint="" Jul 15 11:58:48.918965 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864524 2587 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 15 11:58:48.918965 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864527 2587 flags.go:64] FLAG: --kube-api-burst="100" Jul 15 11:58:48.918965 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864532 2587 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 15 11:58:48.918965 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864538 2587 flags.go:64] FLAG: --kube-api-qps="50" Jul 15 11:58:48.918965 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864542 2587 flags.go:64] FLAG: --kube-reserved="" Jul 15 11:58:48.918965 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864545 2587 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 15 11:58:48.918965 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864548 2587 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 15 11:58:48.918965 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864551 2587 flags.go:64] FLAG: --kubelet-cgroups="" Jul 15 11:58:48.918965 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864556 2587 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 15 11:58:48.918965 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864560 2587 flags.go:64] FLAG: --lock-file="" Jul 15 11:58:48.918965 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864562 2587 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 15 11:58:48.920991 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864567 2587 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 15 11:58:48.920991 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864570 2587 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 15 11:58:48.920991 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864577 2587 flags.go:64] FLAG: --log-json-split-stream="false" Jul 15 11:58:48.920991 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864580 2587 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 15 11:58:48.920991 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864583 2587 flags.go:64] FLAG: --log-text-split-stream="false" Jul 15 11:58:48.920991 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864586 2587 flags.go:64] FLAG: --logging-format="text" Jul 15 11:58:48.920991 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864590 2587 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 15 11:58:48.920991 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864593 2587 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 15 11:58:48.920991 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864596 2587 flags.go:64] FLAG: --manifest-url="" Jul 15 11:58:48.920991 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864600 2587 flags.go:64] FLAG: --manifest-url-header="" Jul 15 11:58:48.920991 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864605 2587 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 15 11:58:48.920991 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864607 2587 flags.go:64] FLAG: --max-open-files="1000000" Jul 15 11:58:48.920991 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864613 2587 flags.go:64] FLAG: --max-pods="110" Jul 15 11:58:48.920991 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864617 2587 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 15 11:58:48.920991 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864620 2587 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 15 11:58:48.920991 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864622 2587 flags.go:64] FLAG: --memory-manager-policy="None" Jul 15 11:58:48.920991 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864625 2587 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 15 11:58:48.920991 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864628 2587 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 15 11:58:48.920991 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864632 2587 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 15 11:58:48.920991 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864636 2587 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 15 11:58:48.920991 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864645 2587 flags.go:64] FLAG: --node-status-max-images="50" Jul 15 11:58:48.920991 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864648 2587 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 15 11:58:48.920991 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864651 2587 flags.go:64] FLAG: --oom-score-adj="-999" Jul 15 11:58:48.920991 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864655 2587 flags.go:64] FLAG: --pod-cidr="" Jul 15 11:58:48.999269 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864667 2587 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 15 11:58:48.999269 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864674 2587 flags.go:64] FLAG: --pod-manifest-path="" Jul 15 11:58:48.999269 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864678 2587 flags.go:64] FLAG: --pod-max-pids="-1" Jul 15 11:58:48.999269 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864682 2587 flags.go:64] FLAG: --pods-per-core="0" Jul 15 11:58:48.999269 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864685 2587 flags.go:64] FLAG: --port="10250" Jul 15 11:58:48.999269 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864689 2587 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 15 11:58:48.999269 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864693 2587 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-098324a7f69348f26" Jul 15 11:58:48.999269 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864697 2587 flags.go:64] FLAG: --qos-reserved="" Jul 15 11:58:48.999269 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864700 2587 flags.go:64] FLAG: --read-only-port="10255" Jul 15 11:58:48.999269 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864704 2587 flags.go:64] FLAG: --register-node="true" Jul 15 11:58:48.999269 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864707 2587 flags.go:64] FLAG: --register-with-taints="" Jul 15 11:58:48.999269 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864717 2587 flags.go:64] FLAG: --registry-burst="10" Jul 15 11:58:48.999269 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864720 2587 flags.go:64] FLAG: --registry-qps="5" Jul 15 11:58:48.999269 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864722 2587 flags.go:64] FLAG: --reserved-cpus="" Jul 15 11:58:48.999269 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864726 2587 flags.go:64] FLAG: --reserved-memory="" Jul 15 11:58:48.999269 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864744 2587 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 15 11:58:48.999269 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864747 2587 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 15 11:58:48.999269 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864751 2587 flags.go:64] FLAG: --rotate-certificates="false" Jul 15 11:58:48.999269 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864755 2587 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 15 11:58:48.999269 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864758 2587 flags.go:64] FLAG: --runonce="false" Jul 15 11:58:48.999269 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864761 2587 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 15 11:58:48.999269 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864766 2587 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 15 11:58:48.999269 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864769 2587 flags.go:64] FLAG: --seccomp-default="false" Jul 15 11:58:48.999269 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864772 2587 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 15 11:58:48.999269 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864776 2587 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 15 11:58:48.966607 ip-10-0-135-243 systemd[1]: Started Kubernetes Kubelet. Jul 15 11:58:49.007659 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864779 2587 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 15 11:58:49.007659 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864782 2587 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 15 11:58:49.007659 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864786 2587 flags.go:64] FLAG: --storage-driver-password="root" Jul 15 11:58:49.007659 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864790 2587 flags.go:64] FLAG: --storage-driver-secure="false" Jul 15 11:58:49.007659 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864793 2587 flags.go:64] FLAG: --storage-driver-table="stats" Jul 15 11:58:49.007659 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864796 2587 flags.go:64] FLAG: --storage-driver-user="root" Jul 15 11:58:49.007659 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864800 2587 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 15 11:58:49.007659 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864803 2587 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 15 11:58:49.007659 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864809 2587 flags.go:64] FLAG: --system-cgroups="" Jul 15 11:58:49.007659 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864813 2587 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 15 11:58:49.007659 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864820 2587 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 15 11:58:49.007659 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864823 2587 flags.go:64] FLAG: --tls-cert-file="" Jul 15 11:58:49.007659 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864827 2587 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 15 11:58:49.007659 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864832 2587 flags.go:64] FLAG: --tls-min-version="" Jul 15 11:58:49.007659 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864835 2587 flags.go:64] FLAG: --tls-private-key-file="" Jul 15 11:58:49.007659 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864838 2587 flags.go:64] FLAG: --topology-manager-policy="none" Jul 15 11:58:49.007659 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864841 2587 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 15 11:58:49.007659 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864846 2587 flags.go:64] FLAG: --topology-manager-scope="container" Jul 15 11:58:49.007659 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864849 2587 flags.go:64] FLAG: --v="2" Jul 15 11:58:49.007659 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864854 2587 flags.go:64] FLAG: --version="false" Jul 15 11:58:49.007659 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864859 2587 flags.go:64] FLAG: --vmodule="" Jul 15 11:58:49.007659 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864863 2587 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 15 11:58:49.007659 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.864867 2587 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 15 11:58:49.007659 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.864996 2587 feature_gate.go:328] unrecognized feature gate: Example Jul 15 11:58:49.007659 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865001 2587 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 11:58:49.008609 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865004 2587 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 11:58:49.008609 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865007 2587 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 11:58:49.008609 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865011 2587 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 11:58:49.008609 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865014 2587 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 11:58:49.008609 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865017 2587 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 11:58:49.008609 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865020 2587 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 11:58:49.008609 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865023 2587 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 11:58:49.008609 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865026 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 11:58:49.008609 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865030 2587 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 11:58:49.008609 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865033 2587 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 11:58:49.008609 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865036 2587 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 11:58:49.008609 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865039 2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 11:58:49.008609 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865041 2587 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 11:58:49.008609 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865044 2587 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 11:58:49.008609 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865046 2587 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 11:58:49.008609 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865051 2587 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 11:58:49.008609 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865055 2587 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 11:58:49.008609 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865057 2587 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 11:58:49.008609 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865060 2587 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 11:58:49.008609 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865065 2587 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 11:58:49.009530 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865068 2587 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 11:58:49.009530 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865072 2587 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 11:58:49.009530 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865077 2587 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 11:58:49.009530 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865080 2587 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 11:58:49.009530 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865083 2587 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 11:58:49.009530 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865085 2587 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 11:58:49.009530 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865089 2587 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 11:58:49.009530 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865092 2587 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 11:58:49.009530 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865094 2587 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 11:58:49.009530 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865097 2587 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 11:58:49.009530 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865101 2587 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 11:58:49.009530 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865103 2587 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 11:58:49.009530 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865106 2587 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 11:58:49.009530 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865108 2587 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 11:58:49.009530 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865112 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 11:58:49.009530 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865115 2587 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 11:58:49.009530 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865118 2587 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 11:58:49.009530 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865120 2587 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 11:58:49.009530 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865123 2587 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 11:58:49.009530 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865127 2587 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 11:58:49.010552 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865129 2587 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 11:58:49.010552 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865132 2587 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 11:58:49.010552 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865135 2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 11:58:49.010552 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865139 2587 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 11:58:49.010552 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865142 2587 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 11:58:49.010552 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865144 2587 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 11:58:49.010552 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865147 2587 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 11:58:49.010552 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865152 2587 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 11:58:49.010552 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865154 2587 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 11:58:49.010552 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865158 2587 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 11:58:49.010552 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865162 2587 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 11:58:49.010552 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865166 2587 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 11:58:49.010552 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865168 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 11:58:49.010552 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865172 2587 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 11:58:49.010552 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865175 2587 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 11:58:49.010552 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865179 2587 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 11:58:49.010552 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865183 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 11:58:49.010552 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865187 2587 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 11:58:49.010552 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865190 2587 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 11:58:49.013403 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865193 2587 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 11:58:49.013403 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865198 2587 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 11:58:49.013403 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865201 2587 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 11:58:49.013403 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865204 2587 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 11:58:49.013403 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865207 2587 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 11:58:49.013403 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865211 2587 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 11:58:49.013403 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865214 2587 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 11:58:49.013403 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865216 2587 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 11:58:49.013403 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865220 2587 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 11:58:49.013403 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865223 2587 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 11:58:49.013403 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865227 2587 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 11:58:49.013403 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865229 2587 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 11:58:49.013403 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865233 2587 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 11:58:49.013403 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865235 2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 11:58:49.013403 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865238 2587 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 11:58:49.013403 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865241 2587 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 11:58:49.013403 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865245 2587 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 11:58:49.013403 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865247 2587 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 11:58:49.013403 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865250 2587 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 11:58:49.015223 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865252 2587 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 11:58:49.015223 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865256 2587 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 11:58:49.015223 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865258 2587 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 11:58:49.015223 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865261 2587 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 11:58:49.015223 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865265 2587 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 11:58:49.015223 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865269 2587 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 11:58:49.015223 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865271 2587 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 11:58:49.015223 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865275 2587 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 11:58:49.015223 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865279 2587 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 11:58:49.015223 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865281 2587 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 11:58:49.015223 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865284 2587 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 11:58:49.015223 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865287 2587 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 11:58:49.015223 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865290 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 11:58:49.015223 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865293 2587 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 11:58:49.015223 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865296 2587 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 11:58:49.015223 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865299 2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 11:58:49.015223 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865302 2587 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 11:58:49.015223 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865305 2587 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 11:58:49.015223 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.865308 2587 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 11:58:49.016244 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.865895 2587 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 11:58:49.016244 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.873052 2587 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 15 11:58:49.016244 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.873067 2587 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 15 11:58:49.016244 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.873748 2587 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 15 11:58:49.016244 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.873755 2587 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 15 11:58:49.016244 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.873824 2587 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 11:58:49.016244 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.873830 2587 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 11:58:49.016244 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.873835 2587 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 11:58:49.016244 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.873838 2587 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 11:58:49.016244 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.873841 2587 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 11:58:49.016244 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.873844 2587 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 11:58:49.016244 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.873846 2587 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 11:58:49.016244 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.873850 2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 11:58:49.016244 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.873852 2587 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 11:58:49.016244 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.873855 2587 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 11:58:49.017269 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.873857 2587 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 11:58:49.017269 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.873860 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 11:58:49.017269 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.873869 2587 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 11:58:49.017269 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.873872 2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 11:58:49.017269 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.873874 2587 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 11:58:49.017269 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.873877 2587 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 11:58:49.017269 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.873881 2587 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 11:58:49.017269 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.873885 2587 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 11:58:49.017269 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.873889 2587 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 11:58:49.017269 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.873892 2587 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 11:58:49.017269 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.873899 2587 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 11:58:49.017269 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.873902 2587 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 11:58:49.017269 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.873905 2587 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 11:58:49.017269 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.873908 2587 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 11:58:49.017269 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.873911 2587 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 11:58:49.017269 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.873913 2587 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 11:58:49.017269 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.873916 2587 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 11:58:49.017269 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.873918 2587 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 11:58:49.017269 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.873921 2587 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 11:58:49.018288 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.873924 2587 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 11:58:49.018288 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.873926 2587 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 11:58:49.018288 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.873929 2587 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 11:58:49.018288 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.873932 2587 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 11:58:49.018288 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.873935 2587 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 11:58:49.018288 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.873937 2587 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 11:58:49.018288 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.873940 2587 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 11:58:49.018288 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.873942 2587 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 11:58:49.018288 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.873945 2587 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 11:58:49.018288 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.873948 2587 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 11:58:49.018288 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.873950 2587 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 11:58:49.018288 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.873953 2587 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 11:58:49.018288 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.873955 2587 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 11:58:49.018288 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.873958 2587 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 11:58:49.018288 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.873960 2587 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 11:58:49.018288 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.873962 2587 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 11:58:49.018288 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.873972 2587 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 11:58:49.018288 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.873975 2587 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 11:58:49.018288 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.873978 2587 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 11:58:49.018288 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.873980 2587 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 11:58:49.019060 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.873983 2587 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 11:58:49.019060 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.873985 2587 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 11:58:49.019060 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.873988 2587 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 11:58:49.019060 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.873990 2587 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 11:58:49.019060 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.873993 2587 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 11:58:49.019060 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.873996 2587 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 11:58:49.019060 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874001 2587 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 11:58:49.019060 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874003 2587 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 11:58:49.019060 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874006 2587 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 11:58:49.019060 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874008 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 11:58:49.019060 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874011 2587 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 11:58:49.019060 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874013 2587 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 11:58:49.019060 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874016 2587 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 11:58:49.019060 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874018 2587 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 11:58:49.019060 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874021 2587 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 11:58:49.019060 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874024 2587 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 11:58:49.019060 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874026 2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 11:58:49.019060 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874029 2587 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 11:58:49.019060 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874031 2587 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 11:58:49.019060 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874034 2587 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 11:58:49.019902 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874036 2587 feature_gate.go:328] unrecognized feature gate: Example Jul 15 11:58:49.019902 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874039 2587 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 11:58:49.019902 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874041 2587 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 11:58:49.019902 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874044 2587 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 11:58:49.019902 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874046 2587 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 11:58:49.019902 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874049 2587 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 11:58:49.019902 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874051 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 11:58:49.019902 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874054 2587 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 11:58:49.019902 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874056 2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 11:58:49.019902 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874065 2587 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 11:58:49.019902 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874068 2587 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 11:58:49.019902 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874070 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 11:58:49.019902 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874073 2587 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 11:58:49.019902 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874075 2587 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 11:58:49.019902 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874078 2587 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 11:58:49.019902 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874080 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 11:58:49.019902 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874083 2587 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 11:58:49.019902 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874085 2587 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 11:58:49.019902 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874091 2587 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 11:58:49.019902 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874094 2587 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 11:58:49.020992 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874096 2587 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 11:58:49.020992 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874099 2587 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 11:58:49.020992 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874101 2587 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 11:58:49.020992 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874104 2587 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 11:58:49.020992 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874107 2587 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 11:58:49.020992 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874110 2587 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 11:58:49.020992 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874113 2587 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 11:58:49.020992 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874115 2587 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 11:58:49.020992 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874118 2587 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 11:58:49.020992 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874120 2587 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 11:58:49.020992 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.874125 2587 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 11:58:49.020992 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874280 2587 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 11:58:49.020992 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874284 2587 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 11:58:49.020992 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874287 2587 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 11:58:49.020992 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874290 2587 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 11:58:49.021466 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874292 2587 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 11:58:49.021466 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874295 2587 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 11:58:49.021466 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874297 2587 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 11:58:49.021466 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874300 2587 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 11:58:49.021466 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874302 2587 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 11:58:49.021466 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874305 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 11:58:49.021466 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874307 2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 11:58:49.021466 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874316 2587 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 11:58:49.021466 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874318 2587 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 11:58:49.021466 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874321 2587 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 11:58:49.021466 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874323 2587 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 11:58:49.021466 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874325 2587 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 11:58:49.021466 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874328 2587 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 11:58:49.021466 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874331 2587 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 11:58:49.021466 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874333 2587 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 11:58:49.021466 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874336 2587 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 11:58:49.021466 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874339 2587 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 11:58:49.021466 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874341 2587 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 11:58:49.021466 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874345 2587 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 11:58:49.021466 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874347 2587 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 11:58:49.022275 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874350 2587 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 11:58:49.022275 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874353 2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 11:58:49.022275 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874357 2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 11:58:49.022275 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874360 2587 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 11:58:49.022275 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874362 2587 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 11:58:49.022275 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874365 2587 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 11:58:49.022275 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874367 2587 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 11:58:49.022275 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874370 2587 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 11:58:49.022275 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874372 2587 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 11:58:49.022275 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874375 2587 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 11:58:49.022275 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874377 2587 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 11:58:49.022275 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874380 2587 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 11:58:49.022275 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874382 2587 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 11:58:49.022275 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874385 2587 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 11:58:49.022275 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874387 2587 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 11:58:49.022275 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874390 2587 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 11:58:49.022275 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874392 2587 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 11:58:49.022275 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874394 2587 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 11:58:49.022275 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874397 2587 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 11:58:49.022275 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874399 2587 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 11:58:49.022964 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874409 2587 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 11:58:49.022964 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874412 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 11:58:49.022964 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874414 2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 11:58:49.022964 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874417 2587 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 11:58:49.022964 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874419 2587 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 11:58:49.022964 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874422 2587 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 11:58:49.022964 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874424 2587 feature_gate.go:328] unrecognized feature gate: Example Jul 15 11:58:49.022964 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874428 2587 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 11:58:49.022964 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874430 2587 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 11:58:49.022964 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874433 2587 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 11:58:49.022964 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874435 2587 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 11:58:49.022964 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874438 2587 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 11:58:49.022964 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874440 2587 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 11:58:49.022964 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874444 2587 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 11:58:49.022964 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874448 2587 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 11:58:49.022964 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874452 2587 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 11:58:49.022964 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874454 2587 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 11:58:49.022964 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874457 2587 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 11:58:49.022964 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874460 2587 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 11:58:49.023518 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874463 2587 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 11:58:49.023518 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874466 2587 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 11:58:49.023518 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874468 2587 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 11:58:49.023518 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874471 2587 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 11:58:49.023518 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874473 2587 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 11:58:49.023518 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874476 2587 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 11:58:49.023518 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874479 2587 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 11:58:49.023518 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874481 2587 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 11:58:49.023518 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874483 2587 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 11:58:49.023518 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874486 2587 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 11:58:49.023518 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874488 2587 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 11:58:49.023518 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874491 2587 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 11:58:49.023518 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874493 2587 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 11:58:49.023518 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874496 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 11:58:49.023518 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874504 2587 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 11:58:49.023518 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874507 2587 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 11:58:49.023518 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874509 2587 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 11:58:49.023518 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874512 2587 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 11:58:49.023518 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874514 2587 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 11:58:49.024048 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874517 2587 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 11:58:49.024048 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874520 2587 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 11:58:49.024048 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874523 2587 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 11:58:49.024048 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874525 2587 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 11:58:49.024048 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874528 2587 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 11:58:49.024048 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874530 2587 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 11:58:49.024048 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874533 2587 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 11:58:49.024048 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874535 2587 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 11:58:49.024048 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874539 2587 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 11:58:49.024048 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874542 2587 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 11:58:49.024048 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874544 2587 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 11:58:49.024048 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874548 2587 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 11:58:49.024048 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874552 2587 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 11:58:49.024048 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874555 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 11:58:49.024048 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874558 2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 11:58:49.024048 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874560 2587 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 11:58:49.024048 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:48.874562 2587 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 11:58:49.024532 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.874567 2587 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 11:58:49.024532 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.874684 2587 server.go:962] "Client rotation is on, will bootstrap in background" Jul 15 11:58:49.024532 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.890617 2587 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 15 11:58:49.024532 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.892005 2587 server.go:1019] "Starting client certificate rotation" Jul 15 11:58:49.024532 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.892142 2587 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 15 11:58:49.024532 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.892171 2587 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 15 11:58:49.024532 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.896261 2587 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 15 11:58:49.024532 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.897627 2587 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 15 11:58:49.024532 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.913322 2587 log.go:25] "Validated CRI v1 runtime API" Jul 15 11:58:49.024532 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.915364 2587 log.go:25] "Validated CRI v1 image API" Jul 15 11:58:49.024532 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.921321 2587 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 15 11:58:49.024532 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.923376 2587 fs.go:135] Filesystem UUIDs: map[602d3bf1-f77a-4332-9b17-f38623ee7f5f:/dev/nvme0n1p4 76c81e8f-c30e-4f68-8d68-ad49d3f51b73:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jul 15 11:58:49.024911 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.923394 2587 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 15 11:58:49.024911 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.927429 2587 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 15 11:58:49.107665 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.927950 2587 manager.go:216] Machine: {Timestamp:2026-07-15 11:58:48.927473201 +0000 UTC m=+0.384850139 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3108852 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec22492f6d61aeb32f07fbf5b7b74b4e SystemUUID:ec22492f-6d61-aeb3-2f07-fbf5b7b74b4e BootID:fb5bc21a-554a-4ae5-a80d-5009987f4f4a Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:33:76:07:18:0f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:33:76:07:18:0f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ba:0f:79:df:50:e8 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 15 11:58:49.107665 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.928047 2587 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 15 11:58:49.107665 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.928118 2587 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 15 11:58:49.107665 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.930892 2587 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 15 11:58:49.107665 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.930921 2587 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-243.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 15 11:58:49.107665 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.931594 2587 topology_manager.go:138] "Creating topology manager with none policy" Jul 15 11:58:49.107665 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.931607 2587 container_manager_linux.go:309] "Creating device plugin manager" Jul 15 11:58:49.107665 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.931617 2587 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 15 11:58:49.107665 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.932270 2587 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 15 11:58:49.107665 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.933145 2587 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 15 11:58:49.107665 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.941526 2587 state_mem.go:36] "Initialized new in-memory state store" Jul 15 11:58:49.107665 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.941655 2587 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 15 11:58:49.107665 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.941686 2587 kubelet.go:486] "Attempting to sync node with API server" Jul 15 11:58:49.107665 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.941695 2587 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 15 11:58:49.107665 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.941712 2587 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 15 11:58:49.107665 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.941721 2587 kubelet.go:398] "Adding apiserver pod source" Jul 15 11:58:49.107665 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.941744 2587 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 15 11:58:49.107665 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.942142 2587 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 15 11:58:49.108779 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.942162 2587 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 15 11:58:49.108779 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.944056 2587 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 15 11:58:49.108779 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.946478 2587 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 15 11:58:49.108779 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.946508 2587 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 15 11:58:49.108779 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.947114 2587 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 15 11:58:49.108779 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.947127 2587 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 15 11:58:49.108779 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.947133 2587 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 15 11:58:49.108779 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.947138 2587 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 15 11:58:49.108779 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.947147 2587 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 15 11:58:49.108779 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.947152 2587 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 15 11:58:49.108779 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.947160 2587 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 15 11:58:49.108779 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.947165 2587 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 15 11:58:49.108779 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.947171 2587 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 15 11:58:49.108779 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.947176 2587 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 15 11:58:49.108779 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.947185 2587 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 15 11:58:49.108779 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.947192 2587 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 15 11:58:49.108779 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.962777 2587 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 15 11:58:49.108779 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.962815 2587 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 15 11:58:49.108779 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.965115 2587 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-243.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 15 11:58:49.108779 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:48.965142 2587 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 15 11:58:49.108779 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:48.965150 2587 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-243.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 15 11:58:49.108779 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.965864 2587 server.go:1268] "Started kubelet" Jul 15 11:58:49.108779 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.966208 2587 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 15 11:58:49.108779 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.967914 2587 server.go:310] "Adding debug handlers to kubelet server" Jul 15 11:58:49.108779 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.968866 2587 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 15 11:58:49.108779 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.968907 2587 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 15 11:58:49.108779 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.979499 2587 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 15 11:58:49.108779 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.988986 2587 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-t876n" Jul 15 11:58:49.108779 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.989006 2587 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-t876n" Jul 15 11:58:49.108779 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:48.990915 2587 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-243.ec2.internal.18c274b65bd74af2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-243.ec2.internal,UID:ip-10-0-135-243.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-243.ec2.internal,},FirstTimestamp:2026-07-15 11:58:48.965843698 +0000 UTC m=+0.423220613,LastTimestamp:2026-07-15 11:58:48.965843698 +0000 UTC m=+0.423220613,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-243.ec2.internal,}" Jul 15 11:58:49.108779 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:48.992132 2587 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 15 11:58:49.108779 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.992959 2587 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 15 11:58:49.108779 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.992968 2587 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 15 11:58:49.108779 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:48.993023 2587 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-243.ec2.internal\" not found" Jul 15 11:58:49.108779 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.993504 2587 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 15 11:58:49.108779 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.993517 2587 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 15 11:58:49.108779 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.993505 2587 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 15 11:58:49.108779 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.993644 2587 reconstruct.go:97] "Volume reconstruction finished" Jul 15 11:58:49.108779 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.993653 2587 reconciler.go:29] "Reconciler: start to sync state" Jul 15 11:58:49.108779 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.993834 2587 factory.go:55] Registering systemd factory Jul 15 11:58:49.110143 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.993851 2587 factory.go:223] Registration of the systemd container factory successfully Jul 15 11:58:49.110143 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.994009 2587 factory.go:153] Registering CRI-O factory Jul 15 11:58:49.110143 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.994022 2587 factory.go:223] Registration of the crio container factory successfully Jul 15 11:58:49.110143 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.994551 2587 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 15 11:58:49.110143 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.994573 2587 factory.go:103] Registering Raw factory Jul 15 11:58:49.110143 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.994583 2587 manager.go:1196] Started watching for new ooms in manager Jul 15 11:58:49.110143 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:48.994937 2587 manager.go:349] Starting recovery of all containers Jul 15 11:58:49.110143 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.001918 2587 manager.go:354] Recovery completed Jul 15 11:58:49.110143 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:49.010901 2587 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 15 11:58:49.110143 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.013814 2587 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 11:58:49.110143 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:49.017350 2587 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-135-243.ec2.internal\" not found" node="ip-10-0-135-243.ec2.internal" Jul 15 11:58:49.110143 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.017493 2587 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 11:58:49.110143 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.019714 2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-243.ec2.internal" event="NodeHasSufficientMemory" Jul 15 11:58:49.110143 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.019765 2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-243.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 11:58:49.110143 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.019780 2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-243.ec2.internal" event="NodeHasSufficientPID" Jul 15 11:58:49.110143 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.020442 2587 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 15 11:58:49.110143 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.020450 2587 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 15 11:58:49.110143 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.020466 2587 state_mem.go:36] "Initialized new in-memory state store" Jul 15 11:58:49.110143 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.020768 2587 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 11:58:49.110143 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.022442 2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-243.ec2.internal" event="NodeHasSufficientMemory" Jul 15 11:58:49.110143 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.022469 2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-243.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 11:58:49.110143 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.022478 2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-243.ec2.internal" event="NodeHasSufficientPID" Jul 15 11:58:49.110143 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.022591 2587 policy_none.go:49] "None policy: Start" Jul 15 11:58:49.110143 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.022604 2587 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 15 11:58:49.110143 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.022614 2587 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 15 11:58:49.110143 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.024802 2587 policy_none.go:47] "Start" Jul 15 11:58:49.110143 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.061451 2587 manager.go:337] "Starting Device Plugin manager" Jul 15 11:58:49.110143 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:49.061594 2587 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 15 11:58:49.110143 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.061605 2587 server.go:87] "Starting device plugin registration server" Jul 15 11:58:49.110143 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.061831 2587 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 15 11:58:49.110143 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.061846 2587 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 15 11:58:49.110143 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.064548 2587 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 15 11:58:49.110143 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.064631 2587 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 15 11:58:49.110143 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.064641 2587 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 15 11:58:49.110143 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:49.065344 2587 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 15 11:58:49.110143 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:49.065402 2587 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-243.ec2.internal\" not found" Jul 15 11:58:49.252052 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.162097 2587 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 11:58:49.252052 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.163272 2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-243.ec2.internal" event="NodeHasSufficientMemory" Jul 15 11:58:49.252052 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.163305 2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-243.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 11:58:49.252052 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.163317 2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-243.ec2.internal" event="NodeHasSufficientPID" Jul 15 11:58:49.252052 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.163361 2587 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-243.ec2.internal" Jul 15 11:58:49.252052 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.174238 2587 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-243.ec2.internal" Jul 15 11:58:49.252052 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:49.174262 2587 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-243.ec2.internal\": node \"ip-10-0-135-243.ec2.internal\" not found" Jul 15 11:58:49.252052 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.176805 2587 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 15 11:58:49.252052 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.178342 2587 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 15 11:58:49.252052 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.178373 2587 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 15 11:58:49.252052 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.178394 2587 kubelet.go:2448] "Starting kubelet main sync loop" Jul 15 11:58:49.252052 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:49.178426 2587 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 15 11:58:49.252052 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.181739 2587 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 11:58:49.252052 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:49.213701 2587 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-243.ec2.internal\" not found" Jul 15 11:58:49.421708 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.280136 2587 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-135-243.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-243.ec2.internal"] Jul 15 11:58:49.421708 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.280215 2587 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 11:58:49.421708 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.281278 2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-243.ec2.internal" event="NodeHasSufficientMemory" Jul 15 11:58:49.421708 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.281307 2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-243.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 11:58:49.421708 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.281317 2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-243.ec2.internal" event="NodeHasSufficientPID" Jul 15 11:58:49.421708 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.284301 2587 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 11:58:49.421708 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.284437 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-243.ec2.internal" Jul 15 11:58:49.421708 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.284468 2587 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 11:58:49.421708 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.285131 2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-243.ec2.internal" event="NodeHasSufficientMemory" Jul 15 11:58:49.421708 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.285159 2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-243.ec2.internal" event="NodeHasSufficientMemory" Jul 15 11:58:49.421708 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.285181 2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-243.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 11:58:49.421708 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.285196 2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-243.ec2.internal" event="NodeHasSufficientPID" Jul 15 11:58:49.421708 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.285163 2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-243.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 11:58:49.421708 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.285250 2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-243.ec2.internal" event="NodeHasSufficientPID" Jul 15 11:58:49.421708 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.290195 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-243.ec2.internal" Jul 15 11:58:49.421708 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.290219 2587 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 11:58:49.421708 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.291106 2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-243.ec2.internal" event="NodeHasSufficientMemory" Jul 15 11:58:49.421708 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.291136 2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-243.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 11:58:49.421708 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.291151 2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-243.ec2.internal" event="NodeHasSufficientPID" Jul 15 11:58:49.421708 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.295897 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c1bd23d6ee74dd41f0a99e2cf1119280-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-243.ec2.internal\" (UID: \"c1bd23d6ee74dd41f0a99e2cf1119280\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-243.ec2.internal" Jul 15 11:58:49.421708 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.295921 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/31cf20077539c85a825a4f13af6feab9-config\") pod \"kube-apiserver-proxy-ip-10-0-135-243.ec2.internal\" (UID: \"31cf20077539c85a825a4f13af6feab9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-243.ec2.internal" Jul 15 11:58:49.421708 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.295941 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c1bd23d6ee74dd41f0a99e2cf1119280-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-243.ec2.internal\" (UID: \"c1bd23d6ee74dd41f0a99e2cf1119280\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-243.ec2.internal" Jul 15 11:58:49.421708 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:49.304407 2587 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-243.ec2.internal\" not found" node="ip-10-0-135-243.ec2.internal" Jul 15 11:58:49.421708 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:49.310413 2587 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-243.ec2.internal\" not found" node="ip-10-0-135-243.ec2.internal" Jul 15 11:58:49.421708 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:49.314345 2587 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-243.ec2.internal\" not found" Jul 15 11:58:49.421708 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.397147 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/31cf20077539c85a825a4f13af6feab9-config\") pod \"kube-apiserver-proxy-ip-10-0-135-243.ec2.internal\" (UID: \"31cf20077539c85a825a4f13af6feab9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-243.ec2.internal" Jul 15 11:58:49.421708 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.397184 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c1bd23d6ee74dd41f0a99e2cf1119280-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-243.ec2.internal\" (UID: \"c1bd23d6ee74dd41f0a99e2cf1119280\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-243.ec2.internal" Jul 15 11:58:49.421708 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.397205 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c1bd23d6ee74dd41f0a99e2cf1119280-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-243.ec2.internal\" (UID: \"c1bd23d6ee74dd41f0a99e2cf1119280\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-243.ec2.internal" Jul 15 11:58:49.422718 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.397258 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c1bd23d6ee74dd41f0a99e2cf1119280-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-243.ec2.internal\" (UID: \"c1bd23d6ee74dd41f0a99e2cf1119280\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-243.ec2.internal" Jul 15 11:58:49.422718 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.397258 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/31cf20077539c85a825a4f13af6feab9-config\") pod \"kube-apiserver-proxy-ip-10-0-135-243.ec2.internal\" (UID: \"31cf20077539c85a825a4f13af6feab9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-243.ec2.internal" Jul 15 11:58:49.422718 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.397263 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c1bd23d6ee74dd41f0a99e2cf1119280-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-243.ec2.internal\" (UID: \"c1bd23d6ee74dd41f0a99e2cf1119280\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-243.ec2.internal" Jul 15 11:58:49.422718 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:49.415604 2587 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-243.ec2.internal\" not found" Jul 15 11:58:49.589584 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:49.516711 2587 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-243.ec2.internal\" not found" Jul 15 11:58:49.685421 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.607342 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-243.ec2.internal" Jul 15 11:58:49.685421 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.613717 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-243.ec2.internal" Jul 15 11:58:49.685421 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:49.617854 2587 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-243.ec2.internal\" not found" Jul 15 11:58:49.823494 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:49.718984 2587 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-243.ec2.internal\" not found" Jul 15 11:58:49.823494 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:49.819716 2587 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-243.ec2.internal\" not found" Jul 15 11:58:49.948562 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.891129 2587 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 15 11:58:49.948562 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.891285 2587 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 15 11:58:49.948562 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.891316 2587 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 15 11:58:49.948562 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:49.920944 2587 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-243.ec2.internal\" not found" Jul 15 11:58:50.096163 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.994994 2587 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-14 11:53:49 +0000 UTC" deadline="2028-01-31 00:53:31.806584895 +0000 UTC" Jul 15 11:58:50.096163 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:49.995026 2587 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13548h54m41.811564341s" Jul 15 11:58:50.096163 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:50.022059 2587 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-243.ec2.internal\" not found" Jul 15 11:58:50.270905 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:50.123554 2587 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-243.ec2.internal\" not found" Jul 15 11:58:50.270905 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.151913 2587 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 11:58:50.270905 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:50.224517 2587 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-243.ec2.internal\" not found" Jul 15 11:58:50.399749 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:50.325527 2587 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-243.ec2.internal\" not found" Jul 15 11:58:50.531300 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:50.426267 2587 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-243.ec2.internal\" not found" Jul 15 11:58:50.531300 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.479561 2587 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 11:58:50.531300 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.494073 2587 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-243.ec2.internal" Jul 15 11:58:50.531300 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:50.504582 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31cf20077539c85a825a4f13af6feab9.slice/crio-f103301c3939fee33fa1e35d7fd8dff7b79fff7732fc3ce3bf6a46f3fc6ad427 WatchSource:0}: Error finding container f103301c3939fee33fa1e35d7fd8dff7b79fff7732fc3ce3bf6a46f3fc6ad427: Status 404 returned error can't find the container with id f103301c3939fee33fa1e35d7fd8dff7b79fff7732fc3ce3bf6a46f3fc6ad427 Jul 15 11:58:50.531300 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.504859 2587 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 15 11:58:50.531300 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.505690 2587 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-243.ec2.internal" Jul 15 11:58:50.531300 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.508401 2587 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 11:58:50.531300 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.519785 2587 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 15 11:58:50.665215 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:50.665185 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1bd23d6ee74dd41f0a99e2cf1119280.slice/crio-0446c33deab54a755c1fc262a53536231f9d7ac8207126fd4e9bf0579d52d0bd WatchSource:0}: Error finding container 0446c33deab54a755c1fc262a53536231f9d7ac8207126fd4e9bf0579d52d0bd: Status 404 returned error can't find the container with id 0446c33deab54a755c1fc262a53536231f9d7ac8207126fd4e9bf0579d52d0bd Jul 15 11:58:50.942919 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.942849 2587 apiserver.go:52] "Watching apiserver" Jul 15 11:58:50.953188 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.953164 2587 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 15 11:58:50.953699 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.953517 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-135-243.ec2.internal","openshift-image-registry/node-ca-5bb6n","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-243.ec2.internal","openshift-multus/multus-fgrpr","openshift-multus/network-metrics-daemon-xtlxm","openshift-network-diagnostics/network-check-target-qf4ll","openshift-network-operator/iptables-alerter-gr6dm","kube-system/konnectivity-agent-x9kt6","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x5ws6","openshift-cluster-node-tuning-operator/tuned-b58gj","openshift-dns/node-resolver-fnhqk","openshift-multus/multus-additional-cni-plugins-lznkd","openshift-ovn-kubernetes/ovnkube-node-mcrkx","kube-system/global-pull-secret-syncer-vn6hr"] Jul 15 11:58:50.956245 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.956225 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-gr6dm" Jul 15 11:58:50.958368 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.958347 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:50.959421 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.959397 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 15 11:58:50.959421 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.959411 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 15 11:58:50.959668 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.959653 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 15 11:58:50.961341 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.960062 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-qgpfq\"" Jul 15 11:58:50.961790 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.961540 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vn6hr" Jul 15 11:58:50.961790 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:50.961624 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vn6hr" podUID="c26f517e-49fe-4da5-a790-e8b77483317a" Jul 15 11:58:50.962712 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.962659 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 15 11:58:50.962712 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.962683 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 15 11:58:50.962712 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.962684 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-w2p9m\"" Jul 15 11:58:50.962919 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.962665 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 15 11:58:50.962919 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.962758 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 15 11:58:50.963041 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.963025 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 15 11:58:50.963107 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.963038 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 15 11:58:50.964430 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.964411 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fgrpr" Jul 15 11:58:50.966615 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.966599 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtlxm" Jul 15 11:58:50.966702 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:50.966664 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtlxm" podUID="e7e5aa88-1817-4a28-8add-88cf29795c94" Jul 15 11:58:50.967325 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.967302 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 15 11:58:50.967424 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.967387 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 15 11:58:50.967424 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.967408 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-59sdv\"" Jul 15 11:58:50.967567 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.967551 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 15 11:58:50.967631 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.967587 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 15 11:58:50.968804 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.968789 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qf4ll" Jul 15 11:58:50.968862 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:50.968835 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qf4ll" podUID="29500681-aa9a-4fd5-b440-487ae81b05c6" Jul 15 11:58:50.971024 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.971009 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x5ws6" Jul 15 11:58:50.973204 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.973187 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-x9kt6" Jul 15 11:58:50.973691 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.973673 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 15 11:58:50.973987 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.973971 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 15 11:58:50.974054 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.973998 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-rp8l6\"" Jul 15 11:58:50.974088 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.973971 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 15 11:58:50.975442 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.975423 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5bb6n" Jul 15 11:58:50.976101 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.976083 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 15 11:58:50.976327 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.976301 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-4gf2x\"" Jul 15 11:58:50.976888 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.976869 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 15 11:58:50.977820 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.977803 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-b58gj" Jul 15 11:58:50.978629 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.978604 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 15 11:58:50.978629 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.978620 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-fxxl9\"" Jul 15 11:58:50.978778 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.978622 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 15 11:58:50.978778 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.978641 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 15 11:58:50.980117 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.980102 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fnhqk" Jul 15 11:58:50.980851 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.980835 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 15 11:58:50.981041 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.981026 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 15 11:58:50.981092 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.981026 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-znv67\"" Jul 15 11:58:50.982902 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.982885 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 15 11:58:50.982995 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.982957 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-lznkd" Jul 15 11:58:50.983067 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.983041 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 15 11:58:50.983154 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.983139 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-t6llw\"" Jul 15 11:58:50.985556 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.985534 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 15 11:58:50.985693 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.985627 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-tl5r8\"" Jul 15 11:58:50.986093 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.986075 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 15 11:58:50.994455 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.994439 2587 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 15 11:58:50.994546 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:50.994505 2587 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 15 11:58:51.004491 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.004424 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d09ffd58-1f50-4daa-8ce4-0c40ad65306b-host\") pod \"node-ca-5bb6n\" (UID: \"d09ffd58-1f50-4daa-8ce4-0c40ad65306b\") " pod="openshift-image-registry/node-ca-5bb6n" Jul 15 11:58:51.004585 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.004514 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/52832a70-d8bb-466e-937a-956df4cec9a0-cnibin\") pod \"multus-fgrpr\" (UID: \"52832a70-d8bb-466e-937a-956df4cec9a0\") " pod="openshift-multus/multus-fgrpr" Jul 15 11:58:51.004585 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.004549 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ea415947-4742-4cf7-8b71-e22b418a131f-ovnkube-config\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.004688 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.004621 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ea415947-4742-4cf7-8b71-e22b418a131f-ovn-node-metrics-cert\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.004688 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.004656 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/52832a70-d8bb-466e-937a-956df4cec9a0-hostroot\") pod \"multus-fgrpr\" (UID: \"52832a70-d8bb-466e-937a-956df4cec9a0\") " pod="openshift-multus/multus-fgrpr" Jul 15 11:58:51.004814 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.004796 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ea415947-4742-4cf7-8b71-e22b418a131f-host-run-netns\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.004865 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.004827 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ea415947-4742-4cf7-8b71-e22b418a131f-run-ovn\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.004964 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.004933 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ea415947-4742-4cf7-8b71-e22b418a131f-systemd-units\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.005140 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.005117 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ea415947-4742-4cf7-8b71-e22b418a131f-ovnkube-script-lib\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.005254 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.005238 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/52832a70-d8bb-466e-937a-956df4cec9a0-system-cni-dir\") pod \"multus-fgrpr\" (UID: \"52832a70-d8bb-466e-937a-956df4cec9a0\") " pod="openshift-multus/multus-fgrpr" Jul 15 11:58:51.005363 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.005349 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/52832a70-d8bb-466e-937a-956df4cec9a0-host-run-multus-certs\") pod \"multus-fgrpr\" (UID: \"52832a70-d8bb-466e-937a-956df4cec9a0\") " pod="openshift-multus/multus-fgrpr" Jul 15 11:58:51.005455 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.005443 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjwbn\" (UniqueName: \"kubernetes.io/projected/d09ffd58-1f50-4daa-8ce4-0c40ad65306b-kube-api-access-gjwbn\") pod \"node-ca-5bb6n\" (UID: \"d09ffd58-1f50-4daa-8ce4-0c40ad65306b\") " pod="openshift-image-registry/node-ca-5bb6n" Jul 15 11:58:51.005557 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.005543 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b0228db4-1648-4484-8e9c-023c68c1e1e5-hosts-file\") pod \"node-resolver-fnhqk\" (UID: \"b0228db4-1648-4484-8e9c-023c68c1e1e5\") " pod="openshift-dns/node-resolver-fnhqk" Jul 15 11:58:51.005669 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.005640 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9646j\" (UniqueName: \"kubernetes.io/projected/b0228db4-1648-4484-8e9c-023c68c1e1e5-kube-api-access-9646j\") pod \"node-resolver-fnhqk\" (UID: \"b0228db4-1648-4484-8e9c-023c68c1e1e5\") " pod="openshift-dns/node-resolver-fnhqk" Jul 15 11:58:51.005774 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.005673 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ea415947-4742-4cf7-8b71-e22b418a131f-host-slash\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.005774 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.005704 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ea415947-4742-4cf7-8b71-e22b418a131f-host-cni-netd\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.005774 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.005749 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/73bb7388-9486-4aa7-beff-f9d6b48e61be-var-lib-kubelet\") pod \"tuned-b58gj\" (UID: \"73bb7388-9486-4aa7-beff-f9d6b48e61be\") " pod="openshift-cluster-node-tuning-operator/tuned-b58gj" Jul 15 11:58:51.005934 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.005778 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/73bb7388-9486-4aa7-beff-f9d6b48e61be-tmp\") pod \"tuned-b58gj\" (UID: \"73bb7388-9486-4aa7-beff-f9d6b48e61be\") " pod="openshift-cluster-node-tuning-operator/tuned-b58gj" Jul 15 11:58:51.005934 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.005812 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mcgk\" (UniqueName: \"kubernetes.io/projected/73bb7388-9486-4aa7-beff-f9d6b48e61be-kube-api-access-8mcgk\") pod \"tuned-b58gj\" (UID: \"73bb7388-9486-4aa7-beff-f9d6b48e61be\") " pod="openshift-cluster-node-tuning-operator/tuned-b58gj" Jul 15 11:58:51.005934 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.005839 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4zct\" (UniqueName: \"kubernetes.io/projected/16a7a1c8-5e68-45e2-bb17-78c299bd91a7-kube-api-access-z4zct\") pod \"iptables-alerter-gr6dm\" (UID: \"16a7a1c8-5e68-45e2-bb17-78c299bd91a7\") " pod="openshift-network-operator/iptables-alerter-gr6dm" Jul 15 11:58:51.006080 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.005970 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f76becda-10bd-4105-a1ee-41422f70c74b-socket-dir\") pod \"aws-ebs-csi-driver-node-x5ws6\" (UID: \"f76becda-10bd-4105-a1ee-41422f70c74b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x5ws6" Jul 15 11:58:51.006080 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.006033 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ea415947-4742-4cf7-8b71-e22b418a131f-etc-openvswitch\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.006080 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.006061 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ea415947-4742-4cf7-8b71-e22b418a131f-run-openvswitch\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.006214 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.006101 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ea415947-4742-4cf7-8b71-e22b418a131f-log-socket\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.006214 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.006126 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lznkd\" (UID: \"ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a\") " pod="openshift-multus/multus-additional-cni-plugins-lznkd" Jul 15 11:58:51.006214 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.006157 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f76becda-10bd-4105-a1ee-41422f70c74b-etc-selinux\") pod \"aws-ebs-csi-driver-node-x5ws6\" (UID: \"f76becda-10bd-4105-a1ee-41422f70c74b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x5ws6" Jul 15 11:58:51.006214 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.006186 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d09ffd58-1f50-4daa-8ce4-0c40ad65306b-serviceca\") pod \"node-ca-5bb6n\" (UID: \"d09ffd58-1f50-4daa-8ce4-0c40ad65306b\") " pod="openshift-image-registry/node-ca-5bb6n" Jul 15 11:58:51.006400 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.006229 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/52832a70-d8bb-466e-937a-956df4cec9a0-host-var-lib-cni-bin\") pod \"multus-fgrpr\" (UID: \"52832a70-d8bb-466e-937a-956df4cec9a0\") " pod="openshift-multus/multus-fgrpr" Jul 15 11:58:51.006400 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.006269 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/52832a70-d8bb-466e-937a-956df4cec9a0-multus-socket-dir-parent\") pod \"multus-fgrpr\" (UID: \"52832a70-d8bb-466e-937a-956df4cec9a0\") " pod="openshift-multus/multus-fgrpr" Jul 15 11:58:51.006400 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.006332 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/52832a70-d8bb-466e-937a-956df4cec9a0-host-run-k8s-cni-cncf-io\") pod \"multus-fgrpr\" (UID: \"52832a70-d8bb-466e-937a-956df4cec9a0\") " pod="openshift-multus/multus-fgrpr" Jul 15 11:58:51.006400 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.006376 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/73bb7388-9486-4aa7-beff-f9d6b48e61be-sys\") pod \"tuned-b58gj\" (UID: \"73bb7388-9486-4aa7-beff-f9d6b48e61be\") " pod="openshift-cluster-node-tuning-operator/tuned-b58gj" Jul 15 11:58:51.006588 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.006420 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7e5aa88-1817-4a28-8add-88cf29795c94-metrics-certs\") pod \"network-metrics-daemon-xtlxm\" (UID: \"e7e5aa88-1817-4a28-8add-88cf29795c94\") " pod="openshift-multus/network-metrics-daemon-xtlxm" Jul 15 11:58:51.006588 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.006458 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/73bb7388-9486-4aa7-beff-f9d6b48e61be-etc-sysctl-conf\") pod \"tuned-b58gj\" (UID: \"73bb7388-9486-4aa7-beff-f9d6b48e61be\") " pod="openshift-cluster-node-tuning-operator/tuned-b58gj" Jul 15 11:58:51.006588 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.006498 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/73bb7388-9486-4aa7-beff-f9d6b48e61be-lib-modules\") pod \"tuned-b58gj\" (UID: \"73bb7388-9486-4aa7-beff-f9d6b48e61be\") " pod="openshift-cluster-node-tuning-operator/tuned-b58gj" Jul 15 11:58:51.006588 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.006533 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a-cnibin\") pod \"multus-additional-cni-plugins-lznkd\" (UID: \"ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a\") " pod="openshift-multus/multus-additional-cni-plugins-lznkd" Jul 15 11:58:51.006588 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.006560 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a-cni-binary-copy\") pod \"multus-additional-cni-plugins-lznkd\" (UID: \"ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a\") " pod="openshift-multus/multus-additional-cni-plugins-lznkd" Jul 15 11:58:51.006844 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.006671 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/16a7a1c8-5e68-45e2-bb17-78c299bd91a7-iptables-alerter-script\") pod \"iptables-alerter-gr6dm\" (UID: \"16a7a1c8-5e68-45e2-bb17-78c299bd91a7\") " pod="openshift-network-operator/iptables-alerter-gr6dm" Jul 15 11:58:51.006844 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.006723 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/16a7a1c8-5e68-45e2-bb17-78c299bd91a7-host-slash\") pod \"iptables-alerter-gr6dm\" (UID: \"16a7a1c8-5e68-45e2-bb17-78c299bd91a7\") " pod="openshift-network-operator/iptables-alerter-gr6dm" Jul 15 11:58:51.006844 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.006783 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/52832a70-d8bb-466e-937a-956df4cec9a0-multus-cni-dir\") pod \"multus-fgrpr\" (UID: \"52832a70-d8bb-466e-937a-956df4cec9a0\") " pod="openshift-multus/multus-fgrpr" Jul 15 11:58:51.006844 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.006815 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ea415947-4742-4cf7-8b71-e22b418a131f-host-run-ovn-kubernetes\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.007018 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.006872 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ea415947-4742-4cf7-8b71-e22b418a131f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.007018 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.006903 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/73bb7388-9486-4aa7-beff-f9d6b48e61be-etc-modprobe-d\") pod \"tuned-b58gj\" (UID: \"73bb7388-9486-4aa7-beff-f9d6b48e61be\") " pod="openshift-cluster-node-tuning-operator/tuned-b58gj" Jul 15 11:58:51.007018 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.006942 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/73bb7388-9486-4aa7-beff-f9d6b48e61be-etc-kubernetes\") pod \"tuned-b58gj\" (UID: \"73bb7388-9486-4aa7-beff-f9d6b48e61be\") " pod="openshift-cluster-node-tuning-operator/tuned-b58gj" Jul 15 11:58:51.007018 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.006968 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/52832a70-d8bb-466e-937a-956df4cec9a0-cni-binary-copy\") pod \"multus-fgrpr\" (UID: \"52832a70-d8bb-466e-937a-956df4cec9a0\") " pod="openshift-multus/multus-fgrpr" Jul 15 11:58:51.007200 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.007062 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knqkh\" (UniqueName: \"kubernetes.io/projected/52832a70-d8bb-466e-937a-956df4cec9a0-kube-api-access-knqkh\") pod \"multus-fgrpr\" (UID: \"52832a70-d8bb-466e-937a-956df4cec9a0\") " pod="openshift-multus/multus-fgrpr" Jul 15 11:58:51.007200 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.007103 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f76becda-10bd-4105-a1ee-41422f70c74b-device-dir\") pod \"aws-ebs-csi-driver-node-x5ws6\" (UID: \"f76becda-10bd-4105-a1ee-41422f70c74b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x5ws6" Jul 15 11:58:51.007200 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.007127 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f76becda-10bd-4105-a1ee-41422f70c74b-sys-fs\") pod \"aws-ebs-csi-driver-node-x5ws6\" (UID: \"f76becda-10bd-4105-a1ee-41422f70c74b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x5ws6" Jul 15 11:58:51.007200 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.007184 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ea415947-4742-4cf7-8b71-e22b418a131f-host-cni-bin\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.007377 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.007235 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/73bb7388-9486-4aa7-beff-f9d6b48e61be-etc-sysconfig\") pod \"tuned-b58gj\" (UID: \"73bb7388-9486-4aa7-beff-f9d6b48e61be\") " pod="openshift-cluster-node-tuning-operator/tuned-b58gj" Jul 15 11:58:51.007464 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.007445 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2nkp\" (UniqueName: \"kubernetes.io/projected/f76becda-10bd-4105-a1ee-41422f70c74b-kube-api-access-p2nkp\") pod \"aws-ebs-csi-driver-node-x5ws6\" (UID: \"f76becda-10bd-4105-a1ee-41422f70c74b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x5ws6" Jul 15 11:58:51.007563 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.007548 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/20ea3927-e5c6-47c0-9ff6-43b7cad247c0-agent-certs\") pod \"konnectivity-agent-x9kt6\" (UID: \"20ea3927-e5c6-47c0-9ff6-43b7cad247c0\") " pod="kube-system/konnectivity-agent-x9kt6" Jul 15 11:58:51.007654 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.007643 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ea415947-4742-4cf7-8b71-e22b418a131f-run-systemd\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.007770 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.007757 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ea415947-4742-4cf7-8b71-e22b418a131f-node-log\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.007881 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.007869 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ea415947-4742-4cf7-8b71-e22b418a131f-env-overrides\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.007974 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.007961 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c26f517e-49fe-4da5-a790-e8b77483317a-original-pull-secret\") pod \"global-pull-secret-syncer-vn6hr\" (UID: \"c26f517e-49fe-4da5-a790-e8b77483317a\") " pod="kube-system/global-pull-secret-syncer-vn6hr" Jul 15 11:58:51.008125 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.008105 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/73bb7388-9486-4aa7-beff-f9d6b48e61be-etc-sysctl-d\") pod \"tuned-b58gj\" (UID: \"73bb7388-9486-4aa7-beff-f9d6b48e61be\") " pod="openshift-cluster-node-tuning-operator/tuned-b58gj" Jul 15 11:58:51.008526 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.008504 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a-os-release\") pod \"multus-additional-cni-plugins-lznkd\" (UID: \"ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a\") " pod="openshift-multus/multus-additional-cni-plugins-lznkd" Jul 15 11:58:51.008618 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.008550 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75vl9\" (UniqueName: \"kubernetes.io/projected/29500681-aa9a-4fd5-b440-487ae81b05c6-kube-api-access-75vl9\") pod \"network-check-target-qf4ll\" (UID: \"29500681-aa9a-4fd5-b440-487ae81b05c6\") " pod="openshift-network-diagnostics/network-check-target-qf4ll" Jul 15 11:58:51.008618 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.008577 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f76becda-10bd-4105-a1ee-41422f70c74b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-x5ws6\" (UID: \"f76becda-10bd-4105-a1ee-41422f70c74b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x5ws6" Jul 15 11:58:51.008618 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.008599 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a-system-cni-dir\") pod \"multus-additional-cni-plugins-lznkd\" (UID: \"ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a\") " pod="openshift-multus/multus-additional-cni-plugins-lznkd" Jul 15 11:58:51.008804 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.008630 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfd5g\" (UniqueName: \"kubernetes.io/projected/ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a-kube-api-access-nfd5g\") pod \"multus-additional-cni-plugins-lznkd\" (UID: \"ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a\") " pod="openshift-multus/multus-additional-cni-plugins-lznkd" Jul 15 11:58:51.008804 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.008654 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/52832a70-d8bb-466e-937a-956df4cec9a0-os-release\") pod \"multus-fgrpr\" (UID: \"52832a70-d8bb-466e-937a-956df4cec9a0\") " pod="openshift-multus/multus-fgrpr" Jul 15 11:58:51.008804 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.008676 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/52832a70-d8bb-466e-937a-956df4cec9a0-host-run-netns\") pod \"multus-fgrpr\" (UID: \"52832a70-d8bb-466e-937a-956df4cec9a0\") " pod="openshift-multus/multus-fgrpr" Jul 15 11:58:51.008804 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.008698 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/52832a70-d8bb-466e-937a-956df4cec9a0-multus-daemon-config\") pod \"multus-fgrpr\" (UID: \"52832a70-d8bb-466e-937a-956df4cec9a0\") " pod="openshift-multus/multus-fgrpr" Jul 15 11:58:51.008804 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.008720 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/20ea3927-e5c6-47c0-9ff6-43b7cad247c0-konnectivity-ca\") pod \"konnectivity-agent-x9kt6\" (UID: \"20ea3927-e5c6-47c0-9ff6-43b7cad247c0\") " pod="kube-system/konnectivity-agent-x9kt6" Jul 15 11:58:51.008804 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.008770 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c26f517e-49fe-4da5-a790-e8b77483317a-dbus\") pod \"global-pull-secret-syncer-vn6hr\" (UID: \"c26f517e-49fe-4da5-a790-e8b77483317a\") " pod="kube-system/global-pull-secret-syncer-vn6hr" Jul 15 11:58:51.008804 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.008803 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cj9bd\" (UniqueName: \"kubernetes.io/projected/e7e5aa88-1817-4a28-8add-88cf29795c94-kube-api-access-cj9bd\") pod \"network-metrics-daemon-xtlxm\" (UID: \"e7e5aa88-1817-4a28-8add-88cf29795c94\") " pod="openshift-multus/network-metrics-daemon-xtlxm" Jul 15 11:58:51.009212 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.008852 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ea415947-4742-4cf7-8b71-e22b418a131f-host-kubelet\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.009212 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.008881 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c26f517e-49fe-4da5-a790-e8b77483317a-kubelet-config\") pod \"global-pull-secret-syncer-vn6hr\" (UID: \"c26f517e-49fe-4da5-a790-e8b77483317a\") " pod="kube-system/global-pull-secret-syncer-vn6hr" Jul 15 11:58:51.009212 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.008905 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/73bb7388-9486-4aa7-beff-f9d6b48e61be-etc-systemd\") pod \"tuned-b58gj\" (UID: \"73bb7388-9486-4aa7-beff-f9d6b48e61be\") " pod="openshift-cluster-node-tuning-operator/tuned-b58gj" Jul 15 11:58:51.009212 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.008926 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/73bb7388-9486-4aa7-beff-f9d6b48e61be-run\") pod \"tuned-b58gj\" (UID: \"73bb7388-9486-4aa7-beff-f9d6b48e61be\") " pod="openshift-cluster-node-tuning-operator/tuned-b58gj" Jul 15 11:58:51.009212 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.008949 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ea415947-4742-4cf7-8b71-e22b418a131f-var-lib-openvswitch\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.009212 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.008971 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ml5zz\" (UniqueName: \"kubernetes.io/projected/ea415947-4742-4cf7-8b71-e22b418a131f-kube-api-access-ml5zz\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.009212 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.008993 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/73bb7388-9486-4aa7-beff-f9d6b48e61be-etc-tuned\") pod \"tuned-b58gj\" (UID: \"73bb7388-9486-4aa7-beff-f9d6b48e61be\") " pod="openshift-cluster-node-tuning-operator/tuned-b58gj" Jul 15 11:58:51.009212 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.009048 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lznkd\" (UID: \"ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a\") " pod="openshift-multus/multus-additional-cni-plugins-lznkd" Jul 15 11:58:51.009212 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.009083 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lznkd\" (UID: \"ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a\") " pod="openshift-multus/multus-additional-cni-plugins-lznkd" Jul 15 11:58:51.009212 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.009110 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/52832a70-d8bb-466e-937a-956df4cec9a0-multus-conf-dir\") pod \"multus-fgrpr\" (UID: \"52832a70-d8bb-466e-937a-956df4cec9a0\") " pod="openshift-multus/multus-fgrpr" Jul 15 11:58:51.009212 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.009133 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/52832a70-d8bb-466e-937a-956df4cec9a0-etc-kubernetes\") pod \"multus-fgrpr\" (UID: \"52832a70-d8bb-466e-937a-956df4cec9a0\") " pod="openshift-multus/multus-fgrpr" Jul 15 11:58:51.009212 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.009156 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0228db4-1648-4484-8e9c-023c68c1e1e5-tmp-dir\") pod \"node-resolver-fnhqk\" (UID: \"b0228db4-1648-4484-8e9c-023c68c1e1e5\") " pod="openshift-dns/node-resolver-fnhqk" Jul 15 11:58:51.009212 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.009177 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/73bb7388-9486-4aa7-beff-f9d6b48e61be-host\") pod \"tuned-b58gj\" (UID: \"73bb7388-9486-4aa7-beff-f9d6b48e61be\") " pod="openshift-cluster-node-tuning-operator/tuned-b58gj" Jul 15 11:58:51.009212 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.009199 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/52832a70-d8bb-466e-937a-956df4cec9a0-host-var-lib-cni-multus\") pod \"multus-fgrpr\" (UID: \"52832a70-d8bb-466e-937a-956df4cec9a0\") " pod="openshift-multus/multus-fgrpr" Jul 15 11:58:51.009910 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.009221 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/52832a70-d8bb-466e-937a-956df4cec9a0-host-var-lib-kubelet\") pod \"multus-fgrpr\" (UID: \"52832a70-d8bb-466e-937a-956df4cec9a0\") " pod="openshift-multus/multus-fgrpr" Jul 15 11:58:51.009910 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.009250 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f76becda-10bd-4105-a1ee-41422f70c74b-registration-dir\") pod \"aws-ebs-csi-driver-node-x5ws6\" (UID: \"f76becda-10bd-4105-a1ee-41422f70c74b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x5ws6" Jul 15 11:58:51.011086 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.011068 2587 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 15 11:58:51.020327 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.020306 2587 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 11:58:51.040593 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.040563 2587 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-9tzvf" Jul 15 11:58:51.050205 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.050182 2587 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-9tzvf" Jul 15 11:58:51.109567 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.109530 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7e5aa88-1817-4a28-8add-88cf29795c94-metrics-certs\") pod \"network-metrics-daemon-xtlxm\" (UID: \"e7e5aa88-1817-4a28-8add-88cf29795c94\") " pod="openshift-multus/network-metrics-daemon-xtlxm" Jul 15 11:58:51.109567 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.109577 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/73bb7388-9486-4aa7-beff-f9d6b48e61be-etc-sysctl-conf\") pod \"tuned-b58gj\" (UID: \"73bb7388-9486-4aa7-beff-f9d6b48e61be\") " pod="openshift-cluster-node-tuning-operator/tuned-b58gj" Jul 15 11:58:51.110757 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.109604 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/73bb7388-9486-4aa7-beff-f9d6b48e61be-lib-modules\") pod \"tuned-b58gj\" (UID: \"73bb7388-9486-4aa7-beff-f9d6b48e61be\") " pod="openshift-cluster-node-tuning-operator/tuned-b58gj" Jul 15 11:58:51.110757 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.109631 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a-cnibin\") pod \"multus-additional-cni-plugins-lznkd\" (UID: \"ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a\") " pod="openshift-multus/multus-additional-cni-plugins-lznkd" Jul 15 11:58:51.110757 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.109692 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a-cni-binary-copy\") pod \"multus-additional-cni-plugins-lznkd\" (UID: \"ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a\") " pod="openshift-multus/multus-additional-cni-plugins-lznkd" Jul 15 11:58:51.110757 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:51.109698 2587 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 11:58:51.110757 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.109719 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/16a7a1c8-5e68-45e2-bb17-78c299bd91a7-iptables-alerter-script\") pod \"iptables-alerter-gr6dm\" (UID: \"16a7a1c8-5e68-45e2-bb17-78c299bd91a7\") " pod="openshift-network-operator/iptables-alerter-gr6dm" Jul 15 11:58:51.110757 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.109782 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/16a7a1c8-5e68-45e2-bb17-78c299bd91a7-host-slash\") pod \"iptables-alerter-gr6dm\" (UID: \"16a7a1c8-5e68-45e2-bb17-78c299bd91a7\") " pod="openshift-network-operator/iptables-alerter-gr6dm" Jul 15 11:58:51.110757 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:51.109821 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7e5aa88-1817-4a28-8add-88cf29795c94-metrics-certs podName:e7e5aa88-1817-4a28-8add-88cf29795c94 nodeName:}" failed. No retries permitted until 2026-07-15 11:58:51.609779349 +0000 UTC m=+3.067156272 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e7e5aa88-1817-4a28-8add-88cf29795c94-metrics-certs") pod "network-metrics-daemon-xtlxm" (UID: "e7e5aa88-1817-4a28-8add-88cf29795c94") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 11:58:51.110757 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.109829 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/16a7a1c8-5e68-45e2-bb17-78c299bd91a7-host-slash\") pod \"iptables-alerter-gr6dm\" (UID: \"16a7a1c8-5e68-45e2-bb17-78c299bd91a7\") " pod="openshift-network-operator/iptables-alerter-gr6dm" Jul 15 11:58:51.110757 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.109840 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/73bb7388-9486-4aa7-beff-f9d6b48e61be-etc-sysctl-conf\") pod \"tuned-b58gj\" (UID: \"73bb7388-9486-4aa7-beff-f9d6b48e61be\") " pod="openshift-cluster-node-tuning-operator/tuned-b58gj" Jul 15 11:58:51.110757 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.109858 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/52832a70-d8bb-466e-937a-956df4cec9a0-multus-cni-dir\") pod \"multus-fgrpr\" (UID: \"52832a70-d8bb-466e-937a-956df4cec9a0\") " pod="openshift-multus/multus-fgrpr" Jul 15 11:58:51.110757 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.109865 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/73bb7388-9486-4aa7-beff-f9d6b48e61be-lib-modules\") pod \"tuned-b58gj\" (UID: \"73bb7388-9486-4aa7-beff-f9d6b48e61be\") " pod="openshift-cluster-node-tuning-operator/tuned-b58gj" Jul 15 11:58:51.110757 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.109889 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ea415947-4742-4cf7-8b71-e22b418a131f-host-run-ovn-kubernetes\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.110757 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.109929 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ea415947-4742-4cf7-8b71-e22b418a131f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.110757 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.109955 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/73bb7388-9486-4aa7-beff-f9d6b48e61be-etc-modprobe-d\") pod \"tuned-b58gj\" (UID: \"73bb7388-9486-4aa7-beff-f9d6b48e61be\") " pod="openshift-cluster-node-tuning-operator/tuned-b58gj" Jul 15 11:58:51.110757 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.109977 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/73bb7388-9486-4aa7-beff-f9d6b48e61be-etc-kubernetes\") pod \"tuned-b58gj\" (UID: \"73bb7388-9486-4aa7-beff-f9d6b48e61be\") " pod="openshift-cluster-node-tuning-operator/tuned-b58gj" Jul 15 11:58:51.110757 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.110018 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/52832a70-d8bb-466e-937a-956df4cec9a0-cni-binary-copy\") pod \"multus-fgrpr\" (UID: \"52832a70-d8bb-466e-937a-956df4cec9a0\") " pod="openshift-multus/multus-fgrpr" Jul 15 11:58:51.110757 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.110043 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knqkh\" (UniqueName: \"kubernetes.io/projected/52832a70-d8bb-466e-937a-956df4cec9a0-kube-api-access-knqkh\") pod \"multus-fgrpr\" (UID: \"52832a70-d8bb-466e-937a-956df4cec9a0\") " pod="openshift-multus/multus-fgrpr" Jul 15 11:58:51.111656 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.110062 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/52832a70-d8bb-466e-937a-956df4cec9a0-multus-cni-dir\") pod \"multus-fgrpr\" (UID: \"52832a70-d8bb-466e-937a-956df4cec9a0\") " pod="openshift-multus/multus-fgrpr" Jul 15 11:58:51.111656 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.110067 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f76becda-10bd-4105-a1ee-41422f70c74b-device-dir\") pod \"aws-ebs-csi-driver-node-x5ws6\" (UID: \"f76becda-10bd-4105-a1ee-41422f70c74b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x5ws6" Jul 15 11:58:51.111656 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.110059 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ea415947-4742-4cf7-8b71-e22b418a131f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.111656 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.110098 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f76becda-10bd-4105-a1ee-41422f70c74b-sys-fs\") pod \"aws-ebs-csi-driver-node-x5ws6\" (UID: \"f76becda-10bd-4105-a1ee-41422f70c74b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x5ws6" Jul 15 11:58:51.111656 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.110121 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ea415947-4742-4cf7-8b71-e22b418a131f-host-cni-bin\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.111656 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.110139 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/73bb7388-9486-4aa7-beff-f9d6b48e61be-etc-kubernetes\") pod \"tuned-b58gj\" (UID: \"73bb7388-9486-4aa7-beff-f9d6b48e61be\") " pod="openshift-cluster-node-tuning-operator/tuned-b58gj" Jul 15 11:58:51.111656 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.110142 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/73bb7388-9486-4aa7-beff-f9d6b48e61be-etc-sysconfig\") pod \"tuned-b58gj\" (UID: \"73bb7388-9486-4aa7-beff-f9d6b48e61be\") " pod="openshift-cluster-node-tuning-operator/tuned-b58gj" Jul 15 11:58:51.111656 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.110194 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2nkp\" (UniqueName: \"kubernetes.io/projected/f76becda-10bd-4105-a1ee-41422f70c74b-kube-api-access-p2nkp\") pod \"aws-ebs-csi-driver-node-x5ws6\" (UID: \"f76becda-10bd-4105-a1ee-41422f70c74b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x5ws6" Jul 15 11:58:51.111656 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.110199 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/73bb7388-9486-4aa7-beff-f9d6b48e61be-etc-sysconfig\") pod \"tuned-b58gj\" (UID: \"73bb7388-9486-4aa7-beff-f9d6b48e61be\") " pod="openshift-cluster-node-tuning-operator/tuned-b58gj" Jul 15 11:58:51.111656 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.110146 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/73bb7388-9486-4aa7-beff-f9d6b48e61be-etc-modprobe-d\") pod \"tuned-b58gj\" (UID: \"73bb7388-9486-4aa7-beff-f9d6b48e61be\") " pod="openshift-cluster-node-tuning-operator/tuned-b58gj" Jul 15 11:58:51.111656 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.110224 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/20ea3927-e5c6-47c0-9ff6-43b7cad247c0-agent-certs\") pod \"konnectivity-agent-x9kt6\" (UID: \"20ea3927-e5c6-47c0-9ff6-43b7cad247c0\") " pod="kube-system/konnectivity-agent-x9kt6" Jul 15 11:58:51.111656 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.110250 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ea415947-4742-4cf7-8b71-e22b418a131f-run-systemd\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.111656 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.110282 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ea415947-4742-4cf7-8b71-e22b418a131f-node-log\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.111656 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.110307 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ea415947-4742-4cf7-8b71-e22b418a131f-env-overrides\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.111656 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.110345 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c26f517e-49fe-4da5-a790-e8b77483317a-original-pull-secret\") pod \"global-pull-secret-syncer-vn6hr\" (UID: \"c26f517e-49fe-4da5-a790-e8b77483317a\") " pod="kube-system/global-pull-secret-syncer-vn6hr" Jul 15 11:58:51.111656 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.110351 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a-cni-binary-copy\") pod \"multus-additional-cni-plugins-lznkd\" (UID: \"ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a\") " pod="openshift-multus/multus-additional-cni-plugins-lznkd" Jul 15 11:58:51.111656 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.110371 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/73bb7388-9486-4aa7-beff-f9d6b48e61be-etc-sysctl-d\") pod \"tuned-b58gj\" (UID: \"73bb7388-9486-4aa7-beff-f9d6b48e61be\") " pod="openshift-cluster-node-tuning-operator/tuned-b58gj" Jul 15 11:58:51.112444 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.110098 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ea415947-4742-4cf7-8b71-e22b418a131f-host-run-ovn-kubernetes\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.112444 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.109823 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a-cnibin\") pod \"multus-additional-cni-plugins-lznkd\" (UID: \"ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a\") " pod="openshift-multus/multus-additional-cni-plugins-lznkd" Jul 15 11:58:51.112444 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.110405 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f76becda-10bd-4105-a1ee-41422f70c74b-device-dir\") pod \"aws-ebs-csi-driver-node-x5ws6\" (UID: \"f76becda-10bd-4105-a1ee-41422f70c74b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x5ws6" Jul 15 11:58:51.112444 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.110440 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a-os-release\") pod \"multus-additional-cni-plugins-lznkd\" (UID: \"ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a\") " pod="openshift-multus/multus-additional-cni-plugins-lznkd" Jul 15 11:58:51.112444 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.110462 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75vl9\" (UniqueName: \"kubernetes.io/projected/29500681-aa9a-4fd5-b440-487ae81b05c6-kube-api-access-75vl9\") pod \"network-check-target-qf4ll\" (UID: \"29500681-aa9a-4fd5-b440-487ae81b05c6\") " pod="openshift-network-diagnostics/network-check-target-qf4ll" Jul 15 11:58:51.112444 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.110454 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ea415947-4742-4cf7-8b71-e22b418a131f-run-systemd\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.112444 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.110475 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/16a7a1c8-5e68-45e2-bb17-78c299bd91a7-iptables-alerter-script\") pod \"iptables-alerter-gr6dm\" (UID: \"16a7a1c8-5e68-45e2-bb17-78c299bd91a7\") " pod="openshift-network-operator/iptables-alerter-gr6dm" Jul 15 11:58:51.112444 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.110481 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f76becda-10bd-4105-a1ee-41422f70c74b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-x5ws6\" (UID: \"f76becda-10bd-4105-a1ee-41422f70c74b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x5ws6" Jul 15 11:58:51.112444 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.110505 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a-system-cni-dir\") pod \"multus-additional-cni-plugins-lznkd\" (UID: \"ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a\") " pod="openshift-multus/multus-additional-cni-plugins-lznkd" Jul 15 11:58:51.112444 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.110515 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ea415947-4742-4cf7-8b71-e22b418a131f-host-cni-bin\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.112444 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.110533 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfd5g\" (UniqueName: \"kubernetes.io/projected/ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a-kube-api-access-nfd5g\") pod \"multus-additional-cni-plugins-lznkd\" (UID: \"ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a\") " pod="openshift-multus/multus-additional-cni-plugins-lznkd" Jul 15 11:58:51.112444 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.110538 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f76becda-10bd-4105-a1ee-41422f70c74b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-x5ws6\" (UID: \"f76becda-10bd-4105-a1ee-41422f70c74b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x5ws6" Jul 15 11:58:51.112444 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.110557 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/52832a70-d8bb-466e-937a-956df4cec9a0-os-release\") pod \"multus-fgrpr\" (UID: \"52832a70-d8bb-466e-937a-956df4cec9a0\") " pod="openshift-multus/multus-fgrpr" Jul 15 11:58:51.112444 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.110567 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a-os-release\") pod \"multus-additional-cni-plugins-lznkd\" (UID: \"ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a\") " pod="openshift-multus/multus-additional-cni-plugins-lznkd" Jul 15 11:58:51.112444 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.110571 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/73bb7388-9486-4aa7-beff-f9d6b48e61be-etc-sysctl-d\") pod \"tuned-b58gj\" (UID: \"73bb7388-9486-4aa7-beff-f9d6b48e61be\") " pod="openshift-cluster-node-tuning-operator/tuned-b58gj" Jul 15 11:58:51.112444 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.110583 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/52832a70-d8bb-466e-937a-956df4cec9a0-host-run-netns\") pod \"multus-fgrpr\" (UID: \"52832a70-d8bb-466e-937a-956df4cec9a0\") " pod="openshift-multus/multus-fgrpr" Jul 15 11:58:51.112444 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.110595 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f76becda-10bd-4105-a1ee-41422f70c74b-sys-fs\") pod \"aws-ebs-csi-driver-node-x5ws6\" (UID: \"f76becda-10bd-4105-a1ee-41422f70c74b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x5ws6" Jul 15 11:58:51.113251 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.110612 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/52832a70-d8bb-466e-937a-956df4cec9a0-multus-daemon-config\") pod \"multus-fgrpr\" (UID: \"52832a70-d8bb-466e-937a-956df4cec9a0\") " pod="openshift-multus/multus-fgrpr" Jul 15 11:58:51.113251 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.110634 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/20ea3927-e5c6-47c0-9ff6-43b7cad247c0-konnectivity-ca\") pod \"konnectivity-agent-x9kt6\" (UID: \"20ea3927-e5c6-47c0-9ff6-43b7cad247c0\") " pod="kube-system/konnectivity-agent-x9kt6" Jul 15 11:58:51.113251 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.110641 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/52832a70-d8bb-466e-937a-956df4cec9a0-os-release\") pod \"multus-fgrpr\" (UID: \"52832a70-d8bb-466e-937a-956df4cec9a0\") " pod="openshift-multus/multus-fgrpr" Jul 15 11:58:51.113251 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.110655 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c26f517e-49fe-4da5-a790-e8b77483317a-dbus\") pod \"global-pull-secret-syncer-vn6hr\" (UID: \"c26f517e-49fe-4da5-a790-e8b77483317a\") " pod="kube-system/global-pull-secret-syncer-vn6hr" Jul 15 11:58:51.113251 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.110641 2587 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 15 11:58:51.113251 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.110682 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cj9bd\" (UniqueName: \"kubernetes.io/projected/e7e5aa88-1817-4a28-8add-88cf29795c94-kube-api-access-cj9bd\") pod \"network-metrics-daemon-xtlxm\" (UID: \"e7e5aa88-1817-4a28-8add-88cf29795c94\") " pod="openshift-multus/network-metrics-daemon-xtlxm" Jul 15 11:58:51.113251 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.110709 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ea415947-4742-4cf7-8b71-e22b418a131f-host-kubelet\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.113251 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.110749 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c26f517e-49fe-4da5-a790-e8b77483317a-kubelet-config\") pod \"global-pull-secret-syncer-vn6hr\" (UID: \"c26f517e-49fe-4da5-a790-e8b77483317a\") " pod="kube-system/global-pull-secret-syncer-vn6hr" Jul 15 11:58:51.113251 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.110761 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/52832a70-d8bb-466e-937a-956df4cec9a0-cni-binary-copy\") pod \"multus-fgrpr\" (UID: \"52832a70-d8bb-466e-937a-956df4cec9a0\") " pod="openshift-multus/multus-fgrpr" Jul 15 11:58:51.113251 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:51.110765 2587 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 11:58:51.113251 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.110775 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/73bb7388-9486-4aa7-beff-f9d6b48e61be-etc-systemd\") pod \"tuned-b58gj\" (UID: \"73bb7388-9486-4aa7-beff-f9d6b48e61be\") " pod="openshift-cluster-node-tuning-operator/tuned-b58gj" Jul 15 11:58:51.113251 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.110796 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/52832a70-d8bb-466e-937a-956df4cec9a0-host-run-netns\") pod \"multus-fgrpr\" (UID: \"52832a70-d8bb-466e-937a-956df4cec9a0\") " pod="openshift-multus/multus-fgrpr" Jul 15 11:58:51.113251 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:51.110835 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c26f517e-49fe-4da5-a790-e8b77483317a-original-pull-secret podName:c26f517e-49fe-4da5-a790-e8b77483317a nodeName:}" failed. No retries permitted until 2026-07-15 11:58:51.610819437 +0000 UTC m=+3.068196350 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c26f517e-49fe-4da5-a790-e8b77483317a-original-pull-secret") pod "global-pull-secret-syncer-vn6hr" (UID: "c26f517e-49fe-4da5-a790-e8b77483317a") : object "kube-system"/"original-pull-secret" not registered Jul 15 11:58:51.113251 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.110854 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ea415947-4742-4cf7-8b71-e22b418a131f-host-kubelet\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.113251 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.110916 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/73bb7388-9486-4aa7-beff-f9d6b48e61be-etc-systemd\") pod \"tuned-b58gj\" (UID: \"73bb7388-9486-4aa7-beff-f9d6b48e61be\") " pod="openshift-cluster-node-tuning-operator/tuned-b58gj" Jul 15 11:58:51.113251 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.110920 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c26f517e-49fe-4da5-a790-e8b77483317a-kubelet-config\") pod \"global-pull-secret-syncer-vn6hr\" (UID: \"c26f517e-49fe-4da5-a790-e8b77483317a\") " pod="kube-system/global-pull-secret-syncer-vn6hr" Jul 15 11:58:51.113251 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.110635 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ea415947-4742-4cf7-8b71-e22b418a131f-node-log\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.113251 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.110951 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/73bb7388-9486-4aa7-beff-f9d6b48e61be-run\") pod \"tuned-b58gj\" (UID: \"73bb7388-9486-4aa7-beff-f9d6b48e61be\") " pod="openshift-cluster-node-tuning-operator/tuned-b58gj" Jul 15 11:58:51.114191 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.110984 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c26f517e-49fe-4da5-a790-e8b77483317a-dbus\") pod \"global-pull-secret-syncer-vn6hr\" (UID: \"c26f517e-49fe-4da5-a790-e8b77483317a\") " pod="kube-system/global-pull-secret-syncer-vn6hr" Jul 15 11:58:51.114191 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.110981 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ea415947-4742-4cf7-8b71-e22b418a131f-var-lib-openvswitch\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.114191 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.111026 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ml5zz\" (UniqueName: \"kubernetes.io/projected/ea415947-4742-4cf7-8b71-e22b418a131f-kube-api-access-ml5zz\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.114191 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.111040 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/73bb7388-9486-4aa7-beff-f9d6b48e61be-run\") pod \"tuned-b58gj\" (UID: \"73bb7388-9486-4aa7-beff-f9d6b48e61be\") " pod="openshift-cluster-node-tuning-operator/tuned-b58gj" Jul 15 11:58:51.114191 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.111046 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/73bb7388-9486-4aa7-beff-f9d6b48e61be-etc-tuned\") pod \"tuned-b58gj\" (UID: \"73bb7388-9486-4aa7-beff-f9d6b48e61be\") " pod="openshift-cluster-node-tuning-operator/tuned-b58gj" Jul 15 11:58:51.114191 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.111079 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lznkd\" (UID: \"ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a\") " pod="openshift-multus/multus-additional-cni-plugins-lznkd" Jul 15 11:58:51.114191 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.111095 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ea415947-4742-4cf7-8b71-e22b418a131f-env-overrides\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.114191 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.111105 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lznkd\" (UID: \"ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a\") " pod="openshift-multus/multus-additional-cni-plugins-lznkd" Jul 15 11:58:51.114191 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.111121 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/52832a70-d8bb-466e-937a-956df4cec9a0-multus-daemon-config\") pod \"multus-fgrpr\" (UID: \"52832a70-d8bb-466e-937a-956df4cec9a0\") " pod="openshift-multus/multus-fgrpr" Jul 15 11:58:51.114191 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.111124 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/52832a70-d8bb-466e-937a-956df4cec9a0-multus-conf-dir\") pod \"multus-fgrpr\" (UID: \"52832a70-d8bb-466e-937a-956df4cec9a0\") " pod="openshift-multus/multus-fgrpr" Jul 15 11:58:51.114191 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.111164 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/52832a70-d8bb-466e-937a-956df4cec9a0-etc-kubernetes\") pod \"multus-fgrpr\" (UID: \"52832a70-d8bb-466e-937a-956df4cec9a0\") " pod="openshift-multus/multus-fgrpr" Jul 15 11:58:51.114191 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.111170 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ea415947-4742-4cf7-8b71-e22b418a131f-var-lib-openvswitch\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.114191 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.111182 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0228db4-1648-4484-8e9c-023c68c1e1e5-tmp-dir\") pod \"node-resolver-fnhqk\" (UID: \"b0228db4-1648-4484-8e9c-023c68c1e1e5\") " pod="openshift-dns/node-resolver-fnhqk" Jul 15 11:58:51.114191 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.111200 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/73bb7388-9486-4aa7-beff-f9d6b48e61be-host\") pod \"tuned-b58gj\" (UID: \"73bb7388-9486-4aa7-beff-f9d6b48e61be\") " pod="openshift-cluster-node-tuning-operator/tuned-b58gj" Jul 15 11:58:51.114191 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.111227 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/52832a70-d8bb-466e-937a-956df4cec9a0-host-var-lib-cni-multus\") pod \"multus-fgrpr\" (UID: \"52832a70-d8bb-466e-937a-956df4cec9a0\") " pod="openshift-multus/multus-fgrpr" Jul 15 11:58:51.114191 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.111246 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/52832a70-d8bb-466e-937a-956df4cec9a0-host-var-lib-kubelet\") pod \"multus-fgrpr\" (UID: \"52832a70-d8bb-466e-937a-956df4cec9a0\") " pod="openshift-multus/multus-fgrpr" Jul 15 11:58:51.114191 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.111264 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f76becda-10bd-4105-a1ee-41422f70c74b-registration-dir\") pod \"aws-ebs-csi-driver-node-x5ws6\" (UID: \"f76becda-10bd-4105-a1ee-41422f70c74b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x5ws6" Jul 15 11:58:51.114191 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.111291 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d09ffd58-1f50-4daa-8ce4-0c40ad65306b-host\") pod \"node-ca-5bb6n\" (UID: \"d09ffd58-1f50-4daa-8ce4-0c40ad65306b\") " pod="openshift-image-registry/node-ca-5bb6n" Jul 15 11:58:51.115097 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.111310 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/52832a70-d8bb-466e-937a-956df4cec9a0-cnibin\") pod \"multus-fgrpr\" (UID: \"52832a70-d8bb-466e-937a-956df4cec9a0\") " pod="openshift-multus/multus-fgrpr" Jul 15 11:58:51.115097 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.111334 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ea415947-4742-4cf7-8b71-e22b418a131f-ovnkube-config\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.115097 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.111355 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ea415947-4742-4cf7-8b71-e22b418a131f-ovn-node-metrics-cert\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.115097 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.111398 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/52832a70-d8bb-466e-937a-956df4cec9a0-hostroot\") pod \"multus-fgrpr\" (UID: \"52832a70-d8bb-466e-937a-956df4cec9a0\") " pod="openshift-multus/multus-fgrpr" Jul 15 11:58:51.115097 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.111419 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ea415947-4742-4cf7-8b71-e22b418a131f-host-run-netns\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.115097 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.111461 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ea415947-4742-4cf7-8b71-e22b418a131f-run-ovn\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.115097 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.110663 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a-system-cni-dir\") pod \"multus-additional-cni-plugins-lznkd\" (UID: \"ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a\") " pod="openshift-multus/multus-additional-cni-plugins-lznkd" Jul 15 11:58:51.115097 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.111492 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/52832a70-d8bb-466e-937a-956df4cec9a0-host-var-lib-kubelet\") pod \"multus-fgrpr\" (UID: \"52832a70-d8bb-466e-937a-956df4cec9a0\") " pod="openshift-multus/multus-fgrpr" Jul 15 11:58:51.115097 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.111541 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f76becda-10bd-4105-a1ee-41422f70c74b-registration-dir\") pod \"aws-ebs-csi-driver-node-x5ws6\" (UID: \"f76becda-10bd-4105-a1ee-41422f70c74b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x5ws6" Jul 15 11:58:51.115097 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.111543 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/52832a70-d8bb-466e-937a-956df4cec9a0-etc-kubernetes\") pod \"multus-fgrpr\" (UID: \"52832a70-d8bb-466e-937a-956df4cec9a0\") " pod="openshift-multus/multus-fgrpr" Jul 15 11:58:51.115097 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.111547 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ea415947-4742-4cf7-8b71-e22b418a131f-systemd-units\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.115097 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.111204 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/52832a70-d8bb-466e-937a-956df4cec9a0-multus-conf-dir\") pod \"multus-fgrpr\" (UID: \"52832a70-d8bb-466e-937a-956df4cec9a0\") " pod="openshift-multus/multus-fgrpr" Jul 15 11:58:51.115097 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.111579 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lznkd\" (UID: \"ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a\") " pod="openshift-multus/multus-additional-cni-plugins-lznkd" Jul 15 11:58:51.115097 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.111581 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d09ffd58-1f50-4daa-8ce4-0c40ad65306b-host\") pod \"node-ca-5bb6n\" (UID: \"d09ffd58-1f50-4daa-8ce4-0c40ad65306b\") " pod="openshift-image-registry/node-ca-5bb6n" Jul 15 11:58:51.115097 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.111601 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/73bb7388-9486-4aa7-beff-f9d6b48e61be-host\") pod \"tuned-b58gj\" (UID: \"73bb7388-9486-4aa7-beff-f9d6b48e61be\") " pod="openshift-cluster-node-tuning-operator/tuned-b58gj" Jul 15 11:58:51.115097 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.111615 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/52832a70-d8bb-466e-937a-956df4cec9a0-cnibin\") pod \"multus-fgrpr\" (UID: \"52832a70-d8bb-466e-937a-956df4cec9a0\") " pod="openshift-multus/multus-fgrpr" Jul 15 11:58:51.115097 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.111485 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ea415947-4742-4cf7-8b71-e22b418a131f-systemd-units\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.115097 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.111631 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/52832a70-d8bb-466e-937a-956df4cec9a0-hostroot\") pod \"multus-fgrpr\" (UID: \"52832a70-d8bb-466e-937a-956df4cec9a0\") " pod="openshift-multus/multus-fgrpr" Jul 15 11:58:51.115813 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.111645 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ea415947-4742-4cf7-8b71-e22b418a131f-ovnkube-script-lib\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.115813 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.111662 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ea415947-4742-4cf7-8b71-e22b418a131f-run-ovn\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.115813 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.111666 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/52832a70-d8bb-466e-937a-956df4cec9a0-system-cni-dir\") pod \"multus-fgrpr\" (UID: \"52832a70-d8bb-466e-937a-956df4cec9a0\") " pod="openshift-multus/multus-fgrpr" Jul 15 11:58:51.115813 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.111603 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/52832a70-d8bb-466e-937a-956df4cec9a0-host-var-lib-cni-multus\") pod \"multus-fgrpr\" (UID: \"52832a70-d8bb-466e-937a-956df4cec9a0\") " pod="openshift-multus/multus-fgrpr" Jul 15 11:58:51.115813 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.111917 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0228db4-1648-4484-8e9c-023c68c1e1e5-tmp-dir\") pod \"node-resolver-fnhqk\" (UID: \"b0228db4-1648-4484-8e9c-023c68c1e1e5\") " pod="openshift-dns/node-resolver-fnhqk" Jul 15 11:58:51.115813 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.111939 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lznkd\" (UID: \"ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a\") " pod="openshift-multus/multus-additional-cni-plugins-lznkd" Jul 15 11:58:51.115813 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.111975 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/20ea3927-e5c6-47c0-9ff6-43b7cad247c0-konnectivity-ca\") pod \"konnectivity-agent-x9kt6\" (UID: \"20ea3927-e5c6-47c0-9ff6-43b7cad247c0\") " pod="kube-system/konnectivity-agent-x9kt6" Jul 15 11:58:51.115813 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.111999 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ea415947-4742-4cf7-8b71-e22b418a131f-host-run-netns\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.115813 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.112049 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/52832a70-d8bb-466e-937a-956df4cec9a0-system-cni-dir\") pod \"multus-fgrpr\" (UID: \"52832a70-d8bb-466e-937a-956df4cec9a0\") " pod="openshift-multus/multus-fgrpr" Jul 15 11:58:51.115813 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.112076 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/52832a70-d8bb-466e-937a-956df4cec9a0-host-run-multus-certs\") pod \"multus-fgrpr\" (UID: \"52832a70-d8bb-466e-937a-956df4cec9a0\") " pod="openshift-multus/multus-fgrpr" Jul 15 11:58:51.115813 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.112109 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjwbn\" (UniqueName: \"kubernetes.io/projected/d09ffd58-1f50-4daa-8ce4-0c40ad65306b-kube-api-access-gjwbn\") pod \"node-ca-5bb6n\" (UID: \"d09ffd58-1f50-4daa-8ce4-0c40ad65306b\") " pod="openshift-image-registry/node-ca-5bb6n" Jul 15 11:58:51.115813 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.112136 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b0228db4-1648-4484-8e9c-023c68c1e1e5-hosts-file\") pod \"node-resolver-fnhqk\" (UID: \"b0228db4-1648-4484-8e9c-023c68c1e1e5\") " pod="openshift-dns/node-resolver-fnhqk" Jul 15 11:58:51.115813 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.112140 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ea415947-4742-4cf7-8b71-e22b418a131f-ovnkube-config\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.115813 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.112161 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9646j\" (UniqueName: \"kubernetes.io/projected/b0228db4-1648-4484-8e9c-023c68c1e1e5-kube-api-access-9646j\") pod \"node-resolver-fnhqk\" (UID: \"b0228db4-1648-4484-8e9c-023c68c1e1e5\") " pod="openshift-dns/node-resolver-fnhqk" Jul 15 11:58:51.115813 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.112186 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ea415947-4742-4cf7-8b71-e22b418a131f-host-slash\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.115813 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.112198 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b0228db4-1648-4484-8e9c-023c68c1e1e5-hosts-file\") pod \"node-resolver-fnhqk\" (UID: \"b0228db4-1648-4484-8e9c-023c68c1e1e5\") " pod="openshift-dns/node-resolver-fnhqk" Jul 15 11:58:51.115813 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.112201 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ea415947-4742-4cf7-8b71-e22b418a131f-ovnkube-script-lib\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.115813 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.112161 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/52832a70-d8bb-466e-937a-956df4cec9a0-host-run-multus-certs\") pod \"multus-fgrpr\" (UID: \"52832a70-d8bb-466e-937a-956df4cec9a0\") " pod="openshift-multus/multus-fgrpr" Jul 15 11:58:51.116394 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.112210 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ea415947-4742-4cf7-8b71-e22b418a131f-host-cni-netd\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.116394 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.112246 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/73bb7388-9486-4aa7-beff-f9d6b48e61be-var-lib-kubelet\") pod \"tuned-b58gj\" (UID: \"73bb7388-9486-4aa7-beff-f9d6b48e61be\") " pod="openshift-cluster-node-tuning-operator/tuned-b58gj" Jul 15 11:58:51.116394 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.112255 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ea415947-4742-4cf7-8b71-e22b418a131f-host-cni-netd\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.116394 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.112248 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ea415947-4742-4cf7-8b71-e22b418a131f-host-slash\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.116394 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.112271 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/73bb7388-9486-4aa7-beff-f9d6b48e61be-tmp\") pod \"tuned-b58gj\" (UID: \"73bb7388-9486-4aa7-beff-f9d6b48e61be\") " pod="openshift-cluster-node-tuning-operator/tuned-b58gj" Jul 15 11:58:51.116394 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.112290 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/73bb7388-9486-4aa7-beff-f9d6b48e61be-var-lib-kubelet\") pod \"tuned-b58gj\" (UID: \"73bb7388-9486-4aa7-beff-f9d6b48e61be\") " pod="openshift-cluster-node-tuning-operator/tuned-b58gj" Jul 15 11:58:51.116394 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.112331 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mcgk\" (UniqueName: \"kubernetes.io/projected/73bb7388-9486-4aa7-beff-f9d6b48e61be-kube-api-access-8mcgk\") pod \"tuned-b58gj\" (UID: \"73bb7388-9486-4aa7-beff-f9d6b48e61be\") " pod="openshift-cluster-node-tuning-operator/tuned-b58gj" Jul 15 11:58:51.116394 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.112357 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4zct\" (UniqueName: \"kubernetes.io/projected/16a7a1c8-5e68-45e2-bb17-78c299bd91a7-kube-api-access-z4zct\") pod \"iptables-alerter-gr6dm\" (UID: \"16a7a1c8-5e68-45e2-bb17-78c299bd91a7\") " pod="openshift-network-operator/iptables-alerter-gr6dm" Jul 15 11:58:51.116394 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.112394 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f76becda-10bd-4105-a1ee-41422f70c74b-socket-dir\") pod \"aws-ebs-csi-driver-node-x5ws6\" (UID: \"f76becda-10bd-4105-a1ee-41422f70c74b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x5ws6" Jul 15 11:58:51.116394 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.112418 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ea415947-4742-4cf7-8b71-e22b418a131f-etc-openvswitch\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.116394 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.112443 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ea415947-4742-4cf7-8b71-e22b418a131f-run-openvswitch\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.116394 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.112466 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ea415947-4742-4cf7-8b71-e22b418a131f-log-socket\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.116394 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.112489 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lznkd\" (UID: \"ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a\") " pod="openshift-multus/multus-additional-cni-plugins-lznkd" Jul 15 11:58:51.116394 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.112513 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f76becda-10bd-4105-a1ee-41422f70c74b-etc-selinux\") pod \"aws-ebs-csi-driver-node-x5ws6\" (UID: \"f76becda-10bd-4105-a1ee-41422f70c74b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x5ws6" Jul 15 11:58:51.116394 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.112527 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f76becda-10bd-4105-a1ee-41422f70c74b-socket-dir\") pod \"aws-ebs-csi-driver-node-x5ws6\" (UID: \"f76becda-10bd-4105-a1ee-41422f70c74b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x5ws6" Jul 15 11:58:51.116394 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.112537 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d09ffd58-1f50-4daa-8ce4-0c40ad65306b-serviceca\") pod \"node-ca-5bb6n\" (UID: \"d09ffd58-1f50-4daa-8ce4-0c40ad65306b\") " pod="openshift-image-registry/node-ca-5bb6n" Jul 15 11:58:51.116394 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.112553 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ea415947-4742-4cf7-8b71-e22b418a131f-run-openvswitch\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.117135 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.112570 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/52832a70-d8bb-466e-937a-956df4cec9a0-host-var-lib-cni-bin\") pod \"multus-fgrpr\" (UID: \"52832a70-d8bb-466e-937a-956df4cec9a0\") " pod="openshift-multus/multus-fgrpr" Jul 15 11:58:51.117135 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.112594 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ea415947-4742-4cf7-8b71-e22b418a131f-etc-openvswitch\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.117135 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.112598 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/52832a70-d8bb-466e-937a-956df4cec9a0-multus-socket-dir-parent\") pod \"multus-fgrpr\" (UID: \"52832a70-d8bb-466e-937a-956df4cec9a0\") " pod="openshift-multus/multus-fgrpr" Jul 15 11:58:51.117135 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.112631 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ea415947-4742-4cf7-8b71-e22b418a131f-log-socket\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.117135 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.112649 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/52832a70-d8bb-466e-937a-956df4cec9a0-multus-socket-dir-parent\") pod \"multus-fgrpr\" (UID: \"52832a70-d8bb-466e-937a-956df4cec9a0\") " pod="openshift-multus/multus-fgrpr" Jul 15 11:58:51.117135 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.112659 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/52832a70-d8bb-466e-937a-956df4cec9a0-host-run-k8s-cni-cncf-io\") pod \"multus-fgrpr\" (UID: \"52832a70-d8bb-466e-937a-956df4cec9a0\") " pod="openshift-multus/multus-fgrpr" Jul 15 11:58:51.117135 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.112688 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/73bb7388-9486-4aa7-beff-f9d6b48e61be-sys\") pod \"tuned-b58gj\" (UID: \"73bb7388-9486-4aa7-beff-f9d6b48e61be\") " pod="openshift-cluster-node-tuning-operator/tuned-b58gj" Jul 15 11:58:51.117135 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.112691 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/52832a70-d8bb-466e-937a-956df4cec9a0-host-var-lib-cni-bin\") pod \"multus-fgrpr\" (UID: \"52832a70-d8bb-466e-937a-956df4cec9a0\") " pod="openshift-multus/multus-fgrpr" Jul 15 11:58:51.117135 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.112778 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f76becda-10bd-4105-a1ee-41422f70c74b-etc-selinux\") pod \"aws-ebs-csi-driver-node-x5ws6\" (UID: \"f76becda-10bd-4105-a1ee-41422f70c74b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x5ws6" Jul 15 11:58:51.117135 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.112781 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/52832a70-d8bb-466e-937a-956df4cec9a0-host-run-k8s-cni-cncf-io\") pod \"multus-fgrpr\" (UID: \"52832a70-d8bb-466e-937a-956df4cec9a0\") " pod="openshift-multus/multus-fgrpr" Jul 15 11:58:51.117135 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.112822 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/73bb7388-9486-4aa7-beff-f9d6b48e61be-sys\") pod \"tuned-b58gj\" (UID: \"73bb7388-9486-4aa7-beff-f9d6b48e61be\") " pod="openshift-cluster-node-tuning-operator/tuned-b58gj" Jul 15 11:58:51.117135 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.112941 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d09ffd58-1f50-4daa-8ce4-0c40ad65306b-serviceca\") pod \"node-ca-5bb6n\" (UID: \"d09ffd58-1f50-4daa-8ce4-0c40ad65306b\") " pod="openshift-image-registry/node-ca-5bb6n" Jul 15 11:58:51.117135 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.113053 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lznkd\" (UID: \"ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a\") " pod="openshift-multus/multus-additional-cni-plugins-lznkd" Jul 15 11:58:51.117135 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.114421 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ea415947-4742-4cf7-8b71-e22b418a131f-ovn-node-metrics-cert\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.117135 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.114509 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/20ea3927-e5c6-47c0-9ff6-43b7cad247c0-agent-certs\") pod \"konnectivity-agent-x9kt6\" (UID: \"20ea3927-e5c6-47c0-9ff6-43b7cad247c0\") " pod="kube-system/konnectivity-agent-x9kt6" Jul 15 11:58:51.117135 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.114542 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/73bb7388-9486-4aa7-beff-f9d6b48e61be-etc-tuned\") pod \"tuned-b58gj\" (UID: \"73bb7388-9486-4aa7-beff-f9d6b48e61be\") " pod="openshift-cluster-node-tuning-operator/tuned-b58gj" Jul 15 11:58:51.117135 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.115505 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/73bb7388-9486-4aa7-beff-f9d6b48e61be-tmp\") pod \"tuned-b58gj\" (UID: \"73bb7388-9486-4aa7-beff-f9d6b48e61be\") " pod="openshift-cluster-node-tuning-operator/tuned-b58gj" Jul 15 11:58:51.120607 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:51.120590 2587 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 11:58:51.120607 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:51.120608 2587 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 11:58:51.120753 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:51.120619 2587 projected.go:196] Error preparing data for projected volume kube-api-access-75vl9 for pod openshift-network-diagnostics/network-check-target-qf4ll: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 11:58:51.120753 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:51.120692 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/29500681-aa9a-4fd5-b440-487ae81b05c6-kube-api-access-75vl9 podName:29500681-aa9a-4fd5-b440-487ae81b05c6 nodeName:}" failed. No retries permitted until 2026-07-15 11:58:51.620669327 +0000 UTC m=+3.078046243 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-75vl9" (UniqueName: "kubernetes.io/projected/29500681-aa9a-4fd5-b440-487ae81b05c6-kube-api-access-75vl9") pod "network-check-target-qf4ll" (UID: "29500681-aa9a-4fd5-b440-487ae81b05c6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 11:58:51.123670 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.123642 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9646j\" (UniqueName: \"kubernetes.io/projected/b0228db4-1648-4484-8e9c-023c68c1e1e5-kube-api-access-9646j\") pod \"node-resolver-fnhqk\" (UID: \"b0228db4-1648-4484-8e9c-023c68c1e1e5\") " pod="openshift-dns/node-resolver-fnhqk" Jul 15 11:58:51.124171 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.124145 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfd5g\" (UniqueName: \"kubernetes.io/projected/ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a-kube-api-access-nfd5g\") pod \"multus-additional-cni-plugins-lznkd\" (UID: \"ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a\") " pod="openshift-multus/multus-additional-cni-plugins-lznkd" Jul 15 11:58:51.124266 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.124231 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ml5zz\" (UniqueName: \"kubernetes.io/projected/ea415947-4742-4cf7-8b71-e22b418a131f-kube-api-access-ml5zz\") pod \"ovnkube-node-mcrkx\" (UID: \"ea415947-4742-4cf7-8b71-e22b418a131f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.124548 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.124479 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2nkp\" (UniqueName: \"kubernetes.io/projected/f76becda-10bd-4105-a1ee-41422f70c74b-kube-api-access-p2nkp\") pod \"aws-ebs-csi-driver-node-x5ws6\" (UID: \"f76becda-10bd-4105-a1ee-41422f70c74b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x5ws6" Jul 15 11:58:51.124957 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.124920 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-knqkh\" (UniqueName: \"kubernetes.io/projected/52832a70-d8bb-466e-937a-956df4cec9a0-kube-api-access-knqkh\") pod \"multus-fgrpr\" (UID: \"52832a70-d8bb-466e-937a-956df4cec9a0\") " pod="openshift-multus/multus-fgrpr" Jul 15 11:58:51.125855 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.125838 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mcgk\" (UniqueName: \"kubernetes.io/projected/73bb7388-9486-4aa7-beff-f9d6b48e61be-kube-api-access-8mcgk\") pod \"tuned-b58gj\" (UID: \"73bb7388-9486-4aa7-beff-f9d6b48e61be\") " pod="openshift-cluster-node-tuning-operator/tuned-b58gj" Jul 15 11:58:51.125937 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.125839 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cj9bd\" (UniqueName: \"kubernetes.io/projected/e7e5aa88-1817-4a28-8add-88cf29795c94-kube-api-access-cj9bd\") pod \"network-metrics-daemon-xtlxm\" (UID: \"e7e5aa88-1817-4a28-8add-88cf29795c94\") " pod="openshift-multus/network-metrics-daemon-xtlxm" Jul 15 11:58:51.126438 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.126417 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjwbn\" (UniqueName: \"kubernetes.io/projected/d09ffd58-1f50-4daa-8ce4-0c40ad65306b-kube-api-access-gjwbn\") pod \"node-ca-5bb6n\" (UID: \"d09ffd58-1f50-4daa-8ce4-0c40ad65306b\") " pod="openshift-image-registry/node-ca-5bb6n" Jul 15 11:58:51.126945 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.126924 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4zct\" (UniqueName: \"kubernetes.io/projected/16a7a1c8-5e68-45e2-bb17-78c299bd91a7-kube-api-access-z4zct\") pod \"iptables-alerter-gr6dm\" (UID: \"16a7a1c8-5e68-45e2-bb17-78c299bd91a7\") " pod="openshift-network-operator/iptables-alerter-gr6dm" Jul 15 11:58:51.185942 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.185887 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-243.ec2.internal" event={"ID":"c1bd23d6ee74dd41f0a99e2cf1119280","Type":"ContainerStarted","Data":"0446c33deab54a755c1fc262a53536231f9d7ac8207126fd4e9bf0579d52d0bd"} Jul 15 11:58:51.187149 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.187118 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-243.ec2.internal" event={"ID":"31cf20077539c85a825a4f13af6feab9","Type":"ContainerStarted","Data":"f103301c3939fee33fa1e35d7fd8dff7b79fff7732fc3ce3bf6a46f3fc6ad427"} Jul 15 11:58:51.268544 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.268462 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-gr6dm" Jul 15 11:58:51.274110 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.274086 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:58:51.281059 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.281038 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fgrpr" Jul 15 11:58:51.281526 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:51.281495 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16a7a1c8_5e68_45e2_bb17_78c299bd91a7.slice/crio-510cc907420cb75f12ac29de9ffbdd578745870afba6995904f39c2295437015 WatchSource:0}: Error finding container 510cc907420cb75f12ac29de9ffbdd578745870afba6995904f39c2295437015: Status 404 returned error can't find the container with id 510cc907420cb75f12ac29de9ffbdd578745870afba6995904f39c2295437015 Jul 15 11:58:51.285074 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:51.285047 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea415947_4742_4cf7_8b71_e22b418a131f.slice/crio-1de021694de917bdb5f0fea92698c589b7ec856fd913d464b2cbb18756fa3553 WatchSource:0}: Error finding container 1de021694de917bdb5f0fea92698c589b7ec856fd913d464b2cbb18756fa3553: Status 404 returned error can't find the container with id 1de021694de917bdb5f0fea92698c589b7ec856fd913d464b2cbb18756fa3553 Jul 15 11:58:51.287028 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.287007 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x5ws6" Jul 15 11:58:51.293038 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:51.293008 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52832a70_d8bb_466e_937a_956df4cec9a0.slice/crio-db5a76f63bfbac4899346d42198a7e9f734eb8716cb3f6cc8d49c0b68cb9bf27 WatchSource:0}: Error finding container db5a76f63bfbac4899346d42198a7e9f734eb8716cb3f6cc8d49c0b68cb9bf27: Status 404 returned error can't find the container with id db5a76f63bfbac4899346d42198a7e9f734eb8716cb3f6cc8d49c0b68cb9bf27 Jul 15 11:58:51.293981 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.293806 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-x9kt6" Jul 15 11:58:51.299321 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.299297 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5bb6n" Jul 15 11:58:51.299703 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:51.299662 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf76becda_10bd_4105_a1ee_41422f70c74b.slice/crio-f01d428d443591c649ed7353c5b9c3678857ad5fd08853040572081a80ff58e5 WatchSource:0}: Error finding container f01d428d443591c649ed7353c5b9c3678857ad5fd08853040572081a80ff58e5: Status 404 returned error can't find the container with id f01d428d443591c649ed7353c5b9c3678857ad5fd08853040572081a80ff58e5 Jul 15 11:58:51.304161 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:51.304136 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20ea3927_e5c6_47c0_9ff6_43b7cad247c0.slice/crio-40adfa2519898217a3bc47408f8fa6a4f6757267676dc09ff8aab15eac32cc99 WatchSource:0}: Error finding container 40adfa2519898217a3bc47408f8fa6a4f6757267676dc09ff8aab15eac32cc99: Status 404 returned error can't find the container with id 40adfa2519898217a3bc47408f8fa6a4f6757267676dc09ff8aab15eac32cc99 Jul 15 11:58:51.309042 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:51.309017 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd09ffd58_1f50_4daa_8ce4_0c40ad65306b.slice/crio-91e379186081b6aaef5c8747114695e54925982b204795100a1b446de608b192 WatchSource:0}: Error finding container 91e379186081b6aaef5c8747114695e54925982b204795100a1b446de608b192: Status 404 returned error can't find the container with id 91e379186081b6aaef5c8747114695e54925982b204795100a1b446de608b192 Jul 15 11:58:51.309959 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.309940 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-b58gj" Jul 15 11:58:51.316201 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.316178 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fnhqk" Jul 15 11:58:51.319657 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:51.319635 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73bb7388_9486_4aa7_beff_f9d6b48e61be.slice/crio-1aa2cfdbfd4f2f4ff4b35994264e301152621a594851720b24394d219c736689 WatchSource:0}: Error finding container 1aa2cfdbfd4f2f4ff4b35994264e301152621a594851720b24394d219c736689: Status 404 returned error can't find the container with id 1aa2cfdbfd4f2f4ff4b35994264e301152621a594851720b24394d219c736689 Jul 15 11:58:51.322776 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.322599 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-lznkd" Jul 15 11:58:51.326687 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:51.326650 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0228db4_1648_4484_8e9c_023c68c1e1e5.slice/crio-f143b8b2f00277b444bda50922071f28d440c4c8321a6968b7661530ae6edcd3 WatchSource:0}: Error finding container f143b8b2f00277b444bda50922071f28d440c4c8321a6968b7661530ae6edcd3: Status 404 returned error can't find the container with id f143b8b2f00277b444bda50922071f28d440c4c8321a6968b7661530ae6edcd3 Jul 15 11:58:51.332158 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:58:51.332134 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab7d4ec2_045c_4ee2_a028_9a0a4b8cc66a.slice/crio-3a1fdc4dcb138fb44c885c4366a9935d8b0e0a4e9c17e649fc38c7690850f2f7 WatchSource:0}: Error finding container 3a1fdc4dcb138fb44c885c4366a9935d8b0e0a4e9c17e649fc38c7690850f2f7: Status 404 returned error can't find the container with id 3a1fdc4dcb138fb44c885c4366a9935d8b0e0a4e9c17e649fc38c7690850f2f7 Jul 15 11:58:51.413161 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.413091 2587 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 11:58:51.617309 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.617234 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7e5aa88-1817-4a28-8add-88cf29795c94-metrics-certs\") pod \"network-metrics-daemon-xtlxm\" (UID: \"e7e5aa88-1817-4a28-8add-88cf29795c94\") " pod="openshift-multus/network-metrics-daemon-xtlxm" Jul 15 11:58:51.617309 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.617291 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c26f517e-49fe-4da5-a790-e8b77483317a-original-pull-secret\") pod \"global-pull-secret-syncer-vn6hr\" (UID: \"c26f517e-49fe-4da5-a790-e8b77483317a\") " pod="kube-system/global-pull-secret-syncer-vn6hr" Jul 15 11:58:51.617536 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:51.617421 2587 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 11:58:51.617536 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:51.617476 2587 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 11:58:51.617536 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:51.617500 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7e5aa88-1817-4a28-8add-88cf29795c94-metrics-certs podName:e7e5aa88-1817-4a28-8add-88cf29795c94 nodeName:}" failed. No retries permitted until 2026-07-15 11:58:52.617477433 +0000 UTC m=+4.074854349 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e7e5aa88-1817-4a28-8add-88cf29795c94-metrics-certs") pod "network-metrics-daemon-xtlxm" (UID: "e7e5aa88-1817-4a28-8add-88cf29795c94") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 11:58:51.617536 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:51.617528 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c26f517e-49fe-4da5-a790-e8b77483317a-original-pull-secret podName:c26f517e-49fe-4da5-a790-e8b77483317a nodeName:}" failed. No retries permitted until 2026-07-15 11:58:52.617511987 +0000 UTC m=+4.074888894 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c26f517e-49fe-4da5-a790-e8b77483317a-original-pull-secret") pod "global-pull-secret-syncer-vn6hr" (UID: "c26f517e-49fe-4da5-a790-e8b77483317a") : object "kube-system"/"original-pull-secret" not registered Jul 15 11:58:51.718139 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:51.718103 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75vl9\" (UniqueName: \"kubernetes.io/projected/29500681-aa9a-4fd5-b440-487ae81b05c6-kube-api-access-75vl9\") pod \"network-check-target-qf4ll\" (UID: \"29500681-aa9a-4fd5-b440-487ae81b05c6\") " pod="openshift-network-diagnostics/network-check-target-qf4ll" Jul 15 11:58:51.718337 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:51.718309 2587 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 11:58:51.718337 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:51.718333 2587 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 11:58:51.718429 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:51.718344 2587 projected.go:196] Error preparing data for projected volume kube-api-access-75vl9 for pod openshift-network-diagnostics/network-check-target-qf4ll: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 11:58:51.718429 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:51.718405 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/29500681-aa9a-4fd5-b440-487ae81b05c6-kube-api-access-75vl9 podName:29500681-aa9a-4fd5-b440-487ae81b05c6 nodeName:}" failed. No retries permitted until 2026-07-15 11:58:52.718384057 +0000 UTC m=+4.175760980 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-75vl9" (UniqueName: "kubernetes.io/projected/29500681-aa9a-4fd5-b440-487ae81b05c6-kube-api-access-75vl9") pod "network-check-target-qf4ll" (UID: "29500681-aa9a-4fd5-b440-487ae81b05c6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 11:58:52.053525 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:52.053193 2587 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 11:53:51 +0000 UTC" deadline="2028-03-29 07:55:01.726932126 +0000 UTC" Jul 15 11:58:52.053525 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:52.053233 2587 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14947h56m9.673704126s" Jul 15 11:58:52.178656 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:52.178624 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtlxm" Jul 15 11:58:52.178846 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:52.178797 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtlxm" podUID="e7e5aa88-1817-4a28-8add-88cf29795c94" Jul 15 11:58:52.179222 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:52.179199 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vn6hr" Jul 15 11:58:52.179326 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:52.179306 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vn6hr" podUID="c26f517e-49fe-4da5-a790-e8b77483317a" Jul 15 11:58:52.202464 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:52.202424 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lznkd" event={"ID":"ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a","Type":"ContainerStarted","Data":"3a1fdc4dcb138fb44c885c4366a9935d8b0e0a4e9c17e649fc38c7690850f2f7"} Jul 15 11:58:52.208668 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:52.208583 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5bb6n" event={"ID":"d09ffd58-1f50-4daa-8ce4-0c40ad65306b","Type":"ContainerStarted","Data":"91e379186081b6aaef5c8747114695e54925982b204795100a1b446de608b192"} Jul 15 11:58:52.217654 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:52.217617 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-x9kt6" event={"ID":"20ea3927-e5c6-47c0-9ff6-43b7cad247c0","Type":"ContainerStarted","Data":"40adfa2519898217a3bc47408f8fa6a4f6757267676dc09ff8aab15eac32cc99"} Jul 15 11:58:52.222781 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:52.222721 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x5ws6" event={"ID":"f76becda-10bd-4105-a1ee-41422f70c74b","Type":"ContainerStarted","Data":"f01d428d443591c649ed7353c5b9c3678857ad5fd08853040572081a80ff58e5"} Jul 15 11:58:52.227482 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:52.226096 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fgrpr" event={"ID":"52832a70-d8bb-466e-937a-956df4cec9a0","Type":"ContainerStarted","Data":"db5a76f63bfbac4899346d42198a7e9f734eb8716cb3f6cc8d49c0b68cb9bf27"} Jul 15 11:58:52.229675 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:52.229553 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-gr6dm" event={"ID":"16a7a1c8-5e68-45e2-bb17-78c299bd91a7","Type":"ContainerStarted","Data":"510cc907420cb75f12ac29de9ffbdd578745870afba6995904f39c2295437015"} Jul 15 11:58:52.243614 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:52.236707 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fnhqk" event={"ID":"b0228db4-1648-4484-8e9c-023c68c1e1e5","Type":"ContainerStarted","Data":"f143b8b2f00277b444bda50922071f28d440c4c8321a6968b7661530ae6edcd3"} Jul 15 11:58:52.243614 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:52.240866 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-b58gj" event={"ID":"73bb7388-9486-4aa7-beff-f9d6b48e61be","Type":"ContainerStarted","Data":"1aa2cfdbfd4f2f4ff4b35994264e301152621a594851720b24394d219c736689"} Jul 15 11:58:52.248994 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:52.248895 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" event={"ID":"ea415947-4742-4cf7-8b71-e22b418a131f","Type":"ContainerStarted","Data":"1de021694de917bdb5f0fea92698c589b7ec856fd913d464b2cbb18756fa3553"} Jul 15 11:58:52.625762 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:52.625348 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7e5aa88-1817-4a28-8add-88cf29795c94-metrics-certs\") pod \"network-metrics-daemon-xtlxm\" (UID: \"e7e5aa88-1817-4a28-8add-88cf29795c94\") " pod="openshift-multus/network-metrics-daemon-xtlxm" Jul 15 11:58:52.625762 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:52.625701 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c26f517e-49fe-4da5-a790-e8b77483317a-original-pull-secret\") pod \"global-pull-secret-syncer-vn6hr\" (UID: \"c26f517e-49fe-4da5-a790-e8b77483317a\") " pod="kube-system/global-pull-secret-syncer-vn6hr" Jul 15 11:58:52.626010 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:52.625946 2587 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 11:58:52.626075 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:52.626019 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c26f517e-49fe-4da5-a790-e8b77483317a-original-pull-secret podName:c26f517e-49fe-4da5-a790-e8b77483317a nodeName:}" failed. No retries permitted until 2026-07-15 11:58:54.625996568 +0000 UTC m=+6.083373473 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c26f517e-49fe-4da5-a790-e8b77483317a-original-pull-secret") pod "global-pull-secret-syncer-vn6hr" (UID: "c26f517e-49fe-4da5-a790-e8b77483317a") : object "kube-system"/"original-pull-secret" not registered Jul 15 11:58:52.626543 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:52.626513 2587 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 11:58:52.626644 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:52.626618 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7e5aa88-1817-4a28-8add-88cf29795c94-metrics-certs podName:e7e5aa88-1817-4a28-8add-88cf29795c94 nodeName:}" failed. No retries permitted until 2026-07-15 11:58:54.62659691 +0000 UTC m=+6.083973816 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e7e5aa88-1817-4a28-8add-88cf29795c94-metrics-certs") pod "network-metrics-daemon-xtlxm" (UID: "e7e5aa88-1817-4a28-8add-88cf29795c94") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 11:58:52.726856 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:52.726818 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75vl9\" (UniqueName: \"kubernetes.io/projected/29500681-aa9a-4fd5-b440-487ae81b05c6-kube-api-access-75vl9\") pod \"network-check-target-qf4ll\" (UID: \"29500681-aa9a-4fd5-b440-487ae81b05c6\") " pod="openshift-network-diagnostics/network-check-target-qf4ll" Jul 15 11:58:52.727037 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:52.727012 2587 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 11:58:52.727114 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:52.727038 2587 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 11:58:52.727114 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:52.727052 2587 projected.go:196] Error preparing data for projected volume kube-api-access-75vl9 for pod openshift-network-diagnostics/network-check-target-qf4ll: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 11:58:52.727114 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:52.727112 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/29500681-aa9a-4fd5-b440-487ae81b05c6-kube-api-access-75vl9 podName:29500681-aa9a-4fd5-b440-487ae81b05c6 nodeName:}" failed. No retries permitted until 2026-07-15 11:58:54.72709249 +0000 UTC m=+6.184469413 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-75vl9" (UniqueName: "kubernetes.io/projected/29500681-aa9a-4fd5-b440-487ae81b05c6-kube-api-access-75vl9") pod "network-check-target-qf4ll" (UID: "29500681-aa9a-4fd5-b440-487ae81b05c6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 11:58:53.054132 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:53.054036 2587 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 11:53:51 +0000 UTC" deadline="2028-02-03 18:32:47.132203834 +0000 UTC" Jul 15 11:58:53.054132 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:53.054098 2587 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13638h33m54.078109498s" Jul 15 11:58:53.182932 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:53.182329 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qf4ll" Jul 15 11:58:53.182932 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:53.182465 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qf4ll" podUID="29500681-aa9a-4fd5-b440-487ae81b05c6" Jul 15 11:58:53.272384 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:53.271550 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-243.ec2.internal" event={"ID":"c1bd23d6ee74dd41f0a99e2cf1119280","Type":"ContainerStarted","Data":"c5ad0a064a511f304cd1597df849b028e97a616954a07bc0bcab16311dffac58"} Jul 15 11:58:54.179605 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:54.179573 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vn6hr" Jul 15 11:58:54.180115 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:54.179712 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vn6hr" podUID="c26f517e-49fe-4da5-a790-e8b77483317a" Jul 15 11:58:54.180289 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:54.180261 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtlxm" Jul 15 11:58:54.180397 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:54.180376 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtlxm" podUID="e7e5aa88-1817-4a28-8add-88cf29795c94" Jul 15 11:58:54.643357 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:54.642749 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c26f517e-49fe-4da5-a790-e8b77483317a-original-pull-secret\") pod \"global-pull-secret-syncer-vn6hr\" (UID: \"c26f517e-49fe-4da5-a790-e8b77483317a\") " pod="kube-system/global-pull-secret-syncer-vn6hr" Jul 15 11:58:54.643357 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:54.642846 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7e5aa88-1817-4a28-8add-88cf29795c94-metrics-certs\") pod \"network-metrics-daemon-xtlxm\" (UID: \"e7e5aa88-1817-4a28-8add-88cf29795c94\") " pod="openshift-multus/network-metrics-daemon-xtlxm" Jul 15 11:58:54.643357 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:54.642923 2587 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 11:58:54.643357 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:54.642938 2587 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 11:58:54.643357 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:54.642985 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c26f517e-49fe-4da5-a790-e8b77483317a-original-pull-secret podName:c26f517e-49fe-4da5-a790-e8b77483317a nodeName:}" failed. No retries permitted until 2026-07-15 11:58:58.642966832 +0000 UTC m=+10.100343741 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c26f517e-49fe-4da5-a790-e8b77483317a-original-pull-secret") pod "global-pull-secret-syncer-vn6hr" (UID: "c26f517e-49fe-4da5-a790-e8b77483317a") : object "kube-system"/"original-pull-secret" not registered Jul 15 11:58:54.643357 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:54.643002 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7e5aa88-1817-4a28-8add-88cf29795c94-metrics-certs podName:e7e5aa88-1817-4a28-8add-88cf29795c94 nodeName:}" failed. No retries permitted until 2026-07-15 11:58:58.642993839 +0000 UTC m=+10.100370746 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e7e5aa88-1817-4a28-8add-88cf29795c94-metrics-certs") pod "network-metrics-daemon-xtlxm" (UID: "e7e5aa88-1817-4a28-8add-88cf29795c94") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 11:58:54.744044 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:54.743379 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75vl9\" (UniqueName: \"kubernetes.io/projected/29500681-aa9a-4fd5-b440-487ae81b05c6-kube-api-access-75vl9\") pod \"network-check-target-qf4ll\" (UID: \"29500681-aa9a-4fd5-b440-487ae81b05c6\") " pod="openshift-network-diagnostics/network-check-target-qf4ll" Jul 15 11:58:54.744044 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:54.743601 2587 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 11:58:54.744044 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:54.743621 2587 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 11:58:54.744044 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:54.743633 2587 projected.go:196] Error preparing data for projected volume kube-api-access-75vl9 for pod openshift-network-diagnostics/network-check-target-qf4ll: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 11:58:54.744044 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:54.743691 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/29500681-aa9a-4fd5-b440-487ae81b05c6-kube-api-access-75vl9 podName:29500681-aa9a-4fd5-b440-487ae81b05c6 nodeName:}" failed. No retries permitted until 2026-07-15 11:58:58.743672944 +0000 UTC m=+10.201049850 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-75vl9" (UniqueName: "kubernetes.io/projected/29500681-aa9a-4fd5-b440-487ae81b05c6-kube-api-access-75vl9") pod "network-check-target-qf4ll" (UID: "29500681-aa9a-4fd5-b440-487ae81b05c6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 11:58:55.179615 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:55.179581 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qf4ll" Jul 15 11:58:55.180068 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:55.179723 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qf4ll" podUID="29500681-aa9a-4fd5-b440-487ae81b05c6" Jul 15 11:58:56.179666 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:56.179633 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtlxm" Jul 15 11:58:56.180143 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:56.179812 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtlxm" podUID="e7e5aa88-1817-4a28-8add-88cf29795c94" Jul 15 11:58:56.180143 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:56.179821 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vn6hr" Jul 15 11:58:56.180143 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:56.179906 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vn6hr" podUID="c26f517e-49fe-4da5-a790-e8b77483317a" Jul 15 11:58:57.179973 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:57.179637 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qf4ll" Jul 15 11:58:57.179973 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:57.179776 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qf4ll" podUID="29500681-aa9a-4fd5-b440-487ae81b05c6" Jul 15 11:58:58.178964 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:58.178824 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtlxm" Jul 15 11:58:58.179146 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:58.178964 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtlxm" podUID="e7e5aa88-1817-4a28-8add-88cf29795c94" Jul 15 11:58:58.179461 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:58.179321 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vn6hr" Jul 15 11:58:58.179461 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:58.179428 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vn6hr" podUID="c26f517e-49fe-4da5-a790-e8b77483317a" Jul 15 11:58:58.674093 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:58.674007 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7e5aa88-1817-4a28-8add-88cf29795c94-metrics-certs\") pod \"network-metrics-daemon-xtlxm\" (UID: \"e7e5aa88-1817-4a28-8add-88cf29795c94\") " pod="openshift-multus/network-metrics-daemon-xtlxm" Jul 15 11:58:58.674093 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:58.674077 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c26f517e-49fe-4da5-a790-e8b77483317a-original-pull-secret\") pod \"global-pull-secret-syncer-vn6hr\" (UID: \"c26f517e-49fe-4da5-a790-e8b77483317a\") " pod="kube-system/global-pull-secret-syncer-vn6hr" Jul 15 11:58:58.674582 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:58.674270 2587 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 11:58:58.674582 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:58.674339 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c26f517e-49fe-4da5-a790-e8b77483317a-original-pull-secret podName:c26f517e-49fe-4da5-a790-e8b77483317a nodeName:}" failed. No retries permitted until 2026-07-15 11:59:06.674320242 +0000 UTC m=+18.131697150 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c26f517e-49fe-4da5-a790-e8b77483317a-original-pull-secret") pod "global-pull-secret-syncer-vn6hr" (UID: "c26f517e-49fe-4da5-a790-e8b77483317a") : object "kube-system"/"original-pull-secret" not registered Jul 15 11:58:58.674582 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:58.674419 2587 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 11:58:58.674582 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:58.674463 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7e5aa88-1817-4a28-8add-88cf29795c94-metrics-certs podName:e7e5aa88-1817-4a28-8add-88cf29795c94 nodeName:}" failed. No retries permitted until 2026-07-15 11:59:06.674449236 +0000 UTC m=+18.131826139 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e7e5aa88-1817-4a28-8add-88cf29795c94-metrics-certs") pod "network-metrics-daemon-xtlxm" (UID: "e7e5aa88-1817-4a28-8add-88cf29795c94") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 11:58:58.774611 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:58.774573 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75vl9\" (UniqueName: \"kubernetes.io/projected/29500681-aa9a-4fd5-b440-487ae81b05c6-kube-api-access-75vl9\") pod \"network-check-target-qf4ll\" (UID: \"29500681-aa9a-4fd5-b440-487ae81b05c6\") " pod="openshift-network-diagnostics/network-check-target-qf4ll" Jul 15 11:58:58.774831 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:58.774802 2587 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 11:58:58.774831 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:58.774821 2587 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 11:58:58.774950 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:58.774834 2587 projected.go:196] Error preparing data for projected volume kube-api-access-75vl9 for pod openshift-network-diagnostics/network-check-target-qf4ll: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 11:58:58.774950 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:58.774891 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/29500681-aa9a-4fd5-b440-487ae81b05c6-kube-api-access-75vl9 podName:29500681-aa9a-4fd5-b440-487ae81b05c6 nodeName:}" failed. No retries permitted until 2026-07-15 11:59:06.774871901 +0000 UTC m=+18.232248819 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-75vl9" (UniqueName: "kubernetes.io/projected/29500681-aa9a-4fd5-b440-487ae81b05c6-kube-api-access-75vl9") pod "network-check-target-qf4ll" (UID: "29500681-aa9a-4fd5-b440-487ae81b05c6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 11:58:59.180211 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:58:59.180180 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qf4ll" Jul 15 11:58:59.180400 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:58:59.180291 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qf4ll" podUID="29500681-aa9a-4fd5-b440-487ae81b05c6" Jul 15 11:59:00.179087 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:00.179049 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtlxm" Jul 15 11:59:00.179531 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:00.179196 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtlxm" podUID="e7e5aa88-1817-4a28-8add-88cf29795c94" Jul 15 11:59:00.179531 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:00.179311 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vn6hr" Jul 15 11:59:00.179531 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:00.179411 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vn6hr" podUID="c26f517e-49fe-4da5-a790-e8b77483317a" Jul 15 11:59:00.290347 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:00.290163 2587 generic.go:356] "Generic (PLEG): container finished" podID="c1bd23d6ee74dd41f0a99e2cf1119280" containerID="c5ad0a064a511f304cd1597df849b028e97a616954a07bc0bcab16311dffac58" exitCode=0 Jul 15 11:59:00.290347 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:00.290211 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-243.ec2.internal" event={"ID":"c1bd23d6ee74dd41f0a99e2cf1119280","Type":"ContainerDied","Data":"c5ad0a064a511f304cd1597df849b028e97a616954a07bc0bcab16311dffac58"} Jul 15 11:59:01.179353 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:01.179316 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qf4ll" Jul 15 11:59:01.179758 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:01.179440 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qf4ll" podUID="29500681-aa9a-4fd5-b440-487ae81b05c6" Jul 15 11:59:02.179356 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:02.179317 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtlxm" Jul 15 11:59:02.179697 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:02.179363 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vn6hr" Jul 15 11:59:02.179697 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:02.179434 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtlxm" podUID="e7e5aa88-1817-4a28-8add-88cf29795c94" Jul 15 11:59:02.179697 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:02.179550 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vn6hr" podUID="c26f517e-49fe-4da5-a790-e8b77483317a" Jul 15 11:59:03.178694 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:03.178597 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qf4ll" Jul 15 11:59:03.178871 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:03.178743 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qf4ll" podUID="29500681-aa9a-4fd5-b440-487ae81b05c6" Jul 15 11:59:04.179048 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:04.179009 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtlxm" Jul 15 11:59:04.179512 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:04.179146 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtlxm" podUID="e7e5aa88-1817-4a28-8add-88cf29795c94" Jul 15 11:59:04.179512 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:04.179201 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vn6hr" Jul 15 11:59:04.179512 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:04.179319 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vn6hr" podUID="c26f517e-49fe-4da5-a790-e8b77483317a" Jul 15 11:59:05.179378 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:05.179345 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qf4ll" Jul 15 11:59:05.179856 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:05.179463 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qf4ll" podUID="29500681-aa9a-4fd5-b440-487ae81b05c6" Jul 15 11:59:06.179011 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:06.178971 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtlxm" Jul 15 11:59:06.179228 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:06.178990 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vn6hr" Jul 15 11:59:06.179228 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:06.179118 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtlxm" podUID="e7e5aa88-1817-4a28-8add-88cf29795c94" Jul 15 11:59:06.179332 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:06.179216 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vn6hr" podUID="c26f517e-49fe-4da5-a790-e8b77483317a" Jul 15 11:59:06.732663 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:06.732633 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7e5aa88-1817-4a28-8add-88cf29795c94-metrics-certs\") pod \"network-metrics-daemon-xtlxm\" (UID: \"e7e5aa88-1817-4a28-8add-88cf29795c94\") " pod="openshift-multus/network-metrics-daemon-xtlxm" Jul 15 11:59:06.733089 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:06.732681 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c26f517e-49fe-4da5-a790-e8b77483317a-original-pull-secret\") pod \"global-pull-secret-syncer-vn6hr\" (UID: \"c26f517e-49fe-4da5-a790-e8b77483317a\") " pod="kube-system/global-pull-secret-syncer-vn6hr" Jul 15 11:59:06.733089 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:06.732801 2587 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 11:59:06.733089 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:06.732837 2587 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 11:59:06.733089 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:06.732861 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7e5aa88-1817-4a28-8add-88cf29795c94-metrics-certs podName:e7e5aa88-1817-4a28-8add-88cf29795c94 nodeName:}" failed. No retries permitted until 2026-07-15 11:59:22.732845051 +0000 UTC m=+34.190221953 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e7e5aa88-1817-4a28-8add-88cf29795c94-metrics-certs") pod "network-metrics-daemon-xtlxm" (UID: "e7e5aa88-1817-4a28-8add-88cf29795c94") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 11:59:06.733089 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:06.732887 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c26f517e-49fe-4da5-a790-e8b77483317a-original-pull-secret podName:c26f517e-49fe-4da5-a790-e8b77483317a nodeName:}" failed. No retries permitted until 2026-07-15 11:59:22.732872182 +0000 UTC m=+34.190249098 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c26f517e-49fe-4da5-a790-e8b77483317a-original-pull-secret") pod "global-pull-secret-syncer-vn6hr" (UID: "c26f517e-49fe-4da5-a790-e8b77483317a") : object "kube-system"/"original-pull-secret" not registered Jul 15 11:59:06.833872 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:06.833843 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75vl9\" (UniqueName: \"kubernetes.io/projected/29500681-aa9a-4fd5-b440-487ae81b05c6-kube-api-access-75vl9\") pod \"network-check-target-qf4ll\" (UID: \"29500681-aa9a-4fd5-b440-487ae81b05c6\") " pod="openshift-network-diagnostics/network-check-target-qf4ll" Jul 15 11:59:06.834062 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:06.833979 2587 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 11:59:06.834062 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:06.833999 2587 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 11:59:06.834062 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:06.834011 2587 projected.go:196] Error preparing data for projected volume kube-api-access-75vl9 for pod openshift-network-diagnostics/network-check-target-qf4ll: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 11:59:06.834216 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:06.834069 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/29500681-aa9a-4fd5-b440-487ae81b05c6-kube-api-access-75vl9 podName:29500681-aa9a-4fd5-b440-487ae81b05c6 nodeName:}" failed. No retries permitted until 2026-07-15 11:59:22.834050051 +0000 UTC m=+34.291426968 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-75vl9" (UniqueName: "kubernetes.io/projected/29500681-aa9a-4fd5-b440-487ae81b05c6-kube-api-access-75vl9") pod "network-check-target-qf4ll" (UID: "29500681-aa9a-4fd5-b440-487ae81b05c6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 11:59:07.179289 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:07.179207 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qf4ll" Jul 15 11:59:07.179441 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:07.179326 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qf4ll" podUID="29500681-aa9a-4fd5-b440-487ae81b05c6" Jul 15 11:59:08.179202 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:08.179162 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtlxm" Jul 15 11:59:08.179656 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:08.179209 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vn6hr" Jul 15 11:59:08.179656 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:08.179315 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtlxm" podUID="e7e5aa88-1817-4a28-8add-88cf29795c94" Jul 15 11:59:08.179656 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:08.179444 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vn6hr" podUID="c26f517e-49fe-4da5-a790-e8b77483317a" Jul 15 11:59:09.179900 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:09.179869 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qf4ll" Jul 15 11:59:09.180331 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:09.179956 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qf4ll" podUID="29500681-aa9a-4fd5-b440-487ae81b05c6" Jul 15 11:59:10.179492 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:10.179456 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtlxm" Jul 15 11:59:10.179690 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:10.179464 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vn6hr" Jul 15 11:59:10.179690 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:10.179613 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtlxm" podUID="e7e5aa88-1817-4a28-8add-88cf29795c94" Jul 15 11:59:10.179690 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:10.179675 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vn6hr" podUID="c26f517e-49fe-4da5-a790-e8b77483317a" Jul 15 11:59:11.179299 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:11.179242 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qf4ll" Jul 15 11:59:11.179719 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:11.179355 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qf4ll" podUID="29500681-aa9a-4fd5-b440-487ae81b05c6" Jul 15 11:59:11.311438 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:11.311187 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-b58gj" event={"ID":"73bb7388-9486-4aa7-beff-f9d6b48e61be","Type":"ContainerStarted","Data":"0f93fdb4f293d2865162054881e1f8504339f534449c674000df5d42730d4d30"} Jul 15 11:59:11.314011 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:11.313921 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" event={"ID":"ea415947-4742-4cf7-8b71-e22b418a131f","Type":"ContainerStarted","Data":"c704ee99129d4285a5dcb7981ebbd34989f419c64feb705a15de6520e5dbbd9b"} Jul 15 11:59:11.315958 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:11.315933 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-243.ec2.internal" event={"ID":"31cf20077539c85a825a4f13af6feab9","Type":"ContainerStarted","Data":"d3564c25b760fee6546132e3c60b482be9276ace972a341f16a7aedd754cea59"} Jul 15 11:59:11.317232 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:11.317209 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lznkd" event={"ID":"ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a","Type":"ContainerStarted","Data":"46bdb3b33349026deabf1a2d992c56fef449e7811345472be0c09b108237792b"} Jul 15 11:59:11.318460 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:11.318438 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5bb6n" event={"ID":"d09ffd58-1f50-4daa-8ce4-0c40ad65306b","Type":"ContainerStarted","Data":"0308cab099f06018b85f08906d9a29a8043a85313fe0990b59cb8f038d624c20"} Jul 15 11:59:11.319610 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:11.319593 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-x9kt6" event={"ID":"20ea3927-e5c6-47c0-9ff6-43b7cad247c0","Type":"ContainerStarted","Data":"aa54804bf2c04fd384b197749c0c2551e0490e67251060d9fb3b5549a0890e00"} Jul 15 11:59:11.320952 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:11.320935 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x5ws6" event={"ID":"f76becda-10bd-4105-a1ee-41422f70c74b","Type":"ContainerStarted","Data":"bfae7a2e4d2f05b60ad48af633e77c6d4ba87d904c2478ea71fe19b43fe73c53"} Jul 15 11:59:11.322258 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:11.322233 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fgrpr" event={"ID":"52832a70-d8bb-466e-937a-956df4cec9a0","Type":"ContainerStarted","Data":"552c7cbb48b4695eb24e0e6dcaee73a3e626e4bdb49cd17398721924d907f4ff"} Jul 15 11:59:11.324016 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:11.323981 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-243.ec2.internal" event={"ID":"c1bd23d6ee74dd41f0a99e2cf1119280","Type":"ContainerStarted","Data":"316d2b41a8cdb560a583a2f7c4e4c1cbb5efc538fa59bac320ab8efb97b4ed11"} Jul 15 11:59:11.325681 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:11.325662 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fnhqk" event={"ID":"b0228db4-1648-4484-8e9c-023c68c1e1e5","Type":"ContainerStarted","Data":"790902fc6c05c0f0091c335a6fca7a68f0f015ade07e46238b48bd1258003cc1"} Jul 15 11:59:11.371767 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:11.370059 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-fgrpr" podStartSLOduration=2.5507337359999998 podStartE2EDuration="22.370039649s" totalImagesPullingTime="19.819305913s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 11:58:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 11:59:11.368925736 +0000 UTC m=+22.826302660" watchObservedRunningTime="2026-07-15 11:59:11.370039649 +0000 UTC m=+22.827416578" Jul 15 11:59:11.390458 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:11.390411 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-x9kt6" podStartSLOduration=2.677091128 podStartE2EDuration="22.390395983s" totalImagesPullingTime="19.713304855s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 11:58:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 11:59:11.389871419 +0000 UTC m=+22.847248365" watchObservedRunningTime="2026-07-15 11:59:11.390395983 +0000 UTC m=+22.847772907" Jul 15 11:59:11.415111 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:11.415067 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-b58gj" podStartSLOduration=2.716628213 podStartE2EDuration="22.415052753s" totalImagesPullingTime="19.69842454s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 11:58:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 11:59:11.414681761 +0000 UTC m=+22.872058686" watchObservedRunningTime="2026-07-15 11:59:11.415052753 +0000 UTC m=+22.872429677" Jul 15 11:59:11.434940 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:11.434868 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-fnhqk" podStartSLOduration=2.785481251 podStartE2EDuration="22.434853302s" totalImagesPullingTime="19.649372051s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 11:58:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 11:59:11.434353787 +0000 UTC m=+22.891730713" watchObservedRunningTime="2026-07-15 11:59:11.434853302 +0000 UTC m=+22.892230224" Jul 15 11:59:11.482872 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:11.482823 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-243.ec2.internal" podStartSLOduration=21.482806922 podStartE2EDuration="21.482806922s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 11:58:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 11:59:11.460259298 +0000 UTC m=+22.917636224" watchObservedRunningTime="2026-07-15 11:59:11.482806922 +0000 UTC m=+22.940183847" Jul 15 11:59:11.483101 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:11.482888 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-5bb6n" podStartSLOduration=2.775086261 podStartE2EDuration="22.482883036s" totalImagesPullingTime="19.707796775s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 11:58:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 11:59:11.482793268 +0000 UTC m=+22.940170194" watchObservedRunningTime="2026-07-15 11:59:11.482883036 +0000 UTC m=+22.940259961" Jul 15 11:59:11.517769 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:11.517705 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-243.ec2.internal" podStartSLOduration=21.51768921 podStartE2EDuration="21.51768921s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 11:58:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 11:59:11.517189986 +0000 UTC m=+22.974566923" watchObservedRunningTime="2026-07-15 11:59:11.51768921 +0000 UTC m=+22.975066134" Jul 15 11:59:12.179340 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:12.179109 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtlxm" Jul 15 11:59:12.180262 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:12.179166 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vn6hr" Jul 15 11:59:12.180262 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:12.179458 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtlxm" podUID="e7e5aa88-1817-4a28-8add-88cf29795c94" Jul 15 11:59:12.180262 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:12.179534 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vn6hr" podUID="c26f517e-49fe-4da5-a790-e8b77483317a" Jul 15 11:59:12.328790 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:12.328759 2587 generic.go:356] "Generic (PLEG): container finished" podID="ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a" containerID="46bdb3b33349026deabf1a2d992c56fef449e7811345472be0c09b108237792b" exitCode=0 Jul 15 11:59:12.328961 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:12.328835 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lznkd" event={"ID":"ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a","Type":"ContainerDied","Data":"46bdb3b33349026deabf1a2d992c56fef449e7811345472be0c09b108237792b"} Jul 15 11:59:12.331765 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:12.331668 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mcrkx_ea415947-4742-4cf7-8b71-e22b418a131f/ovn-acl-logging/0.log" Jul 15 11:59:12.332035 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:12.332015 2587 generic.go:356] "Generic (PLEG): container finished" podID="ea415947-4742-4cf7-8b71-e22b418a131f" containerID="2cf9c3d671d8268f56419873c8d58135ffa455de66f52bc0fd7c23530763118a" exitCode=1 Jul 15 11:59:12.332116 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:12.332090 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" event={"ID":"ea415947-4742-4cf7-8b71-e22b418a131f","Type":"ContainerStarted","Data":"8b1ef32cd405e54c2640a0bb12bd73f788df5e5b6ae2f0ecf16af03b9e5a8e5b"} Jul 15 11:59:12.332170 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:12.332160 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" event={"ID":"ea415947-4742-4cf7-8b71-e22b418a131f","Type":"ContainerStarted","Data":"c18d1ce73860a9210ee0f5cd0134c7402c24b0019a0171cd60e89674f5c74f25"} Jul 15 11:59:12.332211 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:12.332192 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" event={"ID":"ea415947-4742-4cf7-8b71-e22b418a131f","Type":"ContainerStarted","Data":"843a23b3872888a91f1dd814daa31c5293c11ac565d5696c9ab4965c290603b5"} Jul 15 11:59:12.332324 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:12.332227 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" event={"ID":"ea415947-4742-4cf7-8b71-e22b418a131f","Type":"ContainerStarted","Data":"5a865b1c37904412fe68e95006b53e9feb32cc8db7b95e9f326a681338a80f96"} Jul 15 11:59:12.332324 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:12.332256 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" event={"ID":"ea415947-4742-4cf7-8b71-e22b418a131f","Type":"ContainerDied","Data":"2cf9c3d671d8268f56419873c8d58135ffa455de66f52bc0fd7c23530763118a"} Jul 15 11:59:13.029476 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:13.029432 2587 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 15 11:59:13.079293 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:13.079176 2587 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-15T11:59:13.029462404Z","UUID":"6570dc48-6398-4b49-84cb-04d179e2246e","Handler":null,"Name":"","Endpoint":""} Jul 15 11:59:13.080764 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:13.080727 2587 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 15 11:59:13.080867 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:13.080788 2587 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 15 11:59:13.179553 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:13.179471 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qf4ll" Jul 15 11:59:13.179991 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:13.179596 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qf4ll" podUID="29500681-aa9a-4fd5-b440-487ae81b05c6" Jul 15 11:59:13.336749 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:13.336710 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x5ws6" event={"ID":"f76becda-10bd-4105-a1ee-41422f70c74b","Type":"ContainerStarted","Data":"2949b087a0e9a4192a7851e32ce319a07aace276f30b0885b10811f46f5eb4ce"} Jul 15 11:59:13.338223 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:13.338195 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-gr6dm" event={"ID":"16a7a1c8-5e68-45e2-bb17-78c299bd91a7","Type":"ContainerStarted","Data":"2a6098437065b7acf2b333d5172c6816b37d28f8ff1899c22f2125225f75ea42"} Jul 15 11:59:13.357551 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:13.357494 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-gr6dm" podStartSLOduration=4.62121373 podStartE2EDuration="24.357476929s" totalImagesPullingTime="19.736263199s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 11:58:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 11:59:13.356705758 +0000 UTC m=+24.814082683" watchObservedRunningTime="2026-07-15 11:59:13.357476929 +0000 UTC m=+24.814853855" Jul 15 11:59:13.764549 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:13.764514 2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-x9kt6" Jul 15 11:59:13.765145 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:13.765116 2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-x9kt6" Jul 15 11:59:14.179764 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:14.179525 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vn6hr" Jul 15 11:59:14.179764 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:14.179562 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtlxm" Jul 15 11:59:14.180224 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:14.179807 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vn6hr" podUID="c26f517e-49fe-4da5-a790-e8b77483317a" Jul 15 11:59:14.180224 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:14.179846 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtlxm" podUID="e7e5aa88-1817-4a28-8add-88cf29795c94" Jul 15 11:59:14.345066 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:14.345040 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mcrkx_ea415947-4742-4cf7-8b71-e22b418a131f/ovn-acl-logging/0.log" Jul 15 11:59:14.345774 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:14.345743 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" event={"ID":"ea415947-4742-4cf7-8b71-e22b418a131f","Type":"ContainerStarted","Data":"b42e569b38884bb84ced80a1ee222e5fcf69e086878f6ee97caf511f8164d667"} Jul 15 11:59:14.346000 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:14.345984 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-x9kt6" Jul 15 11:59:14.346557 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:14.346537 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-x9kt6" Jul 15 11:59:15.179328 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:15.179289 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qf4ll" Jul 15 11:59:15.179580 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:15.179447 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qf4ll" podUID="29500681-aa9a-4fd5-b440-487ae81b05c6" Jul 15 11:59:15.351842 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:15.351799 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x5ws6" event={"ID":"f76becda-10bd-4105-a1ee-41422f70c74b","Type":"ContainerStarted","Data":"02d76cb46cdafe41426f0f0a6f6c0b5eeb3b36614e31f0db46ec5b06b5fa56f1"} Jul 15 11:59:16.179609 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:16.179573 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtlxm" Jul 15 11:59:16.179813 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:16.179574 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vn6hr" Jul 15 11:59:16.179813 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:16.179714 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtlxm" podUID="e7e5aa88-1817-4a28-8add-88cf29795c94" Jul 15 11:59:16.179813 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:16.179800 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vn6hr" podUID="c26f517e-49fe-4da5-a790-e8b77483317a" Jul 15 11:59:17.179855 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:17.179597 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qf4ll" Jul 15 11:59:17.180498 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:17.179902 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qf4ll" podUID="29500681-aa9a-4fd5-b440-487ae81b05c6" Jul 15 11:59:17.357553 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:17.357515 2587 generic.go:356] "Generic (PLEG): container finished" podID="ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a" containerID="56d85c24b8814e65e0ac7d616087783d9f6a9bfcee8a8b4013acd5e01d8f273e" exitCode=0 Jul 15 11:59:17.357724 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:17.357610 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lznkd" event={"ID":"ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a","Type":"ContainerDied","Data":"56d85c24b8814e65e0ac7d616087783d9f6a9bfcee8a8b4013acd5e01d8f273e"} Jul 15 11:59:17.360826 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:17.360806 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mcrkx_ea415947-4742-4cf7-8b71-e22b418a131f/ovn-acl-logging/0.log" Jul 15 11:59:17.361224 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:17.361171 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" event={"ID":"ea415947-4742-4cf7-8b71-e22b418a131f","Type":"ContainerStarted","Data":"b3c6f3daf1f319c75f40b5b250766bb531beb53ce1479cdcae69654de439bd60"} Jul 15 11:59:17.361475 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:17.361456 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:59:17.361583 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:17.361522 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:59:17.361583 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:17.361570 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:59:17.361698 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:17.361655 2587 scope.go:117] "RemoveContainer" containerID="2cf9c3d671d8268f56419873c8d58135ffa455de66f52bc0fd7c23530763118a" Jul 15 11:59:17.376582 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:17.376564 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:59:17.377148 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:17.377135 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:59:17.383958 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:17.383920 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x5ws6" podStartSLOduration=5.8304980870000005 podStartE2EDuration="28.383907062s" totalImagesPullingTime="22.553408975s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 11:58:49 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 11:59:15.370701281 +0000 UTC m=+26.828078206" watchObservedRunningTime="2026-07-15 11:59:17.383907062 +0000 UTC m=+28.841284061" Jul 15 11:59:18.178811 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:18.178766 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtlxm" Jul 15 11:59:18.179045 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:18.178920 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtlxm" podUID="e7e5aa88-1817-4a28-8add-88cf29795c94" Jul 15 11:59:18.179045 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:18.178973 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vn6hr" Jul 15 11:59:18.179167 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:18.179078 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vn6hr" podUID="c26f517e-49fe-4da5-a790-e8b77483317a" Jul 15 11:59:18.367165 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:18.367139 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mcrkx_ea415947-4742-4cf7-8b71-e22b418a131f/ovn-acl-logging/0.log" Jul 15 11:59:18.367630 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:18.367468 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" event={"ID":"ea415947-4742-4cf7-8b71-e22b418a131f","Type":"ContainerStarted","Data":"00e2ccebb03e85d684d4044c55635defd5fd469c803ed3d4971a813bb5f2410f"} Jul 15 11:59:18.449942 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:18.449835 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" podStartSLOduration=9.623155731 podStartE2EDuration="29.44981786s" totalImagesPullingTime="19.826662129s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 11:58:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 11:59:18.445042715 +0000 UTC m=+29.902419640" watchObservedRunningTime="2026-07-15 11:59:18.44981786 +0000 UTC m=+29.907194786" Jul 15 11:59:19.058555 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:19.058479 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vn6hr"] Jul 15 11:59:19.058697 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:19.058577 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vn6hr" Jul 15 11:59:19.058697 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:19.058677 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vn6hr" podUID="c26f517e-49fe-4da5-a790-e8b77483317a" Jul 15 11:59:19.061461 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:19.061439 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xtlxm"] Jul 15 11:59:19.061565 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:19.061513 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtlxm" Jul 15 11:59:19.061609 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:19.061593 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtlxm" podUID="e7e5aa88-1817-4a28-8add-88cf29795c94" Jul 15 11:59:19.064455 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:19.064433 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qf4ll"] Jul 15 11:59:19.064560 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:19.064507 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qf4ll" Jul 15 11:59:19.064624 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:19.064582 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qf4ll" podUID="29500681-aa9a-4fd5-b440-487ae81b05c6" Jul 15 11:59:19.371222 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:19.371142 2587 generic.go:356] "Generic (PLEG): container finished" podID="ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a" containerID="895334d0d01be4922fe57c5693f24b87314e86ab6e3683eda62d368ae71b821f" exitCode=0 Jul 15 11:59:19.371578 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:19.371233 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lznkd" event={"ID":"ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a","Type":"ContainerDied","Data":"895334d0d01be4922fe57c5693f24b87314e86ab6e3683eda62d368ae71b821f"} Jul 15 11:59:21.179241 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:21.179193 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtlxm" Jul 15 11:59:21.179825 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:21.179193 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vn6hr" Jul 15 11:59:21.179825 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:21.179333 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtlxm" podUID="e7e5aa88-1817-4a28-8add-88cf29795c94" Jul 15 11:59:21.179825 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:21.179375 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vn6hr" podUID="c26f517e-49fe-4da5-a790-e8b77483317a" Jul 15 11:59:21.179825 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:21.179438 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qf4ll" Jul 15 11:59:21.179825 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:21.179536 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qf4ll" podUID="29500681-aa9a-4fd5-b440-487ae81b05c6" Jul 15 11:59:21.377085 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:21.377007 2587 generic.go:356] "Generic (PLEG): container finished" podID="ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a" containerID="89a4d11f60a255e6158f84c11d14e204e8cc942ecef6be552a375b83b9fbb159" exitCode=0 Jul 15 11:59:21.377085 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:21.377059 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lznkd" event={"ID":"ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a","Type":"ContainerDied","Data":"89a4d11f60a255e6158f84c11d14e204e8cc942ecef6be552a375b83b9fbb159"} Jul 15 11:59:22.759894 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:22.759853 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7e5aa88-1817-4a28-8add-88cf29795c94-metrics-certs\") pod \"network-metrics-daemon-xtlxm\" (UID: \"e7e5aa88-1817-4a28-8add-88cf29795c94\") " pod="openshift-multus/network-metrics-daemon-xtlxm" Jul 15 11:59:22.760287 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:22.759908 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c26f517e-49fe-4da5-a790-e8b77483317a-original-pull-secret\") pod \"global-pull-secret-syncer-vn6hr\" (UID: \"c26f517e-49fe-4da5-a790-e8b77483317a\") " pod="kube-system/global-pull-secret-syncer-vn6hr" Jul 15 11:59:22.760287 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:22.760014 2587 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 11:59:22.760287 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:22.760071 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c26f517e-49fe-4da5-a790-e8b77483317a-original-pull-secret podName:c26f517e-49fe-4da5-a790-e8b77483317a nodeName:}" failed. No retries permitted until 2026-07-15 11:59:54.76005801 +0000 UTC m=+66.217434918 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c26f517e-49fe-4da5-a790-e8b77483317a-original-pull-secret") pod "global-pull-secret-syncer-vn6hr" (UID: "c26f517e-49fe-4da5-a790-e8b77483317a") : object "kube-system"/"original-pull-secret" not registered Jul 15 11:59:22.760287 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:22.760014 2587 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 11:59:22.760287 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:22.760137 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7e5aa88-1817-4a28-8add-88cf29795c94-metrics-certs podName:e7e5aa88-1817-4a28-8add-88cf29795c94 nodeName:}" failed. No retries permitted until 2026-07-15 11:59:54.76012439 +0000 UTC m=+66.217501309 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e7e5aa88-1817-4a28-8add-88cf29795c94-metrics-certs") pod "network-metrics-daemon-xtlxm" (UID: "e7e5aa88-1817-4a28-8add-88cf29795c94") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 11:59:22.861167 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:22.861125 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75vl9\" (UniqueName: \"kubernetes.io/projected/29500681-aa9a-4fd5-b440-487ae81b05c6-kube-api-access-75vl9\") pod \"network-check-target-qf4ll\" (UID: \"29500681-aa9a-4fd5-b440-487ae81b05c6\") " pod="openshift-network-diagnostics/network-check-target-qf4ll" Jul 15 11:59:22.861325 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:22.861275 2587 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 11:59:22.861325 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:22.861298 2587 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 11:59:22.861325 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:22.861311 2587 projected.go:196] Error preparing data for projected volume kube-api-access-75vl9 for pod openshift-network-diagnostics/network-check-target-qf4ll: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 11:59:22.861434 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:22.861370 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/29500681-aa9a-4fd5-b440-487ae81b05c6-kube-api-access-75vl9 podName:29500681-aa9a-4fd5-b440-487ae81b05c6 nodeName:}" failed. No retries permitted until 2026-07-15 11:59:54.861354885 +0000 UTC m=+66.318731798 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-75vl9" (UniqueName: "kubernetes.io/projected/29500681-aa9a-4fd5-b440-487ae81b05c6-kube-api-access-75vl9") pod "network-check-target-qf4ll" (UID: "29500681-aa9a-4fd5-b440-487ae81b05c6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 11:59:23.179408 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:23.179372 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vn6hr" Jul 15 11:59:23.179408 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:23.179389 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtlxm" Jul 15 11:59:23.179647 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:23.179515 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qf4ll" Jul 15 11:59:23.179647 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:23.179515 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vn6hr" podUID="c26f517e-49fe-4da5-a790-e8b77483317a" Jul 15 11:59:23.179771 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:23.179640 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtlxm" podUID="e7e5aa88-1817-4a28-8add-88cf29795c94" Jul 15 11:59:23.179771 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:23.179747 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qf4ll" podUID="29500681-aa9a-4fd5-b440-487ae81b05c6" Jul 15 11:59:23.821698 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:23.821670 2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-243.ec2.internal" event="NodeReady" Jul 15 11:59:23.822228 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:23.821830 2587 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 15 11:59:23.863479 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:23.863448 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-b788d9845-z5cx7"] Jul 15 11:59:23.867677 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:23.867645 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b788d9845-z5cx7" Jul 15 11:59:23.870849 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:23.870825 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 15 11:59:23.870975 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:23.870825 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 15 11:59:23.871245 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:23.871216 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 15 11:59:23.871245 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:23.871232 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-vl9cw\"" Jul 15 11:59:23.873128 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:23.873111 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-k5m9c"] Jul 15 11:59:23.877995 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:23.877977 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-k5m9c" Jul 15 11:59:23.878915 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:23.878889 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 15 11:59:23.880132 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:23.880112 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-26qs2"] Jul 15 11:59:23.880955 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:23.880939 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bqsqx\"" Jul 15 11:59:23.881041 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:23.880936 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 15 11:59:23.881300 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:23.881281 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 15 11:59:23.885156 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:23.884360 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-26qs2" Jul 15 11:59:23.889403 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:23.889384 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 15 11:59:23.890139 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:23.889518 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 15 11:59:23.890139 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:23.889826 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-zh86n\"" Jul 15 11:59:23.890139 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:23.890025 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 15 11:59:23.890980 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:23.890961 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-b788d9845-z5cx7"] Jul 15 11:59:23.896346 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:23.896327 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-26qs2"] Jul 15 11:59:23.901865 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:23.901842 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-k5m9c"] Jul 15 11:59:23.970804 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:23.970775 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8db3f4f8-7a28-477d-9a5b-8df04d5a282a-tmp-dir\") pod \"dns-default-k5m9c\" (UID: \"8db3f4f8-7a28-477d-9a5b-8df04d5a282a\") " pod="openshift-dns/dns-default-k5m9c" Jul 15 11:59:23.970804 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:23.970807 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-trusted-ca\") pod \"image-registry-b788d9845-z5cx7\" (UID: \"f6e2e61d-a8c4-475d-a43a-fe32868b16ba\") " pod="openshift-image-registry/image-registry-b788d9845-z5cx7" Jul 15 11:59:23.971084 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:23.970920 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-installation-pull-secrets\") pod \"image-registry-b788d9845-z5cx7\" (UID: \"f6e2e61d-a8c4-475d-a43a-fe32868b16ba\") " pod="openshift-image-registry/image-registry-b788d9845-z5cx7" Jul 15 11:59:23.971084 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:23.970973 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8db3f4f8-7a28-477d-9a5b-8df04d5a282a-metrics-tls\") pod \"dns-default-k5m9c\" (UID: \"8db3f4f8-7a28-477d-9a5b-8df04d5a282a\") " pod="openshift-dns/dns-default-k5m9c" Jul 15 11:59:23.971084 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:23.970992 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/01fb3758-749d-4e30-b529-2250f27f4700-cert\") pod \"ingress-canary-26qs2\" (UID: \"01fb3758-749d-4e30-b529-2250f27f4700\") " pod="openshift-ingress-canary/ingress-canary-26qs2" Jul 15 11:59:23.971084 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:23.971010 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-ca-trust-extracted\") pod \"image-registry-b788d9845-z5cx7\" (UID: \"f6e2e61d-a8c4-475d-a43a-fe32868b16ba\") " pod="openshift-image-registry/image-registry-b788d9845-z5cx7" Jul 15 11:59:23.971084 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:23.971034 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-registry-tls\") pod \"image-registry-b788d9845-z5cx7\" (UID: \"f6e2e61d-a8c4-475d-a43a-fe32868b16ba\") " pod="openshift-image-registry/image-registry-b788d9845-z5cx7" Jul 15 11:59:23.971084 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:23.971052 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-registry-certificates\") pod \"image-registry-b788d9845-z5cx7\" (UID: \"f6e2e61d-a8c4-475d-a43a-fe32868b16ba\") " pod="openshift-image-registry/image-registry-b788d9845-z5cx7" Jul 15 11:59:23.971387 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:23.971120 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drr47\" (UniqueName: \"kubernetes.io/projected/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-kube-api-access-drr47\") pod \"image-registry-b788d9845-z5cx7\" (UID: \"f6e2e61d-a8c4-475d-a43a-fe32868b16ba\") " pod="openshift-image-registry/image-registry-b788d9845-z5cx7" Jul 15 11:59:23.971387 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:23.971154 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvtkw\" (UniqueName: \"kubernetes.io/projected/01fb3758-749d-4e30-b529-2250f27f4700-kube-api-access-cvtkw\") pod \"ingress-canary-26qs2\" (UID: \"01fb3758-749d-4e30-b529-2250f27f4700\") " pod="openshift-ingress-canary/ingress-canary-26qs2" Jul 15 11:59:23.971387 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:23.971170 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-image-registry-private-configuration\") pod \"image-registry-b788d9845-z5cx7\" (UID: \"f6e2e61d-a8c4-475d-a43a-fe32868b16ba\") " pod="openshift-image-registry/image-registry-b788d9845-z5cx7" Jul 15 11:59:23.971387 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:23.971187 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2npx\" (UniqueName: \"kubernetes.io/projected/8db3f4f8-7a28-477d-9a5b-8df04d5a282a-kube-api-access-h2npx\") pod \"dns-default-k5m9c\" (UID: \"8db3f4f8-7a28-477d-9a5b-8df04d5a282a\") " pod="openshift-dns/dns-default-k5m9c" Jul 15 11:59:23.971387 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:23.971206 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-bound-sa-token\") pod \"image-registry-b788d9845-z5cx7\" (UID: \"f6e2e61d-a8c4-475d-a43a-fe32868b16ba\") " pod="openshift-image-registry/image-registry-b788d9845-z5cx7" Jul 15 11:59:23.971387 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:23.971229 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8db3f4f8-7a28-477d-9a5b-8df04d5a282a-config-volume\") pod \"dns-default-k5m9c\" (UID: \"8db3f4f8-7a28-477d-9a5b-8df04d5a282a\") " pod="openshift-dns/dns-default-k5m9c" Jul 15 11:59:24.072133 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:24.072050 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-installation-pull-secrets\") pod \"image-registry-b788d9845-z5cx7\" (UID: \"f6e2e61d-a8c4-475d-a43a-fe32868b16ba\") " pod="openshift-image-registry/image-registry-b788d9845-z5cx7" Jul 15 11:59:24.072133 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:24.072090 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8db3f4f8-7a28-477d-9a5b-8df04d5a282a-metrics-tls\") pod \"dns-default-k5m9c\" (UID: \"8db3f4f8-7a28-477d-9a5b-8df04d5a282a\") " pod="openshift-dns/dns-default-k5m9c" Jul 15 11:59:24.072133 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:24.072115 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/01fb3758-749d-4e30-b529-2250f27f4700-cert\") pod \"ingress-canary-26qs2\" (UID: \"01fb3758-749d-4e30-b529-2250f27f4700\") " pod="openshift-ingress-canary/ingress-canary-26qs2" Jul 15 11:59:24.072433 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:24.072139 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-ca-trust-extracted\") pod \"image-registry-b788d9845-z5cx7\" (UID: \"f6e2e61d-a8c4-475d-a43a-fe32868b16ba\") " pod="openshift-image-registry/image-registry-b788d9845-z5cx7" Jul 15 11:59:24.072433 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:24.072167 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-registry-tls\") pod \"image-registry-b788d9845-z5cx7\" (UID: \"f6e2e61d-a8c4-475d-a43a-fe32868b16ba\") " pod="openshift-image-registry/image-registry-b788d9845-z5cx7" Jul 15 11:59:24.072433 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:24.072187 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-registry-certificates\") pod \"image-registry-b788d9845-z5cx7\" (UID: \"f6e2e61d-a8c4-475d-a43a-fe32868b16ba\") " pod="openshift-image-registry/image-registry-b788d9845-z5cx7" Jul 15 11:59:24.072433 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:24.072216 2587 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 11:59:24.072433 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:24.072292 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8db3f4f8-7a28-477d-9a5b-8df04d5a282a-metrics-tls podName:8db3f4f8-7a28-477d-9a5b-8df04d5a282a nodeName:}" failed. No retries permitted until 2026-07-15 11:59:24.572270524 +0000 UTC m=+36.029647431 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8db3f4f8-7a28-477d-9a5b-8df04d5a282a-metrics-tls") pod "dns-default-k5m9c" (UID: "8db3f4f8-7a28-477d-9a5b-8df04d5a282a") : secret "dns-default-metrics-tls" not found Jul 15 11:59:24.072433 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:24.072224 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drr47\" (UniqueName: \"kubernetes.io/projected/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-kube-api-access-drr47\") pod \"image-registry-b788d9845-z5cx7\" (UID: \"f6e2e61d-a8c4-475d-a43a-fe32868b16ba\") " pod="openshift-image-registry/image-registry-b788d9845-z5cx7" Jul 15 11:59:24.072433 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:24.072344 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvtkw\" (UniqueName: \"kubernetes.io/projected/01fb3758-749d-4e30-b529-2250f27f4700-kube-api-access-cvtkw\") pod \"ingress-canary-26qs2\" (UID: \"01fb3758-749d-4e30-b529-2250f27f4700\") " pod="openshift-ingress-canary/ingress-canary-26qs2" Jul 15 11:59:24.072433 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:24.072377 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-image-registry-private-configuration\") pod \"image-registry-b788d9845-z5cx7\" (UID: \"f6e2e61d-a8c4-475d-a43a-fe32868b16ba\") " pod="openshift-image-registry/image-registry-b788d9845-z5cx7" Jul 15 11:59:24.072433 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:24.072403 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2npx\" (UniqueName: \"kubernetes.io/projected/8db3f4f8-7a28-477d-9a5b-8df04d5a282a-kube-api-access-h2npx\") pod \"dns-default-k5m9c\" (UID: \"8db3f4f8-7a28-477d-9a5b-8df04d5a282a\") " pod="openshift-dns/dns-default-k5m9c" Jul 15 11:59:24.072433 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:24.072427 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-bound-sa-token\") pod \"image-registry-b788d9845-z5cx7\" (UID: \"f6e2e61d-a8c4-475d-a43a-fe32868b16ba\") " pod="openshift-image-registry/image-registry-b788d9845-z5cx7" Jul 15 11:59:24.072961 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:24.072449 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8db3f4f8-7a28-477d-9a5b-8df04d5a282a-config-volume\") pod \"dns-default-k5m9c\" (UID: \"8db3f4f8-7a28-477d-9a5b-8df04d5a282a\") " pod="openshift-dns/dns-default-k5m9c" Jul 15 11:59:24.072961 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:24.072471 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8db3f4f8-7a28-477d-9a5b-8df04d5a282a-tmp-dir\") pod \"dns-default-k5m9c\" (UID: \"8db3f4f8-7a28-477d-9a5b-8df04d5a282a\") " pod="openshift-dns/dns-default-k5m9c" Jul 15 11:59:24.072961 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:24.072493 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-trusted-ca\") pod \"image-registry-b788d9845-z5cx7\" (UID: \"f6e2e61d-a8c4-475d-a43a-fe32868b16ba\") " pod="openshift-image-registry/image-registry-b788d9845-z5cx7" Jul 15 11:59:24.072961 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:24.072565 2587 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 11:59:24.072961 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:24.072585 2587 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b788d9845-z5cx7: secret "image-registry-tls" not found Jul 15 11:59:24.072961 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:24.072611 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-ca-trust-extracted\") pod \"image-registry-b788d9845-z5cx7\" (UID: \"f6e2e61d-a8c4-475d-a43a-fe32868b16ba\") " pod="openshift-image-registry/image-registry-b788d9845-z5cx7" Jul 15 11:59:24.072961 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:24.072630 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-registry-tls podName:f6e2e61d-a8c4-475d-a43a-fe32868b16ba nodeName:}" failed. No retries permitted until 2026-07-15 11:59:24.572614231 +0000 UTC m=+36.029991137 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-registry-tls") pod "image-registry-b788d9845-z5cx7" (UID: "f6e2e61d-a8c4-475d-a43a-fe32868b16ba") : secret "image-registry-tls" not found Jul 15 11:59:24.073307 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:24.072972 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8db3f4f8-7a28-477d-9a5b-8df04d5a282a-tmp-dir\") pod \"dns-default-k5m9c\" (UID: \"8db3f4f8-7a28-477d-9a5b-8df04d5a282a\") " pod="openshift-dns/dns-default-k5m9c" Jul 15 11:59:24.073307 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:24.073189 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-registry-certificates\") pod \"image-registry-b788d9845-z5cx7\" (UID: \"f6e2e61d-a8c4-475d-a43a-fe32868b16ba\") " pod="openshift-image-registry/image-registry-b788d9845-z5cx7" Jul 15 11:59:24.073307 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:24.073216 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8db3f4f8-7a28-477d-9a5b-8df04d5a282a-config-volume\") pod \"dns-default-k5m9c\" (UID: \"8db3f4f8-7a28-477d-9a5b-8df04d5a282a\") " pod="openshift-dns/dns-default-k5m9c" Jul 15 11:59:24.073744 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:24.073703 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-trusted-ca\") pod \"image-registry-b788d9845-z5cx7\" (UID: \"f6e2e61d-a8c4-475d-a43a-fe32868b16ba\") " pod="openshift-image-registry/image-registry-b788d9845-z5cx7" Jul 15 11:59:24.077250 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:24.077094 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-installation-pull-secrets\") pod \"image-registry-b788d9845-z5cx7\" (UID: \"f6e2e61d-a8c4-475d-a43a-fe32868b16ba\") " pod="openshift-image-registry/image-registry-b788d9845-z5cx7" Jul 15 11:59:24.077371 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:24.077112 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-image-registry-private-configuration\") pod \"image-registry-b788d9845-z5cx7\" (UID: \"f6e2e61d-a8c4-475d-a43a-fe32868b16ba\") " pod="openshift-image-registry/image-registry-b788d9845-z5cx7" Jul 15 11:59:24.077371 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:24.077120 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/01fb3758-749d-4e30-b529-2250f27f4700-cert\") pod \"ingress-canary-26qs2\" (UID: \"01fb3758-749d-4e30-b529-2250f27f4700\") " pod="openshift-ingress-canary/ingress-canary-26qs2" Jul 15 11:59:24.081997 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:24.081972 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-bound-sa-token\") pod \"image-registry-b788d9845-z5cx7\" (UID: \"f6e2e61d-a8c4-475d-a43a-fe32868b16ba\") " pod="openshift-image-registry/image-registry-b788d9845-z5cx7" Jul 15 11:59:24.082118 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:24.082073 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2npx\" (UniqueName: \"kubernetes.io/projected/8db3f4f8-7a28-477d-9a5b-8df04d5a282a-kube-api-access-h2npx\") pod \"dns-default-k5m9c\" (UID: \"8db3f4f8-7a28-477d-9a5b-8df04d5a282a\") " pod="openshift-dns/dns-default-k5m9c" Jul 15 11:59:24.082600 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:24.082578 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drr47\" (UniqueName: \"kubernetes.io/projected/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-kube-api-access-drr47\") pod \"image-registry-b788d9845-z5cx7\" (UID: \"f6e2e61d-a8c4-475d-a43a-fe32868b16ba\") " pod="openshift-image-registry/image-registry-b788d9845-z5cx7" Jul 15 11:59:24.082600 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:24.082591 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvtkw\" (UniqueName: \"kubernetes.io/projected/01fb3758-749d-4e30-b529-2250f27f4700-kube-api-access-cvtkw\") pod \"ingress-canary-26qs2\" (UID: \"01fb3758-749d-4e30-b529-2250f27f4700\") " pod="openshift-ingress-canary/ingress-canary-26qs2" Jul 15 11:59:24.199268 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:24.199231 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-26qs2" Jul 15 11:59:24.578359 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:24.578322 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-registry-tls\") pod \"image-registry-b788d9845-z5cx7\" (UID: \"f6e2e61d-a8c4-475d-a43a-fe32868b16ba\") " pod="openshift-image-registry/image-registry-b788d9845-z5cx7" Jul 15 11:59:24.578559 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:24.578445 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8db3f4f8-7a28-477d-9a5b-8df04d5a282a-metrics-tls\") pod \"dns-default-k5m9c\" (UID: \"8db3f4f8-7a28-477d-9a5b-8df04d5a282a\") " pod="openshift-dns/dns-default-k5m9c" Jul 15 11:59:24.578559 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:24.578491 2587 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 11:59:24.578559 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:24.578506 2587 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b788d9845-z5cx7: secret "image-registry-tls" not found Jul 15 11:59:24.578722 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:24.578566 2587 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 11:59:24.578722 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:24.578590 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-registry-tls podName:f6e2e61d-a8c4-475d-a43a-fe32868b16ba nodeName:}" failed. No retries permitted until 2026-07-15 11:59:25.578569354 +0000 UTC m=+37.035946265 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-registry-tls") pod "image-registry-b788d9845-z5cx7" (UID: "f6e2e61d-a8c4-475d-a43a-fe32868b16ba") : secret "image-registry-tls" not found Jul 15 11:59:24.578722 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:24.578631 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8db3f4f8-7a28-477d-9a5b-8df04d5a282a-metrics-tls podName:8db3f4f8-7a28-477d-9a5b-8df04d5a282a nodeName:}" failed. No retries permitted until 2026-07-15 11:59:25.578612899 +0000 UTC m=+37.035989804 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8db3f4f8-7a28-477d-9a5b-8df04d5a282a-metrics-tls") pod "dns-default-k5m9c" (UID: "8db3f4f8-7a28-477d-9a5b-8df04d5a282a") : secret "dns-default-metrics-tls" not found Jul 15 11:59:25.179420 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:25.179384 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vn6hr" Jul 15 11:59:25.179916 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:25.179384 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qf4ll" Jul 15 11:59:25.179916 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:25.179384 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtlxm" Jul 15 11:59:25.183973 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:25.183943 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 15 11:59:25.184105 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:25.183985 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 15 11:59:25.184105 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:25.183949 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-f876h\"" Jul 15 11:59:25.184105 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:25.183959 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7c5z8\"" Jul 15 11:59:25.184105 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:25.183991 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 15 11:59:25.184105 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:25.183951 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 15 11:59:25.586442 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:25.586345 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8db3f4f8-7a28-477d-9a5b-8df04d5a282a-metrics-tls\") pod \"dns-default-k5m9c\" (UID: \"8db3f4f8-7a28-477d-9a5b-8df04d5a282a\") " pod="openshift-dns/dns-default-k5m9c" Jul 15 11:59:25.586442 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:25.586393 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-registry-tls\") pod \"image-registry-b788d9845-z5cx7\" (UID: \"f6e2e61d-a8c4-475d-a43a-fe32868b16ba\") " pod="openshift-image-registry/image-registry-b788d9845-z5cx7" Jul 15 11:59:25.586662 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:25.586514 2587 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 11:59:25.586662 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:25.586594 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8db3f4f8-7a28-477d-9a5b-8df04d5a282a-metrics-tls podName:8db3f4f8-7a28-477d-9a5b-8df04d5a282a nodeName:}" failed. No retries permitted until 2026-07-15 11:59:27.586572537 +0000 UTC m=+39.043949463 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8db3f4f8-7a28-477d-9a5b-8df04d5a282a-metrics-tls") pod "dns-default-k5m9c" (UID: "8db3f4f8-7a28-477d-9a5b-8df04d5a282a") : secret "dns-default-metrics-tls" not found Jul 15 11:59:25.586662 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:25.586513 2587 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 11:59:25.586662 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:25.586628 2587 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b788d9845-z5cx7: secret "image-registry-tls" not found Jul 15 11:59:25.586874 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:25.586705 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-registry-tls podName:f6e2e61d-a8c4-475d-a43a-fe32868b16ba nodeName:}" failed. No retries permitted until 2026-07-15 11:59:27.586685291 +0000 UTC m=+39.044062213 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-registry-tls") pod "image-registry-b788d9845-z5cx7" (UID: "f6e2e61d-a8c4-475d-a43a-fe32868b16ba") : secret "image-registry-tls" not found Jul 15 11:59:25.816048 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:25.816025 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-26qs2"] Jul 15 11:59:25.822299 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:59:25.822266 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01fb3758_749d_4e30_b529_2250f27f4700.slice/crio-91d4d91bca1e4d3a8a1f9c971b65f632f6394a1c1a5c2dda20c75980856acfda WatchSource:0}: Error finding container 91d4d91bca1e4d3a8a1f9c971b65f632f6394a1c1a5c2dda20c75980856acfda: Status 404 returned error can't find the container with id 91d4d91bca1e4d3a8a1f9c971b65f632f6394a1c1a5c2dda20c75980856acfda Jul 15 11:59:26.391521 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:26.391484 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-26qs2" event={"ID":"01fb3758-749d-4e30-b529-2250f27f4700","Type":"ContainerStarted","Data":"91d4d91bca1e4d3a8a1f9c971b65f632f6394a1c1a5c2dda20c75980856acfda"} Jul 15 11:59:26.394725 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:26.394696 2587 generic.go:356] "Generic (PLEG): container finished" podID="ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a" containerID="d072c6cdd5ca0bb2b73f810fe2d2b7ee4cecd06422630a013bf18b38bd45be62" exitCode=0 Jul 15 11:59:26.394868 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:26.394767 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lznkd" event={"ID":"ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a","Type":"ContainerDied","Data":"d072c6cdd5ca0bb2b73f810fe2d2b7ee4cecd06422630a013bf18b38bd45be62"} Jul 15 11:59:27.401824 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:27.401775 2587 generic.go:356] "Generic (PLEG): container finished" podID="ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a" containerID="00137993cbc9ed3a53234adf639698b454ef04015daf63033e4facecd871997d" exitCode=0 Jul 15 11:59:27.402200 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:27.401841 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lznkd" event={"ID":"ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a","Type":"ContainerDied","Data":"00137993cbc9ed3a53234adf639698b454ef04015daf63033e4facecd871997d"} Jul 15 11:59:27.601363 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:27.601150 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8db3f4f8-7a28-477d-9a5b-8df04d5a282a-metrics-tls\") pod \"dns-default-k5m9c\" (UID: \"8db3f4f8-7a28-477d-9a5b-8df04d5a282a\") " pod="openshift-dns/dns-default-k5m9c" Jul 15 11:59:27.601514 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:27.601382 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-registry-tls\") pod \"image-registry-b788d9845-z5cx7\" (UID: \"f6e2e61d-a8c4-475d-a43a-fe32868b16ba\") " pod="openshift-image-registry/image-registry-b788d9845-z5cx7" Jul 15 11:59:27.601514 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:27.601301 2587 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 11:59:27.601514 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:27.601511 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8db3f4f8-7a28-477d-9a5b-8df04d5a282a-metrics-tls podName:8db3f4f8-7a28-477d-9a5b-8df04d5a282a nodeName:}" failed. No retries permitted until 2026-07-15 11:59:31.601494332 +0000 UTC m=+43.058871251 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8db3f4f8-7a28-477d-9a5b-8df04d5a282a-metrics-tls") pod "dns-default-k5m9c" (UID: "8db3f4f8-7a28-477d-9a5b-8df04d5a282a") : secret "dns-default-metrics-tls" not found Jul 15 11:59:27.601688 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:27.601521 2587 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 11:59:27.601688 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:27.601534 2587 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b788d9845-z5cx7: secret "image-registry-tls" not found Jul 15 11:59:27.601688 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:27.601581 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-registry-tls podName:f6e2e61d-a8c4-475d-a43a-fe32868b16ba nodeName:}" failed. No retries permitted until 2026-07-15 11:59:31.601569994 +0000 UTC m=+43.058946917 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-registry-tls") pod "image-registry-b788d9845-z5cx7" (UID: "f6e2e61d-a8c4-475d-a43a-fe32868b16ba") : secret "image-registry-tls" not found Jul 15 11:59:28.407170 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:28.407132 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lznkd" event={"ID":"ab7d4ec2-045c-4ee2-a028-9a0a4b8cc66a","Type":"ContainerStarted","Data":"ec0c1e14efd3e8507703549946c80067bfece28865db36100c624c7cc8fac8f2"} Jul 15 11:59:28.435357 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:28.435263 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-lznkd" podStartSLOduration=8.185244712 podStartE2EDuration="39.43524651s" totalImagesPullingTime="31.250001798s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 11:58:49 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 11:59:28.433616956 +0000 UTC m=+39.890993881" watchObservedRunningTime="2026-07-15 11:59:28.43524651 +0000 UTC m=+39.892623435" Jul 15 11:59:29.410912 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:29.410867 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-26qs2" event={"ID":"01fb3758-749d-4e30-b529-2250f27f4700","Type":"ContainerStarted","Data":"75a04771d12e645dabe46139ed7a3e7c9d4651c6c07bfe886b7e40b0ad1027ae"} Jul 15 11:59:29.434977 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:29.434909 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-26qs2" podStartSLOduration=3.817072533 podStartE2EDuration="6.434893617s" totalImagesPullingTime="2.617821084s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 11:59:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 11:59:29.43310323 +0000 UTC m=+40.890480168" watchObservedRunningTime="2026-07-15 11:59:29.434893617 +0000 UTC m=+40.892270541" Jul 15 11:59:31.632411 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:31.632365 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8db3f4f8-7a28-477d-9a5b-8df04d5a282a-metrics-tls\") pod \"dns-default-k5m9c\" (UID: \"8db3f4f8-7a28-477d-9a5b-8df04d5a282a\") " pod="openshift-dns/dns-default-k5m9c" Jul 15 11:59:31.632411 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:31.632413 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-registry-tls\") pod \"image-registry-b788d9845-z5cx7\" (UID: \"f6e2e61d-a8c4-475d-a43a-fe32868b16ba\") " pod="openshift-image-registry/image-registry-b788d9845-z5cx7" Jul 15 11:59:31.632917 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:31.632521 2587 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 11:59:31.632917 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:31.632552 2587 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 11:59:31.632917 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:31.632570 2587 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b788d9845-z5cx7: secret "image-registry-tls" not found Jul 15 11:59:31.632917 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:31.632587 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8db3f4f8-7a28-477d-9a5b-8df04d5a282a-metrics-tls podName:8db3f4f8-7a28-477d-9a5b-8df04d5a282a nodeName:}" failed. No retries permitted until 2026-07-15 11:59:39.632572473 +0000 UTC m=+51.089949397 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8db3f4f8-7a28-477d-9a5b-8df04d5a282a-metrics-tls") pod "dns-default-k5m9c" (UID: "8db3f4f8-7a28-477d-9a5b-8df04d5a282a") : secret "dns-default-metrics-tls" not found Jul 15 11:59:31.632917 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:31.632618 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-registry-tls podName:f6e2e61d-a8c4-475d-a43a-fe32868b16ba nodeName:}" failed. No retries permitted until 2026-07-15 11:59:39.632603057 +0000 UTC m=+51.089979961 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-registry-tls") pod "image-registry-b788d9845-z5cx7" (UID: "f6e2e61d-a8c4-475d-a43a-fe32868b16ba") : secret "image-registry-tls" not found Jul 15 11:59:39.692840 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:39.692802 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8db3f4f8-7a28-477d-9a5b-8df04d5a282a-metrics-tls\") pod \"dns-default-k5m9c\" (UID: \"8db3f4f8-7a28-477d-9a5b-8df04d5a282a\") " pod="openshift-dns/dns-default-k5m9c" Jul 15 11:59:39.692840 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:39.692844 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-registry-tls\") pod \"image-registry-b788d9845-z5cx7\" (UID: \"f6e2e61d-a8c4-475d-a43a-fe32868b16ba\") " pod="openshift-image-registry/image-registry-b788d9845-z5cx7" Jul 15 11:59:39.693266 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:39.692945 2587 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 11:59:39.693266 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:39.692956 2587 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b788d9845-z5cx7: secret "image-registry-tls" not found Jul 15 11:59:39.693266 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:39.692953 2587 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 11:59:39.693266 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:39.693005 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-registry-tls podName:f6e2e61d-a8c4-475d-a43a-fe32868b16ba nodeName:}" failed. No retries permitted until 2026-07-15 11:59:55.692991804 +0000 UTC m=+67.150368721 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-registry-tls") pod "image-registry-b788d9845-z5cx7" (UID: "f6e2e61d-a8c4-475d-a43a-fe32868b16ba") : secret "image-registry-tls" not found Jul 15 11:59:39.693266 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:39.693025 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8db3f4f8-7a28-477d-9a5b-8df04d5a282a-metrics-tls podName:8db3f4f8-7a28-477d-9a5b-8df04d5a282a nodeName:}" failed. No retries permitted until 2026-07-15 11:59:55.693010112 +0000 UTC m=+67.150387032 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8db3f4f8-7a28-477d-9a5b-8df04d5a282a-metrics-tls") pod "dns-default-k5m9c" (UID: "8db3f4f8-7a28-477d-9a5b-8df04d5a282a") : secret "dns-default-metrics-tls" not found Jul 15 11:59:49.390129 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:49.390101 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mcrkx" Jul 15 11:59:50.757707 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:50.757672 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-rcdwx"] Jul 15 11:59:50.760253 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:50.760232 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-4xrn6"] Jul 15 11:59:50.760371 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:50.760351 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-rcdwx" Jul 15 11:59:50.761850 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:50.761832 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-4xrn6" Jul 15 11:59:50.767751 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:50.767720 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 15 11:59:50.769007 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:50.768968 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-djg7m\"" Jul 15 11:59:50.769007 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:50.768989 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 15 11:59:50.769007 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:50.768993 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 15 11:59:50.772295 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:50.772275 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95fwb\" (UniqueName: \"kubernetes.io/projected/85ce5adf-d361-4258-b4ca-7f13d35fe13d-kube-api-access-95fwb\") pod \"console-operator-7866497494-4xrn6\" (UID: \"85ce5adf-d361-4258-b4ca-7f13d35fe13d\") " pod="openshift-console-operator/console-operator-7866497494-4xrn6" Jul 15 11:59:50.772411 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:50.772364 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a378c375-de95-4373-9788-141a4646e636-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-rcdwx\" (UID: \"a378c375-de95-4373-9788-141a4646e636\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-rcdwx" Jul 15 11:59:50.772581 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:50.772432 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnc6x\" (UniqueName: \"kubernetes.io/projected/a378c375-de95-4373-9788-141a4646e636-kube-api-access-tnc6x\") pod \"cluster-samples-operator-65db858894-rcdwx\" (UID: \"a378c375-de95-4373-9788-141a4646e636\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-rcdwx" Jul 15 11:59:50.772581 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:50.772457 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/85ce5adf-d361-4258-b4ca-7f13d35fe13d-config\") pod \"console-operator-7866497494-4xrn6\" (UID: \"85ce5adf-d361-4258-b4ca-7f13d35fe13d\") " pod="openshift-console-operator/console-operator-7866497494-4xrn6" Jul 15 11:59:50.772581 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:50.772483 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/85ce5adf-d361-4258-b4ca-7f13d35fe13d-trusted-ca\") pod \"console-operator-7866497494-4xrn6\" (UID: \"85ce5adf-d361-4258-b4ca-7f13d35fe13d\") " pod="openshift-console-operator/console-operator-7866497494-4xrn6" Jul 15 11:59:50.772581 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:50.772562 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/85ce5adf-d361-4258-b4ca-7f13d35fe13d-serving-cert\") pod \"console-operator-7866497494-4xrn6\" (UID: \"85ce5adf-d361-4258-b4ca-7f13d35fe13d\") " pod="openshift-console-operator/console-operator-7866497494-4xrn6" Jul 15 11:59:50.776244 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:50.776223 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 15 11:59:50.776341 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:50.776244 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-v5vzw\"" Jul 15 11:59:50.776341 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:50.776252 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 15 11:59:50.776341 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:50.776266 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 15 11:59:50.776341 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:50.776266 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 15 11:59:50.777582 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:50.777565 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 15 11:59:50.782244 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:50.782224 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-rcdwx"] Jul 15 11:59:50.783136 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:50.783116 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-4xrn6"] Jul 15 11:59:50.850700 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:50.850658 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-8fjlk"] Jul 15 11:59:50.852424 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:50.852408 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-x26m7"] Jul 15 11:59:50.852543 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:50.852527 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-8fjlk" Jul 15 11:59:50.857128 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:50.857103 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-x26m7" Jul 15 11:59:50.871056 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:50.871021 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 15 11:59:50.871185 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:50.871058 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 15 11:59:50.871185 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:50.871067 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 15 11:59:50.871185 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:50.871061 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-9ch2s\"" Jul 15 11:59:50.871391 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:50.871354 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 15 11:59:50.871607 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:50.871592 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-qdppb\"" Jul 15 11:59:50.872125 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:50.872104 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-8fjlk"] Jul 15 11:59:50.873519 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:50.873501 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkhd8\" (UniqueName: \"kubernetes.io/projected/d094f781-6c7b-435a-aef1-4f2e30138a3e-kube-api-access-bkhd8\") pod \"volume-data-source-validator-78495597f6-x26m7\" (UID: \"d094f781-6c7b-435a-aef1-4f2e30138a3e\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-x26m7" Jul 15 11:59:50.873572 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:50.873532 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/85ce5adf-d361-4258-b4ca-7f13d35fe13d-serving-cert\") pod \"console-operator-7866497494-4xrn6\" (UID: \"85ce5adf-d361-4258-b4ca-7f13d35fe13d\") " pod="openshift-console-operator/console-operator-7866497494-4xrn6" Jul 15 11:59:50.873572 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:50.873552 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95fwb\" (UniqueName: \"kubernetes.io/projected/85ce5adf-d361-4258-b4ca-7f13d35fe13d-kube-api-access-95fwb\") pod \"console-operator-7866497494-4xrn6\" (UID: \"85ce5adf-d361-4258-b4ca-7f13d35fe13d\") " pod="openshift-console-operator/console-operator-7866497494-4xrn6" Jul 15 11:59:50.873633 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:50.873617 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/379eba97-49a2-4f00-afba-0e65b34fd260-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-8fjlk\" (UID: \"379eba97-49a2-4f00-afba-0e65b34fd260\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-8fjlk" Jul 15 11:59:50.873683 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:50.873669 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a378c375-de95-4373-9788-141a4646e636-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-rcdwx\" (UID: \"a378c375-de95-4373-9788-141a4646e636\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-rcdwx" Jul 15 11:59:50.873751 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:50.873721 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnc6x\" (UniqueName: \"kubernetes.io/projected/a378c375-de95-4373-9788-141a4646e636-kube-api-access-tnc6x\") pod \"cluster-samples-operator-65db858894-rcdwx\" (UID: \"a378c375-de95-4373-9788-141a4646e636\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-rcdwx" Jul 15 11:59:50.873886 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:50.873771 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/85ce5adf-d361-4258-b4ca-7f13d35fe13d-config\") pod \"console-operator-7866497494-4xrn6\" (UID: \"85ce5adf-d361-4258-b4ca-7f13d35fe13d\") " pod="openshift-console-operator/console-operator-7866497494-4xrn6" Jul 15 11:59:50.873886 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:50.873791 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/85ce5adf-d361-4258-b4ca-7f13d35fe13d-trusted-ca\") pod \"console-operator-7866497494-4xrn6\" (UID: \"85ce5adf-d361-4258-b4ca-7f13d35fe13d\") " pod="openshift-console-operator/console-operator-7866497494-4xrn6" Jul 15 11:59:50.873886 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:50.873806 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/379eba97-49a2-4f00-afba-0e65b34fd260-nginx-conf\") pod \"networking-console-plugin-6cf856758c-8fjlk\" (UID: \"379eba97-49a2-4f00-afba-0e65b34fd260\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-8fjlk" Jul 15 11:59:50.873886 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:50.873817 2587 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 11:59:50.873886 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:50.873877 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a378c375-de95-4373-9788-141a4646e636-samples-operator-tls podName:a378c375-de95-4373-9788-141a4646e636 nodeName:}" failed. No retries permitted until 2026-07-15 11:59:51.373860765 +0000 UTC m=+62.831237686 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a378c375-de95-4373-9788-141a4646e636-samples-operator-tls") pod "cluster-samples-operator-65db858894-rcdwx" (UID: "a378c375-de95-4373-9788-141a4646e636") : secret "samples-operator-tls" not found Jul 15 11:59:50.874387 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:50.874367 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/85ce5adf-d361-4258-b4ca-7f13d35fe13d-config\") pod \"console-operator-7866497494-4xrn6\" (UID: \"85ce5adf-d361-4258-b4ca-7f13d35fe13d\") " pod="openshift-console-operator/console-operator-7866497494-4xrn6" Jul 15 11:59:50.874717 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:50.874695 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/85ce5adf-d361-4258-b4ca-7f13d35fe13d-trusted-ca\") pod \"console-operator-7866497494-4xrn6\" (UID: \"85ce5adf-d361-4258-b4ca-7f13d35fe13d\") " pod="openshift-console-operator/console-operator-7866497494-4xrn6" Jul 15 11:59:50.877217 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:50.877197 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/85ce5adf-d361-4258-b4ca-7f13d35fe13d-serving-cert\") pod \"console-operator-7866497494-4xrn6\" (UID: \"85ce5adf-d361-4258-b4ca-7f13d35fe13d\") " pod="openshift-console-operator/console-operator-7866497494-4xrn6" Jul 15 11:59:50.889515 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:50.889489 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnc6x\" (UniqueName: \"kubernetes.io/projected/a378c375-de95-4373-9788-141a4646e636-kube-api-access-tnc6x\") pod \"cluster-samples-operator-65db858894-rcdwx\" (UID: \"a378c375-de95-4373-9788-141a4646e636\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-rcdwx" Jul 15 11:59:50.890883 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:50.890859 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-x26m7"] Jul 15 11:59:50.893647 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:50.893614 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-95fwb\" (UniqueName: \"kubernetes.io/projected/85ce5adf-d361-4258-b4ca-7f13d35fe13d-kube-api-access-95fwb\") pod \"console-operator-7866497494-4xrn6\" (UID: \"85ce5adf-d361-4258-b4ca-7f13d35fe13d\") " pod="openshift-console-operator/console-operator-7866497494-4xrn6" Jul 15 11:59:50.974404 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:50.974360 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/379eba97-49a2-4f00-afba-0e65b34fd260-nginx-conf\") pod \"networking-console-plugin-6cf856758c-8fjlk\" (UID: \"379eba97-49a2-4f00-afba-0e65b34fd260\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-8fjlk" Jul 15 11:59:50.974568 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:50.974418 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkhd8\" (UniqueName: \"kubernetes.io/projected/d094f781-6c7b-435a-aef1-4f2e30138a3e-kube-api-access-bkhd8\") pod \"volume-data-source-validator-78495597f6-x26m7\" (UID: \"d094f781-6c7b-435a-aef1-4f2e30138a3e\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-x26m7" Jul 15 11:59:50.974568 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:50.974474 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/379eba97-49a2-4f00-afba-0e65b34fd260-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-8fjlk\" (UID: \"379eba97-49a2-4f00-afba-0e65b34fd260\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-8fjlk" Jul 15 11:59:50.974683 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:50.974609 2587 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 15 11:59:50.974720 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:50.974705 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/379eba97-49a2-4f00-afba-0e65b34fd260-networking-console-plugin-cert podName:379eba97-49a2-4f00-afba-0e65b34fd260 nodeName:}" failed. No retries permitted until 2026-07-15 11:59:51.474682362 +0000 UTC m=+62.932059280 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/379eba97-49a2-4f00-afba-0e65b34fd260-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-8fjlk" (UID: "379eba97-49a2-4f00-afba-0e65b34fd260") : secret "networking-console-plugin-cert" not found Jul 15 11:59:50.975145 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:50.975129 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/379eba97-49a2-4f00-afba-0e65b34fd260-nginx-conf\") pod \"networking-console-plugin-6cf856758c-8fjlk\" (UID: \"379eba97-49a2-4f00-afba-0e65b34fd260\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-8fjlk" Jul 15 11:59:50.986080 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:50.986057 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkhd8\" (UniqueName: \"kubernetes.io/projected/d094f781-6c7b-435a-aef1-4f2e30138a3e-kube-api-access-bkhd8\") pod \"volume-data-source-validator-78495597f6-x26m7\" (UID: \"d094f781-6c7b-435a-aef1-4f2e30138a3e\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-x26m7" Jul 15 11:59:51.075427 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:51.075388 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-4xrn6" Jul 15 11:59:51.168291 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:51.168260 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-x26m7" Jul 15 11:59:51.200154 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:51.200025 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-4xrn6"] Jul 15 11:59:51.202513 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:59:51.202488 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85ce5adf_d361_4258_b4ca_7f13d35fe13d.slice/crio-0d4f85747fecaaca594a9dcd9aadb384b63f642d7bcff115108c005d837ea7f1 WatchSource:0}: Error finding container 0d4f85747fecaaca594a9dcd9aadb384b63f642d7bcff115108c005d837ea7f1: Status 404 returned error can't find the container with id 0d4f85747fecaaca594a9dcd9aadb384b63f642d7bcff115108c005d837ea7f1 Jul 15 11:59:51.311632 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:51.311596 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-x26m7"] Jul 15 11:59:51.316136 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:59:51.316107 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd094f781_6c7b_435a_aef1_4f2e30138a3e.slice/crio-922bea479739c4c3aef7eed46968d35286ce5b3d14ffeba8d699329ec732cf09 WatchSource:0}: Error finding container 922bea479739c4c3aef7eed46968d35286ce5b3d14ffeba8d699329ec732cf09: Status 404 returned error can't find the container with id 922bea479739c4c3aef7eed46968d35286ce5b3d14ffeba8d699329ec732cf09 Jul 15 11:59:51.378371 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:51.378274 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a378c375-de95-4373-9788-141a4646e636-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-rcdwx\" (UID: \"a378c375-de95-4373-9788-141a4646e636\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-rcdwx" Jul 15 11:59:51.378513 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:51.378415 2587 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 11:59:51.378513 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:51.378476 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a378c375-de95-4373-9788-141a4646e636-samples-operator-tls podName:a378c375-de95-4373-9788-141a4646e636 nodeName:}" failed. No retries permitted until 2026-07-15 11:59:52.378460562 +0000 UTC m=+63.835837468 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a378c375-de95-4373-9788-141a4646e636-samples-operator-tls") pod "cluster-samples-operator-65db858894-rcdwx" (UID: "a378c375-de95-4373-9788-141a4646e636") : secret "samples-operator-tls" not found Jul 15 11:59:51.462027 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:51.461979 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-x26m7" event={"ID":"d094f781-6c7b-435a-aef1-4f2e30138a3e","Type":"ContainerStarted","Data":"922bea479739c4c3aef7eed46968d35286ce5b3d14ffeba8d699329ec732cf09"} Jul 15 11:59:51.463015 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:51.462985 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-4xrn6" event={"ID":"85ce5adf-d361-4258-b4ca-7f13d35fe13d","Type":"ContainerStarted","Data":"0d4f85747fecaaca594a9dcd9aadb384b63f642d7bcff115108c005d837ea7f1"} Jul 15 11:59:51.479487 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:51.479455 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/379eba97-49a2-4f00-afba-0e65b34fd260-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-8fjlk\" (UID: \"379eba97-49a2-4f00-afba-0e65b34fd260\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-8fjlk" Jul 15 11:59:51.479619 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:51.479600 2587 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 15 11:59:51.479673 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:51.479664 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/379eba97-49a2-4f00-afba-0e65b34fd260-networking-console-plugin-cert podName:379eba97-49a2-4f00-afba-0e65b34fd260 nodeName:}" failed. No retries permitted until 2026-07-15 11:59:52.479649422 +0000 UTC m=+63.937026329 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/379eba97-49a2-4f00-afba-0e65b34fd260-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-8fjlk" (UID: "379eba97-49a2-4f00-afba-0e65b34fd260") : secret "networking-console-plugin-cert" not found Jul 15 11:59:52.385284 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:52.385245 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a378c375-de95-4373-9788-141a4646e636-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-rcdwx\" (UID: \"a378c375-de95-4373-9788-141a4646e636\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-rcdwx" Jul 15 11:59:52.385762 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:52.385436 2587 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 11:59:52.385762 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:52.385500 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a378c375-de95-4373-9788-141a4646e636-samples-operator-tls podName:a378c375-de95-4373-9788-141a4646e636 nodeName:}" failed. No retries permitted until 2026-07-15 11:59:54.385483243 +0000 UTC m=+65.842860150 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a378c375-de95-4373-9788-141a4646e636-samples-operator-tls") pod "cluster-samples-operator-65db858894-rcdwx" (UID: "a378c375-de95-4373-9788-141a4646e636") : secret "samples-operator-tls" not found Jul 15 11:59:52.486603 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:52.486579 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/379eba97-49a2-4f00-afba-0e65b34fd260-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-8fjlk\" (UID: \"379eba97-49a2-4f00-afba-0e65b34fd260\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-8fjlk" Jul 15 11:59:52.486765 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:52.486746 2587 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 15 11:59:52.486825 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:52.486814 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/379eba97-49a2-4f00-afba-0e65b34fd260-networking-console-plugin-cert podName:379eba97-49a2-4f00-afba-0e65b34fd260 nodeName:}" failed. No retries permitted until 2026-07-15 11:59:54.486798258 +0000 UTC m=+65.944175178 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/379eba97-49a2-4f00-afba-0e65b34fd260-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-8fjlk" (UID: "379eba97-49a2-4f00-afba-0e65b34fd260") : secret "networking-console-plugin-cert" not found Jul 15 11:59:53.469519 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:53.469469 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-x26m7" event={"ID":"d094f781-6c7b-435a-aef1-4f2e30138a3e","Type":"ContainerStarted","Data":"b82dcf289f746d2371a10f931ed189de8ce02d2b9b82e01e6e2f11bd64d41890"} Jul 15 11:59:53.487448 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:53.487390 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-x26m7" podStartSLOduration=2.359228413 podStartE2EDuration="3.487369757s" totalImagesPullingTime="1.128141344s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 11:59:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 11:59:53.486528185 +0000 UTC m=+64.943905135" watchObservedRunningTime="2026-07-15 11:59:53.487369757 +0000 UTC m=+64.944746683" Jul 15 11:59:54.403507 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:54.401348 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a378c375-de95-4373-9788-141a4646e636-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-rcdwx\" (UID: \"a378c375-de95-4373-9788-141a4646e636\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-rcdwx" Jul 15 11:59:54.403507 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:54.401506 2587 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 11:59:54.403507 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:54.401579 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a378c375-de95-4373-9788-141a4646e636-samples-operator-tls podName:a378c375-de95-4373-9788-141a4646e636 nodeName:}" failed. No retries permitted until 2026-07-15 11:59:58.401559731 +0000 UTC m=+69.858936650 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a378c375-de95-4373-9788-141a4646e636-samples-operator-tls") pod "cluster-samples-operator-65db858894-rcdwx" (UID: "a378c375-de95-4373-9788-141a4646e636") : secret "samples-operator-tls" not found Jul 15 11:59:54.473261 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:54.473236 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-4xrn6_85ce5adf-d361-4258-b4ca-7f13d35fe13d/console-operator/0.log" Jul 15 11:59:54.473585 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:54.473275 2587 generic.go:356] "Generic (PLEG): container finished" podID="85ce5adf-d361-4258-b4ca-7f13d35fe13d" containerID="c9ed9318654781d88bad1868e932c9b36f61d1019f1caa8d0f0b131282c67d2c" exitCode=255 Jul 15 11:59:54.473585 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:54.473364 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-4xrn6" event={"ID":"85ce5adf-d361-4258-b4ca-7f13d35fe13d","Type":"ContainerDied","Data":"c9ed9318654781d88bad1868e932c9b36f61d1019f1caa8d0f0b131282c67d2c"} Jul 15 11:59:54.473585 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:54.473528 2587 scope.go:117] "RemoveContainer" containerID="c9ed9318654781d88bad1868e932c9b36f61d1019f1caa8d0f0b131282c67d2c" Jul 15 11:59:54.502670 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:54.502630 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/379eba97-49a2-4f00-afba-0e65b34fd260-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-8fjlk\" (UID: \"379eba97-49a2-4f00-afba-0e65b34fd260\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-8fjlk" Jul 15 11:59:54.503052 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:54.502833 2587 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 15 11:59:54.503052 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:54.502907 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/379eba97-49a2-4f00-afba-0e65b34fd260-networking-console-plugin-cert podName:379eba97-49a2-4f00-afba-0e65b34fd260 nodeName:}" failed. No retries permitted until 2026-07-15 11:59:58.502889176 +0000 UTC m=+69.960266099 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/379eba97-49a2-4f00-afba-0e65b34fd260-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-8fjlk" (UID: "379eba97-49a2-4f00-afba-0e65b34fd260") : secret "networking-console-plugin-cert" not found Jul 15 11:59:54.804756 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:54.804695 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7e5aa88-1817-4a28-8add-88cf29795c94-metrics-certs\") pod \"network-metrics-daemon-xtlxm\" (UID: \"e7e5aa88-1817-4a28-8add-88cf29795c94\") " pod="openshift-multus/network-metrics-daemon-xtlxm" Jul 15 11:59:54.804932 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:54.804781 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c26f517e-49fe-4da5-a790-e8b77483317a-original-pull-secret\") pod \"global-pull-secret-syncer-vn6hr\" (UID: \"c26f517e-49fe-4da5-a790-e8b77483317a\") " pod="kube-system/global-pull-secret-syncer-vn6hr" Jul 15 11:59:54.807878 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:54.807859 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 15 11:59:54.807934 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:54.807858 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 15 11:59:54.815404 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:54.815382 2587 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 15 11:59:54.815468 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:54.815456 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7e5aa88-1817-4a28-8add-88cf29795c94-metrics-certs podName:e7e5aa88-1817-4a28-8add-88cf29795c94 nodeName:}" failed. No retries permitted until 2026-07-15 12:00:58.815434204 +0000 UTC m=+130.272811127 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e7e5aa88-1817-4a28-8add-88cf29795c94-metrics-certs") pod "network-metrics-daemon-xtlxm" (UID: "e7e5aa88-1817-4a28-8add-88cf29795c94") : secret "metrics-daemon-secret" not found Jul 15 11:59:54.817523 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:54.817505 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c26f517e-49fe-4da5-a790-e8b77483317a-original-pull-secret\") pod \"global-pull-secret-syncer-vn6hr\" (UID: \"c26f517e-49fe-4da5-a790-e8b77483317a\") " pod="kube-system/global-pull-secret-syncer-vn6hr" Jul 15 11:59:54.892685 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:54.892646 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vn6hr" Jul 15 11:59:54.905510 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:54.905481 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75vl9\" (UniqueName: \"kubernetes.io/projected/29500681-aa9a-4fd5-b440-487ae81b05c6-kube-api-access-75vl9\") pod \"network-check-target-qf4ll\" (UID: \"29500681-aa9a-4fd5-b440-487ae81b05c6\") " pod="openshift-network-diagnostics/network-check-target-qf4ll" Jul 15 11:59:54.909005 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:54.908985 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 15 11:59:54.918705 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:54.918669 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 15 11:59:54.928796 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:54.928760 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75vl9\" (UniqueName: \"kubernetes.io/projected/29500681-aa9a-4fd5-b440-487ae81b05c6-kube-api-access-75vl9\") pod \"network-check-target-qf4ll\" (UID: \"29500681-aa9a-4fd5-b440-487ae81b05c6\") " pod="openshift-network-diagnostics/network-check-target-qf4ll" Jul 15 11:59:55.016759 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:55.016721 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vn6hr"] Jul 15 11:59:55.018971 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:59:55.018948 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc26f517e_49fe_4da5_a790_e8b77483317a.slice/crio-60615642d4c45cb4c490ca75630fed00205fa87b5d7364ce2227b9f5c7bca3b2 WatchSource:0}: Error finding container 60615642d4c45cb4c490ca75630fed00205fa87b5d7364ce2227b9f5c7bca3b2: Status 404 returned error can't find the container with id 60615642d4c45cb4c490ca75630fed00205fa87b5d7364ce2227b9f5c7bca3b2 Jul 15 11:59:55.203098 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:55.203021 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7c5z8\"" Jul 15 11:59:55.210326 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:55.210308 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qf4ll" Jul 15 11:59:55.339707 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:55.339676 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qf4ll"] Jul 15 11:59:55.342529 ip-10-0-135-243 kubenswrapper[2587]: W0715 11:59:55.342502 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29500681_aa9a_4fd5_b440_487ae81b05c6.slice/crio-d6e093dec99d9dfb860edcf6903e7da0f5df34be486768052437a4bd24ba13f8 WatchSource:0}: Error finding container d6e093dec99d9dfb860edcf6903e7da0f5df34be486768052437a4bd24ba13f8: Status 404 returned error can't find the container with id d6e093dec99d9dfb860edcf6903e7da0f5df34be486768052437a4bd24ba13f8 Jul 15 11:59:55.478005 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:55.477925 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-4xrn6_85ce5adf-d361-4258-b4ca-7f13d35fe13d/console-operator/1.log" Jul 15 11:59:55.478433 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:55.478385 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-4xrn6_85ce5adf-d361-4258-b4ca-7f13d35fe13d/console-operator/0.log" Jul 15 11:59:55.478488 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:55.478431 2587 generic.go:356] "Generic (PLEG): container finished" podID="85ce5adf-d361-4258-b4ca-7f13d35fe13d" containerID="fbfd08d19c3a05548670f9622a603edc408f8e746278c9ef8673f2cf24a3fe30" exitCode=255 Jul 15 11:59:55.478540 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:55.478511 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-4xrn6" event={"ID":"85ce5adf-d361-4258-b4ca-7f13d35fe13d","Type":"ContainerDied","Data":"fbfd08d19c3a05548670f9622a603edc408f8e746278c9ef8673f2cf24a3fe30"} Jul 15 11:59:55.478592 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:55.478569 2587 scope.go:117] "RemoveContainer" containerID="c9ed9318654781d88bad1868e932c9b36f61d1019f1caa8d0f0b131282c67d2c" Jul 15 11:59:55.478832 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:55.478810 2587 scope.go:117] "RemoveContainer" containerID="fbfd08d19c3a05548670f9622a603edc408f8e746278c9ef8673f2cf24a3fe30" Jul 15 11:59:55.479024 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:55.478996 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-4xrn6_openshift-console-operator(85ce5adf-d361-4258-b4ca-7f13d35fe13d)\"" pod="openshift-console-operator/console-operator-7866497494-4xrn6" podUID="85ce5adf-d361-4258-b4ca-7f13d35fe13d" Jul 15 11:59:55.479965 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:55.479939 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vn6hr" event={"ID":"c26f517e-49fe-4da5-a790-e8b77483317a","Type":"ContainerStarted","Data":"60615642d4c45cb4c490ca75630fed00205fa87b5d7364ce2227b9f5c7bca3b2"} Jul 15 11:59:55.481157 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:55.481130 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qf4ll" event={"ID":"29500681-aa9a-4fd5-b440-487ae81b05c6","Type":"ContainerStarted","Data":"d6e093dec99d9dfb860edcf6903e7da0f5df34be486768052437a4bd24ba13f8"} Jul 15 11:59:55.711834 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:55.711796 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8db3f4f8-7a28-477d-9a5b-8df04d5a282a-metrics-tls\") pod \"dns-default-k5m9c\" (UID: \"8db3f4f8-7a28-477d-9a5b-8df04d5a282a\") " pod="openshift-dns/dns-default-k5m9c" Jul 15 11:59:55.711834 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:55.711836 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-registry-tls\") pod \"image-registry-b788d9845-z5cx7\" (UID: \"f6e2e61d-a8c4-475d-a43a-fe32868b16ba\") " pod="openshift-image-registry/image-registry-b788d9845-z5cx7" Jul 15 11:59:55.712041 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:55.711951 2587 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 11:59:55.712041 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:55.712010 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8db3f4f8-7a28-477d-9a5b-8df04d5a282a-metrics-tls podName:8db3f4f8-7a28-477d-9a5b-8df04d5a282a nodeName:}" failed. No retries permitted until 2026-07-15 12:00:27.711995062 +0000 UTC m=+99.169371977 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8db3f4f8-7a28-477d-9a5b-8df04d5a282a-metrics-tls") pod "dns-default-k5m9c" (UID: "8db3f4f8-7a28-477d-9a5b-8df04d5a282a") : secret "dns-default-metrics-tls" not found Jul 15 11:59:55.712132 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:55.711954 2587 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 11:59:55.712132 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:55.712053 2587 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b788d9845-z5cx7: secret "image-registry-tls" not found Jul 15 11:59:55.712132 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:55.712094 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-registry-tls podName:f6e2e61d-a8c4-475d-a43a-fe32868b16ba nodeName:}" failed. No retries permitted until 2026-07-15 12:00:27.712082915 +0000 UTC m=+99.169459818 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-registry-tls") pod "image-registry-b788d9845-z5cx7" (UID: "f6e2e61d-a8c4-475d-a43a-fe32868b16ba") : secret "image-registry-tls" not found Jul 15 11:59:56.485750 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:56.485700 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-4xrn6_85ce5adf-d361-4258-b4ca-7f13d35fe13d/console-operator/1.log" Jul 15 11:59:56.486166 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:56.486114 2587 scope.go:117] "RemoveContainer" containerID="fbfd08d19c3a05548670f9622a603edc408f8e746278c9ef8673f2cf24a3fe30" Jul 15 11:59:56.486313 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:56.486287 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-4xrn6_openshift-console-operator(85ce5adf-d361-4258-b4ca-7f13d35fe13d)\"" pod="openshift-console-operator/console-operator-7866497494-4xrn6" podUID="85ce5adf-d361-4258-b4ca-7f13d35fe13d" Jul 15 11:59:57.419681 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:57.419646 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-8j9km"] Jul 15 11:59:57.440721 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:57.440690 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8j9km"] Jul 15 11:59:57.440890 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:57.440796 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8j9km" Jul 15 11:59:57.444113 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:57.443979 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 15 11:59:57.444113 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:57.444084 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-7kwbt\"" Jul 15 11:59:57.444113 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:57.443988 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 15 11:59:57.445451 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:57.445310 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 15 11:59:57.445451 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:57.445341 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 15 11:59:57.527780 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:57.527701 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b1be73b4-e61c-4c4b-9830-749d3c5a544b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8j9km\" (UID: \"b1be73b4-e61c-4c4b-9830-749d3c5a544b\") " pod="openshift-insights/insights-runtime-extractor-8j9km" Jul 15 11:59:57.527780 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:57.527773 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b1be73b4-e61c-4c4b-9830-749d3c5a544b-data-volume\") pod \"insights-runtime-extractor-8j9km\" (UID: \"b1be73b4-e61c-4c4b-9830-749d3c5a544b\") " pod="openshift-insights/insights-runtime-extractor-8j9km" Jul 15 11:59:57.528199 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:57.527911 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b1be73b4-e61c-4c4b-9830-749d3c5a544b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8j9km\" (UID: \"b1be73b4-e61c-4c4b-9830-749d3c5a544b\") " pod="openshift-insights/insights-runtime-extractor-8j9km" Jul 15 11:59:57.528199 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:57.527939 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttvz6\" (UniqueName: \"kubernetes.io/projected/b1be73b4-e61c-4c4b-9830-749d3c5a544b-kube-api-access-ttvz6\") pod \"insights-runtime-extractor-8j9km\" (UID: \"b1be73b4-e61c-4c4b-9830-749d3c5a544b\") " pod="openshift-insights/insights-runtime-extractor-8j9km" Jul 15 11:59:57.528199 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:57.527960 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b1be73b4-e61c-4c4b-9830-749d3c5a544b-crio-socket\") pod \"insights-runtime-extractor-8j9km\" (UID: \"b1be73b4-e61c-4c4b-9830-749d3c5a544b\") " pod="openshift-insights/insights-runtime-extractor-8j9km" Jul 15 11:59:57.629682 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:57.629639 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b1be73b4-e61c-4c4b-9830-749d3c5a544b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8j9km\" (UID: \"b1be73b4-e61c-4c4b-9830-749d3c5a544b\") " pod="openshift-insights/insights-runtime-extractor-8j9km" Jul 15 11:59:57.629862 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:57.629725 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b1be73b4-e61c-4c4b-9830-749d3c5a544b-data-volume\") pod \"insights-runtime-extractor-8j9km\" (UID: \"b1be73b4-e61c-4c4b-9830-749d3c5a544b\") " pod="openshift-insights/insights-runtime-extractor-8j9km" Jul 15 11:59:57.629937 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:57.629860 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b1be73b4-e61c-4c4b-9830-749d3c5a544b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8j9km\" (UID: \"b1be73b4-e61c-4c4b-9830-749d3c5a544b\") " pod="openshift-insights/insights-runtime-extractor-8j9km" Jul 15 11:59:57.629937 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:57.629902 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttvz6\" (UniqueName: \"kubernetes.io/projected/b1be73b4-e61c-4c4b-9830-749d3c5a544b-kube-api-access-ttvz6\") pod \"insights-runtime-extractor-8j9km\" (UID: \"b1be73b4-e61c-4c4b-9830-749d3c5a544b\") " pod="openshift-insights/insights-runtime-extractor-8j9km" Jul 15 11:59:57.630043 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:57.629937 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b1be73b4-e61c-4c4b-9830-749d3c5a544b-crio-socket\") pod \"insights-runtime-extractor-8j9km\" (UID: \"b1be73b4-e61c-4c4b-9830-749d3c5a544b\") " pod="openshift-insights/insights-runtime-extractor-8j9km" Jul 15 11:59:57.630101 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:57.630046 2587 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 11:59:57.630151 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:57.630142 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b1be73b4-e61c-4c4b-9830-749d3c5a544b-kube-rbac-proxy-cm podName:b1be73b4-e61c-4c4b-9830-749d3c5a544b nodeName:}" failed. No retries permitted until 2026-07-15 11:59:58.130120051 +0000 UTC m=+69.587496973 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b1be73b4-e61c-4c4b-9830-749d3c5a544b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8j9km" (UID: "b1be73b4-e61c-4c4b-9830-749d3c5a544b") : configmap "kube-rbac-proxy" not found Jul 15 11:59:57.630393 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:57.630369 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b1be73b4-e61c-4c4b-9830-749d3c5a544b-data-volume\") pod \"insights-runtime-extractor-8j9km\" (UID: \"b1be73b4-e61c-4c4b-9830-749d3c5a544b\") " pod="openshift-insights/insights-runtime-extractor-8j9km" Jul 15 11:59:57.630487 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:57.630470 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b1be73b4-e61c-4c4b-9830-749d3c5a544b-crio-socket\") pod \"insights-runtime-extractor-8j9km\" (UID: \"b1be73b4-e61c-4c4b-9830-749d3c5a544b\") " pod="openshift-insights/insights-runtime-extractor-8j9km" Jul 15 11:59:57.630599 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:57.630576 2587 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 11:59:57.630664 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:57.630640 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1be73b4-e61c-4c4b-9830-749d3c5a544b-insights-runtime-extractor-tls podName:b1be73b4-e61c-4c4b-9830-749d3c5a544b nodeName:}" failed. No retries permitted until 2026-07-15 11:59:58.130624601 +0000 UTC m=+69.588001505 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b1be73b4-e61c-4c4b-9830-749d3c5a544b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8j9km" (UID: "b1be73b4-e61c-4c4b-9830-749d3c5a544b") : secret "insights-runtime-extractor-tls" not found Jul 15 11:59:57.646965 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:57.646939 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttvz6\" (UniqueName: \"kubernetes.io/projected/b1be73b4-e61c-4c4b-9830-749d3c5a544b-kube-api-access-ttvz6\") pod \"insights-runtime-extractor-8j9km\" (UID: \"b1be73b4-e61c-4c4b-9830-749d3c5a544b\") " pod="openshift-insights/insights-runtime-extractor-8j9km" Jul 15 11:59:58.135626 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:58.135582 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b1be73b4-e61c-4c4b-9830-749d3c5a544b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8j9km\" (UID: \"b1be73b4-e61c-4c4b-9830-749d3c5a544b\") " pod="openshift-insights/insights-runtime-extractor-8j9km" Jul 15 11:59:58.135880 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:58.135678 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b1be73b4-e61c-4c4b-9830-749d3c5a544b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8j9km\" (UID: \"b1be73b4-e61c-4c4b-9830-749d3c5a544b\") " pod="openshift-insights/insights-runtime-extractor-8j9km" Jul 15 11:59:58.135880 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:58.135754 2587 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 11:59:58.135880 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:58.135781 2587 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 11:59:58.135880 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:58.135837 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1be73b4-e61c-4c4b-9830-749d3c5a544b-insights-runtime-extractor-tls podName:b1be73b4-e61c-4c4b-9830-749d3c5a544b nodeName:}" failed. No retries permitted until 2026-07-15 11:59:59.135815282 +0000 UTC m=+70.593192197 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b1be73b4-e61c-4c4b-9830-749d3c5a544b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8j9km" (UID: "b1be73b4-e61c-4c4b-9830-749d3c5a544b") : secret "insights-runtime-extractor-tls" not found Jul 15 11:59:58.135880 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:58.135858 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b1be73b4-e61c-4c4b-9830-749d3c5a544b-kube-rbac-proxy-cm podName:b1be73b4-e61c-4c4b-9830-749d3c5a544b nodeName:}" failed. No retries permitted until 2026-07-15 11:59:59.13584794 +0000 UTC m=+70.593224868 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b1be73b4-e61c-4c4b-9830-749d3c5a544b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8j9km" (UID: "b1be73b4-e61c-4c4b-9830-749d3c5a544b") : configmap "kube-rbac-proxy" not found Jul 15 11:59:58.439335 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:58.439247 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a378c375-de95-4373-9788-141a4646e636-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-rcdwx\" (UID: \"a378c375-de95-4373-9788-141a4646e636\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-rcdwx" Jul 15 11:59:58.439507 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:58.439407 2587 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 11:59:58.439507 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:58.439469 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a378c375-de95-4373-9788-141a4646e636-samples-operator-tls podName:a378c375-de95-4373-9788-141a4646e636 nodeName:}" failed. No retries permitted until 2026-07-15 12:00:06.439454441 +0000 UTC m=+77.896831344 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a378c375-de95-4373-9788-141a4646e636-samples-operator-tls") pod "cluster-samples-operator-65db858894-rcdwx" (UID: "a378c375-de95-4373-9788-141a4646e636") : secret "samples-operator-tls" not found Jul 15 11:59:58.539781 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:58.539725 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/379eba97-49a2-4f00-afba-0e65b34fd260-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-8fjlk\" (UID: \"379eba97-49a2-4f00-afba-0e65b34fd260\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-8fjlk" Jul 15 11:59:58.540255 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:58.539853 2587 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 15 11:59:58.540255 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:58.539942 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/379eba97-49a2-4f00-afba-0e65b34fd260-networking-console-plugin-cert podName:379eba97-49a2-4f00-afba-0e65b34fd260 nodeName:}" failed. No retries permitted until 2026-07-15 12:00:06.539918614 +0000 UTC m=+77.997295540 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/379eba97-49a2-4f00-afba-0e65b34fd260-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-8fjlk" (UID: "379eba97-49a2-4f00-afba-0e65b34fd260") : secret "networking-console-plugin-cert" not found Jul 15 11:59:59.144533 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:59.144494 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b1be73b4-e61c-4c4b-9830-749d3c5a544b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8j9km\" (UID: \"b1be73b4-e61c-4c4b-9830-749d3c5a544b\") " pod="openshift-insights/insights-runtime-extractor-8j9km" Jul 15 11:59:59.144753 ip-10-0-135-243 kubenswrapper[2587]: I0715 11:59:59.144594 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b1be73b4-e61c-4c4b-9830-749d3c5a544b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8j9km\" (UID: \"b1be73b4-e61c-4c4b-9830-749d3c5a544b\") " pod="openshift-insights/insights-runtime-extractor-8j9km" Jul 15 11:59:59.144753 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:59.144675 2587 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 11:59:59.144753 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:59.144678 2587 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 11:59:59.144910 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:59.144765 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1be73b4-e61c-4c4b-9830-749d3c5a544b-insights-runtime-extractor-tls podName:b1be73b4-e61c-4c4b-9830-749d3c5a544b nodeName:}" failed. No retries permitted until 2026-07-15 12:00:01.144726053 +0000 UTC m=+72.602102955 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b1be73b4-e61c-4c4b-9830-749d3c5a544b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8j9km" (UID: "b1be73b4-e61c-4c4b-9830-749d3c5a544b") : secret "insights-runtime-extractor-tls" not found Jul 15 11:59:59.144910 ip-10-0-135-243 kubenswrapper[2587]: E0715 11:59:59.144785 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b1be73b4-e61c-4c4b-9830-749d3c5a544b-kube-rbac-proxy-cm podName:b1be73b4-e61c-4c4b-9830-749d3c5a544b nodeName:}" failed. No retries permitted until 2026-07-15 12:00:01.144775464 +0000 UTC m=+72.602152369 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b1be73b4-e61c-4c4b-9830-749d3c5a544b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8j9km" (UID: "b1be73b4-e61c-4c4b-9830-749d3c5a544b") : configmap "kube-rbac-proxy" not found Jul 15 12:00:00.205911 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:00.205885 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fnhqk_b0228db4-1648-4484-8e9c-023c68c1e1e5/dns-node-resolver/0.log" Jul 15 12:00:00.497996 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:00.497960 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qf4ll" event={"ID":"29500681-aa9a-4fd5-b440-487ae81b05c6","Type":"ContainerStarted","Data":"b510f6c8faf6682102b89ab4112ba6e8dfe97b5542658a80e1279f9fd5f4d064"} Jul 15 12:00:00.498169 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:00.498045 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-qf4ll" Jul 15 12:00:00.499241 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:00.499219 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vn6hr" event={"ID":"c26f517e-49fe-4da5-a790-e8b77483317a","Type":"ContainerStarted","Data":"6d630d2b2cf5dd1ae56dfd0350203c4b5ea58f30525763a970af9e110bbe19a2"} Jul 15 12:00:00.517805 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:00.517763 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-qf4ll" podStartSLOduration=66.824096821 podStartE2EDuration="1m11.517751642s" totalImagesPullingTime="4.693654821s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 11:58:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:00:00.517295748 +0000 UTC m=+71.974672674" watchObservedRunningTime="2026-07-15 12:00:00.517751642 +0000 UTC m=+71.975128558" Jul 15 12:00:00.537349 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:00.537312 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-vn6hr" podStartSLOduration=66.51999725 podStartE2EDuration="1m11.537299889s" totalImagesPullingTime="5.017302639s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 11:58:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:00:00.536159276 +0000 UTC m=+71.993536212" watchObservedRunningTime="2026-07-15 12:00:00.537299889 +0000 UTC m=+71.994676818" Jul 15 12:00:01.001265 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:01.001239 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5bb6n_d09ffd58-1f50-4daa-8ce4-0c40ad65306b/node-ca/0.log" Jul 15 12:00:01.075528 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:01.075496 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-4xrn6" Jul 15 12:00:01.075690 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:01.075559 2587 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-4xrn6" Jul 15 12:00:01.075895 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:01.075883 2587 scope.go:117] "RemoveContainer" containerID="fbfd08d19c3a05548670f9622a603edc408f8e746278c9ef8673f2cf24a3fe30" Jul 15 12:00:01.076035 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:00:01.076018 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-4xrn6_openshift-console-operator(85ce5adf-d361-4258-b4ca-7f13d35fe13d)\"" pod="openshift-console-operator/console-operator-7866497494-4xrn6" podUID="85ce5adf-d361-4258-b4ca-7f13d35fe13d" Jul 15 12:00:01.165380 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:01.165352 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b1be73b4-e61c-4c4b-9830-749d3c5a544b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8j9km\" (UID: \"b1be73b4-e61c-4c4b-9830-749d3c5a544b\") " pod="openshift-insights/insights-runtime-extractor-8j9km" Jul 15 12:00:01.165500 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:01.165425 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b1be73b4-e61c-4c4b-9830-749d3c5a544b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8j9km\" (UID: \"b1be73b4-e61c-4c4b-9830-749d3c5a544b\") " pod="openshift-insights/insights-runtime-extractor-8j9km" Jul 15 12:00:01.165562 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:00:01.165512 2587 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 12:00:01.165609 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:00:01.165553 2587 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 12:00:01.165609 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:00:01.165586 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1be73b4-e61c-4c4b-9830-749d3c5a544b-insights-runtime-extractor-tls podName:b1be73b4-e61c-4c4b-9830-749d3c5a544b nodeName:}" failed. No retries permitted until 2026-07-15 12:00:05.16556283 +0000 UTC m=+76.622939751 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b1be73b4-e61c-4c4b-9830-749d3c5a544b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8j9km" (UID: "b1be73b4-e61c-4c4b-9830-749d3c5a544b") : secret "insights-runtime-extractor-tls" not found Jul 15 12:00:01.165698 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:00:01.165627 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b1be73b4-e61c-4c4b-9830-749d3c5a544b-kube-rbac-proxy-cm podName:b1be73b4-e61c-4c4b-9830-749d3c5a544b nodeName:}" failed. No retries permitted until 2026-07-15 12:00:05.165608302 +0000 UTC m=+76.622985416 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b1be73b4-e61c-4c4b-9830-749d3c5a544b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8j9km" (UID: "b1be73b4-e61c-4c4b-9830-749d3c5a544b") : configmap "kube-rbac-proxy" not found Jul 15 12:00:01.601382 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:01.601353 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-26qs2_01fb3758-749d-4e30-b529-2250f27f4700/serve-healthcheck-canary/0.log" Jul 15 12:00:05.198221 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:05.198192 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b1be73b4-e61c-4c4b-9830-749d3c5a544b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8j9km\" (UID: \"b1be73b4-e61c-4c4b-9830-749d3c5a544b\") " pod="openshift-insights/insights-runtime-extractor-8j9km" Jul 15 12:00:05.198593 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:05.198250 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b1be73b4-e61c-4c4b-9830-749d3c5a544b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8j9km\" (UID: \"b1be73b4-e61c-4c4b-9830-749d3c5a544b\") " pod="openshift-insights/insights-runtime-extractor-8j9km" Jul 15 12:00:05.198593 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:00:05.198316 2587 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 12:00:05.198593 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:00:05.198349 2587 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 12:00:05.198593 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:00:05.198369 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b1be73b4-e61c-4c4b-9830-749d3c5a544b-kube-rbac-proxy-cm podName:b1be73b4-e61c-4c4b-9830-749d3c5a544b nodeName:}" failed. No retries permitted until 2026-07-15 12:00:13.198354718 +0000 UTC m=+84.655731620 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b1be73b4-e61c-4c4b-9830-749d3c5a544b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8j9km" (UID: "b1be73b4-e61c-4c4b-9830-749d3c5a544b") : configmap "kube-rbac-proxy" not found Jul 15 12:00:05.198593 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:00:05.198405 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1be73b4-e61c-4c4b-9830-749d3c5a544b-insights-runtime-extractor-tls podName:b1be73b4-e61c-4c4b-9830-749d3c5a544b nodeName:}" failed. No retries permitted until 2026-07-15 12:00:13.198390415 +0000 UTC m=+84.655767321 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b1be73b4-e61c-4c4b-9830-749d3c5a544b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8j9km" (UID: "b1be73b4-e61c-4c4b-9830-749d3c5a544b") : secret "insights-runtime-extractor-tls" not found Jul 15 12:00:06.508584 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:06.508550 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a378c375-de95-4373-9788-141a4646e636-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-rcdwx\" (UID: \"a378c375-de95-4373-9788-141a4646e636\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-rcdwx" Jul 15 12:00:06.511044 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:06.511019 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a378c375-de95-4373-9788-141a4646e636-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-rcdwx\" (UID: \"a378c375-de95-4373-9788-141a4646e636\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-rcdwx" Jul 15 12:00:06.609600 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:06.609562 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/379eba97-49a2-4f00-afba-0e65b34fd260-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-8fjlk\" (UID: \"379eba97-49a2-4f00-afba-0e65b34fd260\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-8fjlk" Jul 15 12:00:06.609787 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:00:06.609686 2587 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 15 12:00:06.609787 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:00:06.609759 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/379eba97-49a2-4f00-afba-0e65b34fd260-networking-console-plugin-cert podName:379eba97-49a2-4f00-afba-0e65b34fd260 nodeName:}" failed. No retries permitted until 2026-07-15 12:00:22.609723963 +0000 UTC m=+94.067100879 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/379eba97-49a2-4f00-afba-0e65b34fd260-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-8fjlk" (UID: "379eba97-49a2-4f00-afba-0e65b34fd260") : secret "networking-console-plugin-cert" not found Jul 15 12:00:06.670291 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:06.670258 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-rcdwx" Jul 15 12:00:06.797307 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:06.797278 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-rcdwx"] Jul 15 12:00:07.520440 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:07.520403 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-rcdwx" event={"ID":"a378c375-de95-4373-9788-141a4646e636","Type":"ContainerStarted","Data":"34eda9164890c9f447abe6b82070d7dc5a3870289daa74b136a5344787175d63"} Jul 15 12:00:11.535151 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:11.535031 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-rcdwx" event={"ID":"a378c375-de95-4373-9788-141a4646e636","Type":"ContainerStarted","Data":"a7893fb17a6700b6c3501acf723f8a2b680f648e9a4d38e91cd98a4aca85c682"} Jul 15 12:00:11.535151 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:11.535086 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-rcdwx" event={"ID":"a378c375-de95-4373-9788-141a4646e636","Type":"ContainerStarted","Data":"75a9260688bf58044a63b438f28ac2cca629cde10ef7aa260e53a05e9d045d16"} Jul 15 12:00:11.555893 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:11.555839 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-rcdwx" podStartSLOduration=17.887930719 podStartE2EDuration="21.555824174s" totalImagesPullingTime="3.667893455s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 11:59:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:00:11.554207105 +0000 UTC m=+83.011584030" watchObservedRunningTime="2026-07-15 12:00:11.555824174 +0000 UTC m=+83.013201095" Jul 15 12:00:13.258416 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:13.258379 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b1be73b4-e61c-4c4b-9830-749d3c5a544b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8j9km\" (UID: \"b1be73b4-e61c-4c4b-9830-749d3c5a544b\") " pod="openshift-insights/insights-runtime-extractor-8j9km" Jul 15 12:00:13.258799 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:13.258448 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b1be73b4-e61c-4c4b-9830-749d3c5a544b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8j9km\" (UID: \"b1be73b4-e61c-4c4b-9830-749d3c5a544b\") " pod="openshift-insights/insights-runtime-extractor-8j9km" Jul 15 12:00:13.258932 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:13.258905 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b1be73b4-e61c-4c4b-9830-749d3c5a544b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8j9km\" (UID: \"b1be73b4-e61c-4c4b-9830-749d3c5a544b\") " pod="openshift-insights/insights-runtime-extractor-8j9km" Jul 15 12:00:13.260655 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:13.260638 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b1be73b4-e61c-4c4b-9830-749d3c5a544b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8j9km\" (UID: \"b1be73b4-e61c-4c4b-9830-749d3c5a544b\") " pod="openshift-insights/insights-runtime-extractor-8j9km" Jul 15 12:00:13.352873 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:13.352823 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8j9km" Jul 15 12:00:13.478534 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:13.478498 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8j9km"] Jul 15 12:00:13.481252 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:00:13.481225 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1be73b4_e61c_4c4b_9830_749d3c5a544b.slice/crio-0703a8fd56bed11065381f78adf377fec2f775abcd2306b7452d0b2301b7fef3 WatchSource:0}: Error finding container 0703a8fd56bed11065381f78adf377fec2f775abcd2306b7452d0b2301b7fef3: Status 404 returned error can't find the container with id 0703a8fd56bed11065381f78adf377fec2f775abcd2306b7452d0b2301b7fef3 Jul 15 12:00:13.542258 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:13.542223 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8j9km" event={"ID":"b1be73b4-e61c-4c4b-9830-749d3c5a544b","Type":"ContainerStarted","Data":"0703a8fd56bed11065381f78adf377fec2f775abcd2306b7452d0b2301b7fef3"} Jul 15 12:00:15.179665 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:15.179633 2587 scope.go:117] "RemoveContainer" containerID="fbfd08d19c3a05548670f9622a603edc408f8e746278c9ef8673f2cf24a3fe30" Jul 15 12:00:15.551615 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:15.551589 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-4xrn6_85ce5adf-d361-4258-b4ca-7f13d35fe13d/console-operator/1.log" Jul 15 12:00:15.551822 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:15.551676 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-4xrn6" event={"ID":"85ce5adf-d361-4258-b4ca-7f13d35fe13d","Type":"ContainerStarted","Data":"396374c47c98a58a83095e797c3f807a0e8ddfa43d1e9db7aff1e5292295b9b9"} Jul 15 12:00:15.552017 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:15.551994 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-4xrn6" Jul 15 12:00:15.574144 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:15.574086 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-4xrn6" podStartSLOduration=23.223593448 podStartE2EDuration="25.57406981s" totalImagesPullingTime="2.350476362s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 11:59:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:00:15.5728965 +0000 UTC m=+87.030273447" watchObservedRunningTime="2026-07-15 12:00:15.57406981 +0000 UTC m=+87.031446735" Jul 15 12:00:15.994197 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:15.994115 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-4xrn6" Jul 15 12:00:18.562391 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:18.562359 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8j9km" event={"ID":"b1be73b4-e61c-4c4b-9830-749d3c5a544b","Type":"ContainerStarted","Data":"9a8e716a0b6ec26d259cbc48aacdafa36518b70bd8f49b7d497474e6078fac10"} Jul 15 12:00:21.698274 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:21.698038 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-7kt4q"] Jul 15 12:00:21.723333 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:21.723288 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-7kt4q"] Jul 15 12:00:21.723486 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:21.723389 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-7kt4q" Jul 15 12:00:21.726756 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:21.726446 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-mtpjm\"" Jul 15 12:00:21.726756 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:21.726659 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 15 12:00:21.726756 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:21.726451 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 15 12:00:21.829197 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:21.829159 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdgl2\" (UniqueName: \"kubernetes.io/projected/56b02112-6aa5-42dd-aeab-0a296a6314b7-kube-api-access-sdgl2\") pod \"downloads-68c9cbb497-7kt4q\" (UID: \"56b02112-6aa5-42dd-aeab-0a296a6314b7\") " pod="openshift-console/downloads-68c9cbb497-7kt4q" Jul 15 12:00:21.930659 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:21.930623 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdgl2\" (UniqueName: \"kubernetes.io/projected/56b02112-6aa5-42dd-aeab-0a296a6314b7-kube-api-access-sdgl2\") pod \"downloads-68c9cbb497-7kt4q\" (UID: \"56b02112-6aa5-42dd-aeab-0a296a6314b7\") " pod="openshift-console/downloads-68c9cbb497-7kt4q" Jul 15 12:00:21.941867 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:21.941836 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdgl2\" (UniqueName: \"kubernetes.io/projected/56b02112-6aa5-42dd-aeab-0a296a6314b7-kube-api-access-sdgl2\") pod \"downloads-68c9cbb497-7kt4q\" (UID: \"56b02112-6aa5-42dd-aeab-0a296a6314b7\") " pod="openshift-console/downloads-68c9cbb497-7kt4q" Jul 15 12:00:22.036150 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:22.036049 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-7kt4q" Jul 15 12:00:22.636989 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:22.636958 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/379eba97-49a2-4f00-afba-0e65b34fd260-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-8fjlk\" (UID: \"379eba97-49a2-4f00-afba-0e65b34fd260\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-8fjlk" Jul 15 12:00:22.639071 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:22.639052 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/379eba97-49a2-4f00-afba-0e65b34fd260-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-8fjlk\" (UID: \"379eba97-49a2-4f00-afba-0e65b34fd260\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-8fjlk" Jul 15 12:00:22.645853 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:22.645827 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-7kt4q"] Jul 15 12:00:22.648821 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:00:22.648797 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56b02112_6aa5_42dd_aeab_0a296a6314b7.slice/crio-89ec09a7f3a675f845bdff29f7b6c55611dfda4e18be741f079835e3d66dd31e WatchSource:0}: Error finding container 89ec09a7f3a675f845bdff29f7b6c55611dfda4e18be741f079835e3d66dd31e: Status 404 returned error can't find the container with id 89ec09a7f3a675f845bdff29f7b6c55611dfda4e18be741f079835e3d66dd31e Jul 15 12:00:22.663480 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:22.663449 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-8fjlk" Jul 15 12:00:22.786508 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:22.786432 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-8fjlk"] Jul 15 12:00:22.789399 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:00:22.789370 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod379eba97_49a2_4f00_afba_0e65b34fd260.slice/crio-70f3270337badbbb19757a91174ec29d543a634fe384d3e14da8dbf28d5487ab WatchSource:0}: Error finding container 70f3270337badbbb19757a91174ec29d543a634fe384d3e14da8dbf28d5487ab: Status 404 returned error can't find the container with id 70f3270337badbbb19757a91174ec29d543a634fe384d3e14da8dbf28d5487ab Jul 15 12:00:23.582458 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:23.582406 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8j9km" event={"ID":"b1be73b4-e61c-4c4b-9830-749d3c5a544b","Type":"ContainerStarted","Data":"81ca4f310bb297aa3c3da713ac0fc6f1479190fc5271bbc8eb3cc5cb4e1d2f1e"} Jul 15 12:00:23.583929 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:23.583900 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-8fjlk" event={"ID":"379eba97-49a2-4f00-afba-0e65b34fd260","Type":"ContainerStarted","Data":"70f3270337badbbb19757a91174ec29d543a634fe384d3e14da8dbf28d5487ab"} Jul 15 12:00:23.585201 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:23.585176 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-7kt4q" event={"ID":"56b02112-6aa5-42dd-aeab-0a296a6314b7","Type":"ContainerStarted","Data":"89ec09a7f3a675f845bdff29f7b6c55611dfda4e18be741f079835e3d66dd31e"} Jul 15 12:00:24.591225 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:24.591184 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-8fjlk" event={"ID":"379eba97-49a2-4f00-afba-0e65b34fd260","Type":"ContainerStarted","Data":"15f349ca06b8feae590779a7c6f8f0c78247278c6fc2fd61b430df4d97d24ecb"} Jul 15 12:00:24.611870 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:24.611810 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-8fjlk" podStartSLOduration=33.173193797 podStartE2EDuration="34.611787945s" totalImagesPullingTime="1.438594148s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 11:59:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:00:24.610910916 +0000 UTC m=+96.068287841" watchObservedRunningTime="2026-07-15 12:00:24.611787945 +0000 UTC m=+96.069164871" Jul 15 12:00:26.599012 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:26.598979 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8j9km" event={"ID":"b1be73b4-e61c-4c4b-9830-749d3c5a544b","Type":"ContainerStarted","Data":"b9c9ad1a2605501966630342aeeaba5884aacacaebe78453f5739685cea75fde"} Jul 15 12:00:26.625095 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:26.625043 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-8j9km" podStartSLOduration=17.129638494 podStartE2EDuration="29.625020664s" totalImagesPullingTime="12.49538217s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 11:59:57 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:00:26.624422576 +0000 UTC m=+98.081799521" watchObservedRunningTime="2026-07-15 12:00:26.625020664 +0000 UTC m=+98.082397590" Jul 15 12:00:27.241111 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:27.241076 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bbrmm"] Jul 15 12:00:27.244177 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:27.244160 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bbrmm" Jul 15 12:00:27.246911 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:27.246891 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 15 12:00:27.247175 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:27.247160 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-qts45\"" Jul 15 12:00:27.253306 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:27.253284 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bbrmm"] Jul 15 12:00:27.382201 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:27.382171 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a4a87ac3-6521-499a-8566-67c0f2b073e4-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-bbrmm\" (UID: \"a4a87ac3-6521-499a-8566-67c0f2b073e4\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bbrmm" Jul 15 12:00:27.482899 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:27.482859 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a4a87ac3-6521-499a-8566-67c0f2b073e4-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-bbrmm\" (UID: \"a4a87ac3-6521-499a-8566-67c0f2b073e4\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bbrmm" Jul 15 12:00:27.483067 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:00:27.482982 2587 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 15 12:00:27.483067 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:00:27.483045 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4a87ac3-6521-499a-8566-67c0f2b073e4-tls-certificates podName:a4a87ac3-6521-499a-8566-67c0f2b073e4 nodeName:}" failed. No retries permitted until 2026-07-15 12:00:27.983030846 +0000 UTC m=+99.440407749 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/a4a87ac3-6521-499a-8566-67c0f2b073e4-tls-certificates") pod "prometheus-operator-admission-webhook-77d98bc786-bbrmm" (UID: "a4a87ac3-6521-499a-8566-67c0f2b073e4") : secret "prometheus-operator-admission-webhook-tls" not found Jul 15 12:00:27.784996 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:27.784950 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8db3f4f8-7a28-477d-9a5b-8df04d5a282a-metrics-tls\") pod \"dns-default-k5m9c\" (UID: \"8db3f4f8-7a28-477d-9a5b-8df04d5a282a\") " pod="openshift-dns/dns-default-k5m9c" Jul 15 12:00:27.784996 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:27.784999 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-registry-tls\") pod \"image-registry-b788d9845-z5cx7\" (UID: \"f6e2e61d-a8c4-475d-a43a-fe32868b16ba\") " pod="openshift-image-registry/image-registry-b788d9845-z5cx7" Jul 15 12:00:27.787319 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:27.787283 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8db3f4f8-7a28-477d-9a5b-8df04d5a282a-metrics-tls\") pod \"dns-default-k5m9c\" (UID: \"8db3f4f8-7a28-477d-9a5b-8df04d5a282a\") " pod="openshift-dns/dns-default-k5m9c" Jul 15 12:00:27.787449 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:27.787338 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-registry-tls\") pod \"image-registry-b788d9845-z5cx7\" (UID: \"f6e2e61d-a8c4-475d-a43a-fe32868b16ba\") " pod="openshift-image-registry/image-registry-b788d9845-z5cx7" Jul 15 12:00:27.795127 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:27.795106 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bqsqx\"" Jul 15 12:00:27.802438 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:27.802417 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-k5m9c" Jul 15 12:00:27.936345 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:27.936308 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-k5m9c"] Jul 15 12:00:27.939604 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:00:27.939562 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8db3f4f8_7a28_477d_9a5b_8df04d5a282a.slice/crio-57228cf0fb3c0eaafa8222a8b3424dc133f3e9beefb9ff9c65eba5cc446693e4 WatchSource:0}: Error finding container 57228cf0fb3c0eaafa8222a8b3424dc133f3e9beefb9ff9c65eba5cc446693e4: Status 404 returned error can't find the container with id 57228cf0fb3c0eaafa8222a8b3424dc133f3e9beefb9ff9c65eba5cc446693e4 Jul 15 12:00:27.987523 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:27.987481 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a4a87ac3-6521-499a-8566-67c0f2b073e4-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-bbrmm\" (UID: \"a4a87ac3-6521-499a-8566-67c0f2b073e4\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bbrmm" Jul 15 12:00:27.990136 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:27.990113 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a4a87ac3-6521-499a-8566-67c0f2b073e4-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-bbrmm\" (UID: \"a4a87ac3-6521-499a-8566-67c0f2b073e4\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bbrmm" Jul 15 12:00:28.083484 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:28.083451 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-vl9cw\"" Jul 15 12:00:28.091061 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:28.091030 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b788d9845-z5cx7" Jul 15 12:00:28.153275 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:28.153018 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bbrmm" Jul 15 12:00:28.248750 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:28.248698 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-b788d9845-z5cx7"] Jul 15 12:00:28.252321 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:00:28.252287 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6e2e61d_a8c4_475d_a43a_fe32868b16ba.slice/crio-226c25d142b27974be391371e4fbf4cdb9c0bbbbf41d0e057c661cd33a3af631 WatchSource:0}: Error finding container 226c25d142b27974be391371e4fbf4cdb9c0bbbbf41d0e057c661cd33a3af631: Status 404 returned error can't find the container with id 226c25d142b27974be391371e4fbf4cdb9c0bbbbf41d0e057c661cd33a3af631 Jul 15 12:00:28.307999 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:28.307962 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bbrmm"] Jul 15 12:00:28.311837 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:00:28.311803 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4a87ac3_6521_499a_8566_67c0f2b073e4.slice/crio-ec9c7230397a539fc5436f5e3f108586a98b131e868800fb3cd3f58754b6daf8 WatchSource:0}: Error finding container ec9c7230397a539fc5436f5e3f108586a98b131e868800fb3cd3f58754b6daf8: Status 404 returned error can't find the container with id ec9c7230397a539fc5436f5e3f108586a98b131e868800fb3cd3f58754b6daf8 Jul 15 12:00:28.607941 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:28.607902 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bbrmm" event={"ID":"a4a87ac3-6521-499a-8566-67c0f2b073e4","Type":"ContainerStarted","Data":"ec9c7230397a539fc5436f5e3f108586a98b131e868800fb3cd3f58754b6daf8"} Jul 15 12:00:28.610122 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:28.610087 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b788d9845-z5cx7" event={"ID":"f6e2e61d-a8c4-475d-a43a-fe32868b16ba","Type":"ContainerStarted","Data":"220f3ca26f11d0b185463e3bc63aff11e5edc46a6c72d17b259e8f874dc26ef1"} Jul 15 12:00:28.610247 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:28.610144 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b788d9845-z5cx7" event={"ID":"f6e2e61d-a8c4-475d-a43a-fe32868b16ba","Type":"ContainerStarted","Data":"226c25d142b27974be391371e4fbf4cdb9c0bbbbf41d0e057c661cd33a3af631"} Jul 15 12:00:28.610403 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:28.610357 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-b788d9845-z5cx7" Jul 15 12:00:28.611658 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:28.611632 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-k5m9c" event={"ID":"8db3f4f8-7a28-477d-9a5b-8df04d5a282a","Type":"ContainerStarted","Data":"57228cf0fb3c0eaafa8222a8b3424dc133f3e9beefb9ff9c65eba5cc446693e4"} Jul 15 12:00:28.633201 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:28.633142 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-b788d9845-z5cx7" podStartSLOduration=99.633123767 podStartE2EDuration="1m39.633123767s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 11:58:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:00:28.631188072 +0000 UTC m=+100.088565008" watchObservedRunningTime="2026-07-15 12:00:28.633123767 +0000 UTC m=+100.090500691" Jul 15 12:00:30.621963 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:30.621921 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bbrmm" event={"ID":"a4a87ac3-6521-499a-8566-67c0f2b073e4","Type":"ContainerStarted","Data":"95713ff358ba09c4c34c8755fc10111b2bd9aa7136f855e28401964c2b9c5c66"} Jul 15 12:00:30.622521 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:30.622185 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bbrmm" Jul 15 12:00:30.624367 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:30.624160 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-k5m9c" event={"ID":"8db3f4f8-7a28-477d-9a5b-8df04d5a282a","Type":"ContainerStarted","Data":"d6c746a0528b69c65e3cac24d9d4e7e771812b5239ee65fe69fa168e3db6a3c3"} Jul 15 12:00:30.624367 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:30.624211 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-k5m9c" event={"ID":"8db3f4f8-7a28-477d-9a5b-8df04d5a282a","Type":"ContainerStarted","Data":"2bb2692265dcccb0be08be64eea8b8e2195afd27b0ba7e5eac3ccb4dbbda33c3"} Jul 15 12:00:30.624367 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:30.624261 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-k5m9c" Jul 15 12:00:30.628076 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:30.628054 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bbrmm" Jul 15 12:00:30.640867 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:30.640810 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bbrmm" podStartSLOduration=1.763853822 podStartE2EDuration="3.640791323s" totalImagesPullingTime="1.876937501s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:00:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:00:30.639663934 +0000 UTC m=+102.097040860" watchObservedRunningTime="2026-07-15 12:00:30.640791323 +0000 UTC m=+102.098168250" Jul 15 12:00:30.680623 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:30.680589 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7d9c8b4588-89wzq"] Jul 15 12:00:30.684157 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:30.684130 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d9c8b4588-89wzq" Jul 15 12:00:30.686147 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:30.685828 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-k5m9c" podStartSLOduration=65.430792666 podStartE2EDuration="1m7.685811893s" totalImagesPullingTime="2.255019227s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 11:59:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:00:30.685403939 +0000 UTC m=+102.142780865" watchObservedRunningTime="2026-07-15 12:00:30.685811893 +0000 UTC m=+102.143188819" Jul 15 12:00:30.688033 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:30.687443 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 15 12:00:30.688033 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:30.687486 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 15 12:00:30.688033 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:30.687491 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-p4glj\"" Jul 15 12:00:30.688033 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:30.687796 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 15 12:00:30.688033 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:30.687822 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 15 12:00:30.688033 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:30.687796 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 15 12:00:30.692776 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:30.692579 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 15 12:00:30.697058 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:30.697035 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d9c8b4588-89wzq"] Jul 15 12:00:30.808960 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:30.808925 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ea719dc7-8860-4933-bb40-609215906a91-console-config\") pod \"console-7d9c8b4588-89wzq\" (UID: \"ea719dc7-8860-4933-bb40-609215906a91\") " pod="openshift-console/console-7d9c8b4588-89wzq" Jul 15 12:00:30.809141 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:30.808969 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pc7zm\" (UniqueName: \"kubernetes.io/projected/ea719dc7-8860-4933-bb40-609215906a91-kube-api-access-pc7zm\") pod \"console-7d9c8b4588-89wzq\" (UID: \"ea719dc7-8860-4933-bb40-609215906a91\") " pod="openshift-console/console-7d9c8b4588-89wzq" Jul 15 12:00:30.809141 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:30.808998 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ea719dc7-8860-4933-bb40-609215906a91-console-serving-cert\") pod \"console-7d9c8b4588-89wzq\" (UID: \"ea719dc7-8860-4933-bb40-609215906a91\") " pod="openshift-console/console-7d9c8b4588-89wzq" Jul 15 12:00:30.809141 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:30.809024 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ea719dc7-8860-4933-bb40-609215906a91-oauth-serving-cert\") pod \"console-7d9c8b4588-89wzq\" (UID: \"ea719dc7-8860-4933-bb40-609215906a91\") " pod="openshift-console/console-7d9c8b4588-89wzq" Jul 15 12:00:30.809141 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:30.809092 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea719dc7-8860-4933-bb40-609215906a91-trusted-ca-bundle\") pod \"console-7d9c8b4588-89wzq\" (UID: \"ea719dc7-8860-4933-bb40-609215906a91\") " pod="openshift-console/console-7d9c8b4588-89wzq" Jul 15 12:00:30.809141 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:30.809126 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ea719dc7-8860-4933-bb40-609215906a91-service-ca\") pod \"console-7d9c8b4588-89wzq\" (UID: \"ea719dc7-8860-4933-bb40-609215906a91\") " pod="openshift-console/console-7d9c8b4588-89wzq" Jul 15 12:00:30.809370 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:30.809150 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ea719dc7-8860-4933-bb40-609215906a91-console-oauth-config\") pod \"console-7d9c8b4588-89wzq\" (UID: \"ea719dc7-8860-4933-bb40-609215906a91\") " pod="openshift-console/console-7d9c8b4588-89wzq" Jul 15 12:00:30.910660 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:30.910558 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea719dc7-8860-4933-bb40-609215906a91-trusted-ca-bundle\") pod \"console-7d9c8b4588-89wzq\" (UID: \"ea719dc7-8860-4933-bb40-609215906a91\") " pod="openshift-console/console-7d9c8b4588-89wzq" Jul 15 12:00:30.910660 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:30.910608 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ea719dc7-8860-4933-bb40-609215906a91-service-ca\") pod \"console-7d9c8b4588-89wzq\" (UID: \"ea719dc7-8860-4933-bb40-609215906a91\") " pod="openshift-console/console-7d9c8b4588-89wzq" Jul 15 12:00:30.910660 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:30.910638 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ea719dc7-8860-4933-bb40-609215906a91-console-oauth-config\") pod \"console-7d9c8b4588-89wzq\" (UID: \"ea719dc7-8860-4933-bb40-609215906a91\") " pod="openshift-console/console-7d9c8b4588-89wzq" Jul 15 12:00:30.910998 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:30.910707 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ea719dc7-8860-4933-bb40-609215906a91-console-config\") pod \"console-7d9c8b4588-89wzq\" (UID: \"ea719dc7-8860-4933-bb40-609215906a91\") " pod="openshift-console/console-7d9c8b4588-89wzq" Jul 15 12:00:30.910998 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:30.910766 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pc7zm\" (UniqueName: \"kubernetes.io/projected/ea719dc7-8860-4933-bb40-609215906a91-kube-api-access-pc7zm\") pod \"console-7d9c8b4588-89wzq\" (UID: \"ea719dc7-8860-4933-bb40-609215906a91\") " pod="openshift-console/console-7d9c8b4588-89wzq" Jul 15 12:00:30.910998 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:30.910808 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ea719dc7-8860-4933-bb40-609215906a91-console-serving-cert\") pod \"console-7d9c8b4588-89wzq\" (UID: \"ea719dc7-8860-4933-bb40-609215906a91\") " pod="openshift-console/console-7d9c8b4588-89wzq" Jul 15 12:00:30.910998 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:30.910834 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ea719dc7-8860-4933-bb40-609215906a91-oauth-serving-cert\") pod \"console-7d9c8b4588-89wzq\" (UID: \"ea719dc7-8860-4933-bb40-609215906a91\") " pod="openshift-console/console-7d9c8b4588-89wzq" Jul 15 12:00:30.911515 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:30.911484 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ea719dc7-8860-4933-bb40-609215906a91-service-ca\") pod \"console-7d9c8b4588-89wzq\" (UID: \"ea719dc7-8860-4933-bb40-609215906a91\") " pod="openshift-console/console-7d9c8b4588-89wzq" Jul 15 12:00:30.911657 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:30.911629 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ea719dc7-8860-4933-bb40-609215906a91-console-config\") pod \"console-7d9c8b4588-89wzq\" (UID: \"ea719dc7-8860-4933-bb40-609215906a91\") " pod="openshift-console/console-7d9c8b4588-89wzq" Jul 15 12:00:30.911787 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:30.911721 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea719dc7-8860-4933-bb40-609215906a91-trusted-ca-bundle\") pod \"console-7d9c8b4588-89wzq\" (UID: \"ea719dc7-8860-4933-bb40-609215906a91\") " pod="openshift-console/console-7d9c8b4588-89wzq" Jul 15 12:00:30.912003 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:30.911979 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ea719dc7-8860-4933-bb40-609215906a91-oauth-serving-cert\") pod \"console-7d9c8b4588-89wzq\" (UID: \"ea719dc7-8860-4933-bb40-609215906a91\") " pod="openshift-console/console-7d9c8b4588-89wzq" Jul 15 12:00:30.913840 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:30.913821 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ea719dc7-8860-4933-bb40-609215906a91-console-oauth-config\") pod \"console-7d9c8b4588-89wzq\" (UID: \"ea719dc7-8860-4933-bb40-609215906a91\") " pod="openshift-console/console-7d9c8b4588-89wzq" Jul 15 12:00:30.914126 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:30.914092 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ea719dc7-8860-4933-bb40-609215906a91-console-serving-cert\") pod \"console-7d9c8b4588-89wzq\" (UID: \"ea719dc7-8860-4933-bb40-609215906a91\") " pod="openshift-console/console-7d9c8b4588-89wzq" Jul 15 12:00:30.920892 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:30.920868 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pc7zm\" (UniqueName: \"kubernetes.io/projected/ea719dc7-8860-4933-bb40-609215906a91-kube-api-access-pc7zm\") pod \"console-7d9c8b4588-89wzq\" (UID: \"ea719dc7-8860-4933-bb40-609215906a91\") " pod="openshift-console/console-7d9c8b4588-89wzq" Jul 15 12:00:30.996677 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:30.996634 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d9c8b4588-89wzq" Jul 15 12:00:31.141286 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:31.141251 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d9c8b4588-89wzq"] Jul 15 12:00:31.144283 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:00:31.144252 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea719dc7_8860_4933_bb40_609215906a91.slice/crio-047a4249a902bc4a166f373e33be395d4197f4d65b6b31f62b95bafdfc390ebb WatchSource:0}: Error finding container 047a4249a902bc4a166f373e33be395d4197f4d65b6b31f62b95bafdfc390ebb: Status 404 returned error can't find the container with id 047a4249a902bc4a166f373e33be395d4197f4d65b6b31f62b95bafdfc390ebb Jul 15 12:00:31.296109 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:31.296069 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-9pvkq"] Jul 15 12:00:31.326096 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:31.326063 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-9pvkq"] Jul 15 12:00:31.326280 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:31.326177 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-9pvkq" Jul 15 12:00:31.329150 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:31.329099 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 15 12:00:31.329392 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:31.329375 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-jqlg5\"" Jul 15 12:00:31.329498 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:31.329430 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 15 12:00:31.330563 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:31.330421 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 15 12:00:31.330563 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:31.330444 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 15 12:00:31.330563 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:31.330483 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 15 12:00:31.415721 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:31.415680 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d1c5ae78-48ab-4245-912c-3497e6c503b5-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-9pvkq\" (UID: \"d1c5ae78-48ab-4245-912c-3497e6c503b5\") " pod="openshift-monitoring/prometheus-operator-64785b6675-9pvkq" Jul 15 12:00:31.415890 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:31.415763 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nr82c\" (UniqueName: \"kubernetes.io/projected/d1c5ae78-48ab-4245-912c-3497e6c503b5-kube-api-access-nr82c\") pod \"prometheus-operator-64785b6675-9pvkq\" (UID: \"d1c5ae78-48ab-4245-912c-3497e6c503b5\") " pod="openshift-monitoring/prometheus-operator-64785b6675-9pvkq" Jul 15 12:00:31.415890 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:31.415790 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d1c5ae78-48ab-4245-912c-3497e6c503b5-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-9pvkq\" (UID: \"d1c5ae78-48ab-4245-912c-3497e6c503b5\") " pod="openshift-monitoring/prometheus-operator-64785b6675-9pvkq" Jul 15 12:00:31.415890 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:31.415876 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d1c5ae78-48ab-4245-912c-3497e6c503b5-metrics-client-ca\") pod \"prometheus-operator-64785b6675-9pvkq\" (UID: \"d1c5ae78-48ab-4245-912c-3497e6c503b5\") " pod="openshift-monitoring/prometheus-operator-64785b6675-9pvkq" Jul 15 12:00:31.504123 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:31.504093 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-qf4ll" Jul 15 12:00:31.516714 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:31.516684 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d1c5ae78-48ab-4245-912c-3497e6c503b5-metrics-client-ca\") pod \"prometheus-operator-64785b6675-9pvkq\" (UID: \"d1c5ae78-48ab-4245-912c-3497e6c503b5\") " pod="openshift-monitoring/prometheus-operator-64785b6675-9pvkq" Jul 15 12:00:31.516916 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:31.516756 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d1c5ae78-48ab-4245-912c-3497e6c503b5-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-9pvkq\" (UID: \"d1c5ae78-48ab-4245-912c-3497e6c503b5\") " pod="openshift-monitoring/prometheus-operator-64785b6675-9pvkq" Jul 15 12:00:31.516916 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:31.516807 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nr82c\" (UniqueName: \"kubernetes.io/projected/d1c5ae78-48ab-4245-912c-3497e6c503b5-kube-api-access-nr82c\") pod \"prometheus-operator-64785b6675-9pvkq\" (UID: \"d1c5ae78-48ab-4245-912c-3497e6c503b5\") " pod="openshift-monitoring/prometheus-operator-64785b6675-9pvkq" Jul 15 12:00:31.516916 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:31.516840 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d1c5ae78-48ab-4245-912c-3497e6c503b5-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-9pvkq\" (UID: \"d1c5ae78-48ab-4245-912c-3497e6c503b5\") " pod="openshift-monitoring/prometheus-operator-64785b6675-9pvkq" Jul 15 12:00:31.517079 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:00:31.516967 2587 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 15 12:00:31.517079 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:00:31.517032 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1c5ae78-48ab-4245-912c-3497e6c503b5-prometheus-operator-tls podName:d1c5ae78-48ab-4245-912c-3497e6c503b5 nodeName:}" failed. No retries permitted until 2026-07-15 12:00:32.017013289 +0000 UTC m=+103.474390195 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/d1c5ae78-48ab-4245-912c-3497e6c503b5-prometheus-operator-tls") pod "prometheus-operator-64785b6675-9pvkq" (UID: "d1c5ae78-48ab-4245-912c-3497e6c503b5") : secret "prometheus-operator-tls" not found Jul 15 12:00:31.517402 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:31.517371 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d1c5ae78-48ab-4245-912c-3497e6c503b5-metrics-client-ca\") pod \"prometheus-operator-64785b6675-9pvkq\" (UID: \"d1c5ae78-48ab-4245-912c-3497e6c503b5\") " pod="openshift-monitoring/prometheus-operator-64785b6675-9pvkq" Jul 15 12:00:31.519563 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:31.519535 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d1c5ae78-48ab-4245-912c-3497e6c503b5-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-9pvkq\" (UID: \"d1c5ae78-48ab-4245-912c-3497e6c503b5\") " pod="openshift-monitoring/prometheus-operator-64785b6675-9pvkq" Jul 15 12:00:31.529415 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:31.529360 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nr82c\" (UniqueName: \"kubernetes.io/projected/d1c5ae78-48ab-4245-912c-3497e6c503b5-kube-api-access-nr82c\") pod \"prometheus-operator-64785b6675-9pvkq\" (UID: \"d1c5ae78-48ab-4245-912c-3497e6c503b5\") " pod="openshift-monitoring/prometheus-operator-64785b6675-9pvkq" Jul 15 12:00:31.631873 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:31.631291 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d9c8b4588-89wzq" event={"ID":"ea719dc7-8860-4933-bb40-609215906a91","Type":"ContainerStarted","Data":"047a4249a902bc4a166f373e33be395d4197f4d65b6b31f62b95bafdfc390ebb"} Jul 15 12:00:32.020959 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:32.020870 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d1c5ae78-48ab-4245-912c-3497e6c503b5-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-9pvkq\" (UID: \"d1c5ae78-48ab-4245-912c-3497e6c503b5\") " pod="openshift-monitoring/prometheus-operator-64785b6675-9pvkq" Jul 15 12:00:32.023631 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:32.023599 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d1c5ae78-48ab-4245-912c-3497e6c503b5-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-9pvkq\" (UID: \"d1c5ae78-48ab-4245-912c-3497e6c503b5\") " pod="openshift-monitoring/prometheus-operator-64785b6675-9pvkq" Jul 15 12:00:32.236980 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:32.236919 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-9pvkq" Jul 15 12:00:32.409692 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:32.409616 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-9pvkq"] Jul 15 12:00:32.412919 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:00:32.412876 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1c5ae78_48ab_4245_912c_3497e6c503b5.slice/crio-db160a346764b8e34a2347f83478e73468013c7fb5a422a1619c330ad3dc7263 WatchSource:0}: Error finding container db160a346764b8e34a2347f83478e73468013c7fb5a422a1619c330ad3dc7263: Status 404 returned error can't find the container with id db160a346764b8e34a2347f83478e73468013c7fb5a422a1619c330ad3dc7263 Jul 15 12:00:32.637351 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:32.637260 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-9pvkq" event={"ID":"d1c5ae78-48ab-4245-912c-3497e6c503b5","Type":"ContainerStarted","Data":"db160a346764b8e34a2347f83478e73468013c7fb5a422a1619c330ad3dc7263"} Jul 15 12:00:34.647173 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:34.646992 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d9c8b4588-89wzq" event={"ID":"ea719dc7-8860-4933-bb40-609215906a91","Type":"ContainerStarted","Data":"9ef327aa6cb0712377a8ebf8209b86c1f868e7ebadd0d28cc82644b9ebff4c68"} Jul 15 12:00:34.668028 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:34.667969 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7d9c8b4588-89wzq" podStartSLOduration=1.353711601 podStartE2EDuration="4.667946941s" totalImagesPullingTime="3.31423534s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:00:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:00:34.667129851 +0000 UTC m=+106.124506776" watchObservedRunningTime="2026-07-15 12:00:34.667946941 +0000 UTC m=+106.125323868" Jul 15 12:00:35.655001 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:35.654700 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-9pvkq" event={"ID":"d1c5ae78-48ab-4245-912c-3497e6c503b5","Type":"ContainerStarted","Data":"55baaea5d87252c59b089e71176bc9f8c30c544c212c561aad4e148b0f250cf8"} Jul 15 12:00:35.655001 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:35.654780 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-9pvkq" event={"ID":"d1c5ae78-48ab-4245-912c-3497e6c503b5","Type":"ContainerStarted","Data":"215b4e976838a09c74b00047839cc644dafb270f14662da297de3b92eeb9b07d"} Jul 15 12:00:35.678347 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:35.675192 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-9pvkq" podStartSLOduration=1.600756536 podStartE2EDuration="4.675173145s" totalImagesPullingTime="3.074416609s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:00:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:00:35.674257124 +0000 UTC m=+107.131634052" watchObservedRunningTime="2026-07-15 12:00:35.675173145 +0000 UTC m=+107.132550072" Jul 15 12:00:37.648783 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:37.648743 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-7vz7j"] Jul 15 12:00:37.653348 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:37.653313 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7vz7j" Jul 15 12:00:37.656325 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:37.656298 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 15 12:00:37.656325 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:37.656318 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 15 12:00:37.656507 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:37.656298 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 15 12:00:37.656643 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:37.656622 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-262w7\"" Jul 15 12:00:37.771931 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:37.771892 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cec409e8-1241-411e-834f-7d82377b1f8b-node-exporter-wtmp\") pod \"node-exporter-7vz7j\" (UID: \"cec409e8-1241-411e-834f-7d82377b1f8b\") " pod="openshift-monitoring/node-exporter-7vz7j" Jul 15 12:00:37.771931 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:37.771941 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cec409e8-1241-411e-834f-7d82377b1f8b-node-exporter-textfile\") pod \"node-exporter-7vz7j\" (UID: \"cec409e8-1241-411e-834f-7d82377b1f8b\") " pod="openshift-monitoring/node-exporter-7vz7j" Jul 15 12:00:37.772160 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:37.771963 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cec409e8-1241-411e-834f-7d82377b1f8b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7vz7j\" (UID: \"cec409e8-1241-411e-834f-7d82377b1f8b\") " pod="openshift-monitoring/node-exporter-7vz7j" Jul 15 12:00:37.772160 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:37.772053 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cec409e8-1241-411e-834f-7d82377b1f8b-root\") pod \"node-exporter-7vz7j\" (UID: \"cec409e8-1241-411e-834f-7d82377b1f8b\") " pod="openshift-monitoring/node-exporter-7vz7j" Jul 15 12:00:37.772160 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:37.772091 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnmpk\" (UniqueName: \"kubernetes.io/projected/cec409e8-1241-411e-834f-7d82377b1f8b-kube-api-access-bnmpk\") pod \"node-exporter-7vz7j\" (UID: \"cec409e8-1241-411e-834f-7d82377b1f8b\") " pod="openshift-monitoring/node-exporter-7vz7j" Jul 15 12:00:37.772160 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:37.772118 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cec409e8-1241-411e-834f-7d82377b1f8b-metrics-client-ca\") pod \"node-exporter-7vz7j\" (UID: \"cec409e8-1241-411e-834f-7d82377b1f8b\") " pod="openshift-monitoring/node-exporter-7vz7j" Jul 15 12:00:37.772160 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:37.772154 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cec409e8-1241-411e-834f-7d82377b1f8b-node-exporter-tls\") pod \"node-exporter-7vz7j\" (UID: \"cec409e8-1241-411e-834f-7d82377b1f8b\") " pod="openshift-monitoring/node-exporter-7vz7j" Jul 15 12:00:37.772348 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:37.772218 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cec409e8-1241-411e-834f-7d82377b1f8b-sys\") pod \"node-exporter-7vz7j\" (UID: \"cec409e8-1241-411e-834f-7d82377b1f8b\") " pod="openshift-monitoring/node-exporter-7vz7j" Jul 15 12:00:37.772348 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:37.772256 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cec409e8-1241-411e-834f-7d82377b1f8b-node-exporter-accelerators-collector-config\") pod \"node-exporter-7vz7j\" (UID: \"cec409e8-1241-411e-834f-7d82377b1f8b\") " pod="openshift-monitoring/node-exporter-7vz7j" Jul 15 12:00:37.874214 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:37.873501 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cec409e8-1241-411e-834f-7d82377b1f8b-root\") pod \"node-exporter-7vz7j\" (UID: \"cec409e8-1241-411e-834f-7d82377b1f8b\") " pod="openshift-monitoring/node-exporter-7vz7j" Jul 15 12:00:37.874214 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:37.873551 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnmpk\" (UniqueName: \"kubernetes.io/projected/cec409e8-1241-411e-834f-7d82377b1f8b-kube-api-access-bnmpk\") pod \"node-exporter-7vz7j\" (UID: \"cec409e8-1241-411e-834f-7d82377b1f8b\") " pod="openshift-monitoring/node-exporter-7vz7j" Jul 15 12:00:37.874214 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:37.873584 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cec409e8-1241-411e-834f-7d82377b1f8b-metrics-client-ca\") pod \"node-exporter-7vz7j\" (UID: \"cec409e8-1241-411e-834f-7d82377b1f8b\") " pod="openshift-monitoring/node-exporter-7vz7j" Jul 15 12:00:37.874214 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:37.873609 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cec409e8-1241-411e-834f-7d82377b1f8b-node-exporter-tls\") pod \"node-exporter-7vz7j\" (UID: \"cec409e8-1241-411e-834f-7d82377b1f8b\") " pod="openshift-monitoring/node-exporter-7vz7j" Jul 15 12:00:37.874214 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:37.873650 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cec409e8-1241-411e-834f-7d82377b1f8b-sys\") pod \"node-exporter-7vz7j\" (UID: \"cec409e8-1241-411e-834f-7d82377b1f8b\") " pod="openshift-monitoring/node-exporter-7vz7j" Jul 15 12:00:37.874214 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:00:37.873959 2587 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 15 12:00:37.874214 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:37.874166 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cec409e8-1241-411e-834f-7d82377b1f8b-root\") pod \"node-exporter-7vz7j\" (UID: \"cec409e8-1241-411e-834f-7d82377b1f8b\") " pod="openshift-monitoring/node-exporter-7vz7j" Jul 15 12:00:37.874679 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:37.874324 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cec409e8-1241-411e-834f-7d82377b1f8b-sys\") pod \"node-exporter-7vz7j\" (UID: \"cec409e8-1241-411e-834f-7d82377b1f8b\") " pod="openshift-monitoring/node-exporter-7vz7j" Jul 15 12:00:37.874679 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:37.874390 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cec409e8-1241-411e-834f-7d82377b1f8b-metrics-client-ca\") pod \"node-exporter-7vz7j\" (UID: \"cec409e8-1241-411e-834f-7d82377b1f8b\") " pod="openshift-monitoring/node-exporter-7vz7j" Jul 15 12:00:37.874679 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:37.874419 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cec409e8-1241-411e-834f-7d82377b1f8b-node-exporter-accelerators-collector-config\") pod \"node-exporter-7vz7j\" (UID: \"cec409e8-1241-411e-834f-7d82377b1f8b\") " pod="openshift-monitoring/node-exporter-7vz7j" Jul 15 12:00:37.874679 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:00:37.874445 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cec409e8-1241-411e-834f-7d82377b1f8b-node-exporter-tls podName:cec409e8-1241-411e-834f-7d82377b1f8b nodeName:}" failed. No retries permitted until 2026-07-15 12:00:38.374423536 +0000 UTC m=+109.831800452 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/cec409e8-1241-411e-834f-7d82377b1f8b-node-exporter-tls") pod "node-exporter-7vz7j" (UID: "cec409e8-1241-411e-834f-7d82377b1f8b") : secret "node-exporter-tls" not found Jul 15 12:00:37.874679 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:37.874501 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cec409e8-1241-411e-834f-7d82377b1f8b-node-exporter-wtmp\") pod \"node-exporter-7vz7j\" (UID: \"cec409e8-1241-411e-834f-7d82377b1f8b\") " pod="openshift-monitoring/node-exporter-7vz7j" Jul 15 12:00:37.874679 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:37.874537 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cec409e8-1241-411e-834f-7d82377b1f8b-node-exporter-textfile\") pod \"node-exporter-7vz7j\" (UID: \"cec409e8-1241-411e-834f-7d82377b1f8b\") " pod="openshift-monitoring/node-exporter-7vz7j" Jul 15 12:00:37.874679 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:37.874565 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cec409e8-1241-411e-834f-7d82377b1f8b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7vz7j\" (UID: \"cec409e8-1241-411e-834f-7d82377b1f8b\") " pod="openshift-monitoring/node-exporter-7vz7j" Jul 15 12:00:37.875068 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:37.874953 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cec409e8-1241-411e-834f-7d82377b1f8b-node-exporter-accelerators-collector-config\") pod \"node-exporter-7vz7j\" (UID: \"cec409e8-1241-411e-834f-7d82377b1f8b\") " pod="openshift-monitoring/node-exporter-7vz7j" Jul 15 12:00:37.875068 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:37.875058 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cec409e8-1241-411e-834f-7d82377b1f8b-node-exporter-wtmp\") pod \"node-exporter-7vz7j\" (UID: \"cec409e8-1241-411e-834f-7d82377b1f8b\") " pod="openshift-monitoring/node-exporter-7vz7j" Jul 15 12:00:37.875242 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:37.875205 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cec409e8-1241-411e-834f-7d82377b1f8b-node-exporter-textfile\") pod \"node-exporter-7vz7j\" (UID: \"cec409e8-1241-411e-834f-7d82377b1f8b\") " pod="openshift-monitoring/node-exporter-7vz7j" Jul 15 12:00:37.877363 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:37.877343 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cec409e8-1241-411e-834f-7d82377b1f8b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7vz7j\" (UID: \"cec409e8-1241-411e-834f-7d82377b1f8b\") " pod="openshift-monitoring/node-exporter-7vz7j" Jul 15 12:00:37.882865 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:37.882819 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnmpk\" (UniqueName: \"kubernetes.io/projected/cec409e8-1241-411e-834f-7d82377b1f8b-kube-api-access-bnmpk\") pod \"node-exporter-7vz7j\" (UID: \"cec409e8-1241-411e-834f-7d82377b1f8b\") " pod="openshift-monitoring/node-exporter-7vz7j" Jul 15 12:00:38.379922 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:38.379885 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cec409e8-1241-411e-834f-7d82377b1f8b-node-exporter-tls\") pod \"node-exporter-7vz7j\" (UID: \"cec409e8-1241-411e-834f-7d82377b1f8b\") " pod="openshift-monitoring/node-exporter-7vz7j" Jul 15 12:00:38.382532 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:38.382498 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cec409e8-1241-411e-834f-7d82377b1f8b-node-exporter-tls\") pod \"node-exporter-7vz7j\" (UID: \"cec409e8-1241-411e-834f-7d82377b1f8b\") " pod="openshift-monitoring/node-exporter-7vz7j" Jul 15 12:00:38.564172 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:38.564134 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7vz7j" Jul 15 12:00:38.718408 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:38.718333 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 15 12:00:38.722404 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:38.722379 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:00:38.725304 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:38.725279 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 15 12:00:38.725438 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:38.725280 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 15 12:00:38.725559 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:38.725511 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 15 12:00:38.725708 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:38.725650 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 15 12:00:38.725811 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:38.725717 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 15 12:00:38.725811 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:38.725787 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 15 12:00:38.725920 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:38.725898 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 15 12:00:38.726015 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:38.725997 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 15 12:00:38.726234 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:38.726212 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-b6c6m\"" Jul 15 12:00:38.726984 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:38.726966 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 15 12:00:38.746199 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:38.746168 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 15 12:00:38.885219 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:38.885187 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7db5bbc3-42f4-4da9-8b50-ef83c65b502d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7db5bbc3-42f4-4da9-8b50-ef83c65b502d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:00:38.885405 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:38.885229 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7db5bbc3-42f4-4da9-8b50-ef83c65b502d-web-config\") pod \"alertmanager-main-0\" (UID: \"7db5bbc3-42f4-4da9-8b50-ef83c65b502d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:00:38.885405 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:38.885283 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7db5bbc3-42f4-4da9-8b50-ef83c65b502d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7db5bbc3-42f4-4da9-8b50-ef83c65b502d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:00:38.885405 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:38.885310 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7db5bbc3-42f4-4da9-8b50-ef83c65b502d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7db5bbc3-42f4-4da9-8b50-ef83c65b502d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:00:38.885405 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:38.885339 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7db5bbc3-42f4-4da9-8b50-ef83c65b502d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7db5bbc3-42f4-4da9-8b50-ef83c65b502d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:00:38.885405 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:38.885371 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hv4k7\" (UniqueName: \"kubernetes.io/projected/7db5bbc3-42f4-4da9-8b50-ef83c65b502d-kube-api-access-hv4k7\") pod \"alertmanager-main-0\" (UID: \"7db5bbc3-42f4-4da9-8b50-ef83c65b502d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:00:38.885405 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:38.885396 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7db5bbc3-42f4-4da9-8b50-ef83c65b502d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7db5bbc3-42f4-4da9-8b50-ef83c65b502d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:00:38.885657 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:38.885421 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7db5bbc3-42f4-4da9-8b50-ef83c65b502d-config-volume\") pod \"alertmanager-main-0\" (UID: \"7db5bbc3-42f4-4da9-8b50-ef83c65b502d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:00:38.885657 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:38.885451 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7db5bbc3-42f4-4da9-8b50-ef83c65b502d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7db5bbc3-42f4-4da9-8b50-ef83c65b502d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:00:38.885657 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:38.885474 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7db5bbc3-42f4-4da9-8b50-ef83c65b502d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7db5bbc3-42f4-4da9-8b50-ef83c65b502d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:00:38.885657 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:38.885560 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7db5bbc3-42f4-4da9-8b50-ef83c65b502d-config-out\") pod \"alertmanager-main-0\" (UID: \"7db5bbc3-42f4-4da9-8b50-ef83c65b502d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:00:38.885657 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:38.885585 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7db5bbc3-42f4-4da9-8b50-ef83c65b502d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7db5bbc3-42f4-4da9-8b50-ef83c65b502d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:00:38.885657 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:38.885609 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7db5bbc3-42f4-4da9-8b50-ef83c65b502d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7db5bbc3-42f4-4da9-8b50-ef83c65b502d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:00:38.986765 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:38.986656 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7db5bbc3-42f4-4da9-8b50-ef83c65b502d-config-out\") pod \"alertmanager-main-0\" (UID: \"7db5bbc3-42f4-4da9-8b50-ef83c65b502d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:00:38.986765 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:38.986696 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7db5bbc3-42f4-4da9-8b50-ef83c65b502d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7db5bbc3-42f4-4da9-8b50-ef83c65b502d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:00:38.986765 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:38.986749 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7db5bbc3-42f4-4da9-8b50-ef83c65b502d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7db5bbc3-42f4-4da9-8b50-ef83c65b502d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:00:38.987066 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:38.986828 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7db5bbc3-42f4-4da9-8b50-ef83c65b502d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7db5bbc3-42f4-4da9-8b50-ef83c65b502d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:00:38.987066 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:38.986885 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7db5bbc3-42f4-4da9-8b50-ef83c65b502d-web-config\") pod \"alertmanager-main-0\" (UID: \"7db5bbc3-42f4-4da9-8b50-ef83c65b502d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:00:38.987066 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:38.986913 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7db5bbc3-42f4-4da9-8b50-ef83c65b502d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7db5bbc3-42f4-4da9-8b50-ef83c65b502d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:00:38.987066 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:38.986937 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7db5bbc3-42f4-4da9-8b50-ef83c65b502d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7db5bbc3-42f4-4da9-8b50-ef83c65b502d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:00:38.987066 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:38.986966 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7db5bbc3-42f4-4da9-8b50-ef83c65b502d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7db5bbc3-42f4-4da9-8b50-ef83c65b502d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:00:38.987312 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:38.987134 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hv4k7\" (UniqueName: \"kubernetes.io/projected/7db5bbc3-42f4-4da9-8b50-ef83c65b502d-kube-api-access-hv4k7\") pod \"alertmanager-main-0\" (UID: \"7db5bbc3-42f4-4da9-8b50-ef83c65b502d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:00:38.987312 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:38.987190 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7db5bbc3-42f4-4da9-8b50-ef83c65b502d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7db5bbc3-42f4-4da9-8b50-ef83c65b502d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:00:38.987312 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:38.987231 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7db5bbc3-42f4-4da9-8b50-ef83c65b502d-config-volume\") pod \"alertmanager-main-0\" (UID: \"7db5bbc3-42f4-4da9-8b50-ef83c65b502d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:00:38.987312 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:38.987257 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7db5bbc3-42f4-4da9-8b50-ef83c65b502d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7db5bbc3-42f4-4da9-8b50-ef83c65b502d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:00:38.987312 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:38.987280 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7db5bbc3-42f4-4da9-8b50-ef83c65b502d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7db5bbc3-42f4-4da9-8b50-ef83c65b502d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:00:38.987755 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:38.987367 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7db5bbc3-42f4-4da9-8b50-ef83c65b502d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7db5bbc3-42f4-4da9-8b50-ef83c65b502d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:00:38.987755 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:38.987646 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7db5bbc3-42f4-4da9-8b50-ef83c65b502d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7db5bbc3-42f4-4da9-8b50-ef83c65b502d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:00:38.988165 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:38.988141 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7db5bbc3-42f4-4da9-8b50-ef83c65b502d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7db5bbc3-42f4-4da9-8b50-ef83c65b502d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:00:38.990402 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:38.990322 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7db5bbc3-42f4-4da9-8b50-ef83c65b502d-config-out\") pod \"alertmanager-main-0\" (UID: \"7db5bbc3-42f4-4da9-8b50-ef83c65b502d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:00:38.990402 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:38.990386 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7db5bbc3-42f4-4da9-8b50-ef83c65b502d-web-config\") pod \"alertmanager-main-0\" (UID: \"7db5bbc3-42f4-4da9-8b50-ef83c65b502d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:00:38.990940 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:38.990652 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7db5bbc3-42f4-4da9-8b50-ef83c65b502d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7db5bbc3-42f4-4da9-8b50-ef83c65b502d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:00:38.990940 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:38.990864 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7db5bbc3-42f4-4da9-8b50-ef83c65b502d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7db5bbc3-42f4-4da9-8b50-ef83c65b502d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:00:38.991336 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:38.991313 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7db5bbc3-42f4-4da9-8b50-ef83c65b502d-config-volume\") pod \"alertmanager-main-0\" (UID: \"7db5bbc3-42f4-4da9-8b50-ef83c65b502d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:00:38.991336 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:38.991328 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7db5bbc3-42f4-4da9-8b50-ef83c65b502d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7db5bbc3-42f4-4da9-8b50-ef83c65b502d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:00:38.991871 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:38.991853 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7db5bbc3-42f4-4da9-8b50-ef83c65b502d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7db5bbc3-42f4-4da9-8b50-ef83c65b502d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:00:38.992007 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:38.991986 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7db5bbc3-42f4-4da9-8b50-ef83c65b502d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7db5bbc3-42f4-4da9-8b50-ef83c65b502d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:00:38.992222 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:38.992201 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7db5bbc3-42f4-4da9-8b50-ef83c65b502d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7db5bbc3-42f4-4da9-8b50-ef83c65b502d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:00:38.997485 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:38.997464 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hv4k7\" (UniqueName: \"kubernetes.io/projected/7db5bbc3-42f4-4da9-8b50-ef83c65b502d-kube-api-access-hv4k7\") pod \"alertmanager-main-0\" (UID: \"7db5bbc3-42f4-4da9-8b50-ef83c65b502d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:00:39.034011 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:39.033971 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 15 12:00:40.633894 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:40.633863 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-k5m9c" Jul 15 12:00:40.997144 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:40.997029 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7d9c8b4588-89wzq" Jul 15 12:00:40.997144 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:40.997109 2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7d9c8b4588-89wzq" Jul 15 12:00:40.998711 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:40.998672 2587 patch_prober.go:28] interesting pod/console-7d9c8b4588-89wzq container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.132.0.16:8443/health\": dial tcp 10.132.0.16:8443: connect: connection refused" start-of-body= Jul 15 12:00:40.998877 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:40.998752 2587 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-7d9c8b4588-89wzq" podUID="ea719dc7-8860-4933-bb40-609215906a91" containerName="console" probeResult="failure" output="Get \"https://10.132.0.16:8443/health\": dial tcp 10.132.0.16:8443: connect: connection refused" Jul 15 12:00:42.510580 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:42.510536 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-59977679d8-kh5dp"] Jul 15 12:00:42.521900 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:42.521870 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59977679d8-kh5dp" Jul 15 12:00:42.531006 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:42.530531 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59977679d8-kh5dp"] Jul 15 12:00:42.620099 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:42.620054 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e7b17bf5-1086-48f3-a2a3-336e97deeec3-oauth-serving-cert\") pod \"console-59977679d8-kh5dp\" (UID: \"e7b17bf5-1086-48f3-a2a3-336e97deeec3\") " pod="openshift-console/console-59977679d8-kh5dp" Jul 15 12:00:42.620292 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:42.620114 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e7b17bf5-1086-48f3-a2a3-336e97deeec3-console-serving-cert\") pod \"console-59977679d8-kh5dp\" (UID: \"e7b17bf5-1086-48f3-a2a3-336e97deeec3\") " pod="openshift-console/console-59977679d8-kh5dp" Jul 15 12:00:42.620292 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:42.620210 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e7b17bf5-1086-48f3-a2a3-336e97deeec3-console-config\") pod \"console-59977679d8-kh5dp\" (UID: \"e7b17bf5-1086-48f3-a2a3-336e97deeec3\") " pod="openshift-console/console-59977679d8-kh5dp" Jul 15 12:00:42.620292 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:42.620245 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e7b17bf5-1086-48f3-a2a3-336e97deeec3-service-ca\") pod \"console-59977679d8-kh5dp\" (UID: \"e7b17bf5-1086-48f3-a2a3-336e97deeec3\") " pod="openshift-console/console-59977679d8-kh5dp" Jul 15 12:00:42.620292 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:42.620272 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7b17bf5-1086-48f3-a2a3-336e97deeec3-trusted-ca-bundle\") pod \"console-59977679d8-kh5dp\" (UID: \"e7b17bf5-1086-48f3-a2a3-336e97deeec3\") " pod="openshift-console/console-59977679d8-kh5dp" Jul 15 12:00:42.620496 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:42.620300 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e7b17bf5-1086-48f3-a2a3-336e97deeec3-console-oauth-config\") pod \"console-59977679d8-kh5dp\" (UID: \"e7b17bf5-1086-48f3-a2a3-336e97deeec3\") " pod="openshift-console/console-59977679d8-kh5dp" Jul 15 12:00:42.620496 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:42.620325 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vq6db\" (UniqueName: \"kubernetes.io/projected/e7b17bf5-1086-48f3-a2a3-336e97deeec3-kube-api-access-vq6db\") pod \"console-59977679d8-kh5dp\" (UID: \"e7b17bf5-1086-48f3-a2a3-336e97deeec3\") " pod="openshift-console/console-59977679d8-kh5dp" Jul 15 12:00:42.721500 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:42.721465 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e7b17bf5-1086-48f3-a2a3-336e97deeec3-console-config\") pod \"console-59977679d8-kh5dp\" (UID: \"e7b17bf5-1086-48f3-a2a3-336e97deeec3\") " pod="openshift-console/console-59977679d8-kh5dp" Jul 15 12:00:42.721500 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:42.721503 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e7b17bf5-1086-48f3-a2a3-336e97deeec3-service-ca\") pod \"console-59977679d8-kh5dp\" (UID: \"e7b17bf5-1086-48f3-a2a3-336e97deeec3\") " pod="openshift-console/console-59977679d8-kh5dp" Jul 15 12:00:42.721777 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:42.721532 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7b17bf5-1086-48f3-a2a3-336e97deeec3-trusted-ca-bundle\") pod \"console-59977679d8-kh5dp\" (UID: \"e7b17bf5-1086-48f3-a2a3-336e97deeec3\") " pod="openshift-console/console-59977679d8-kh5dp" Jul 15 12:00:42.721777 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:42.721556 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e7b17bf5-1086-48f3-a2a3-336e97deeec3-console-oauth-config\") pod \"console-59977679d8-kh5dp\" (UID: \"e7b17bf5-1086-48f3-a2a3-336e97deeec3\") " pod="openshift-console/console-59977679d8-kh5dp" Jul 15 12:00:42.721777 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:42.721613 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vq6db\" (UniqueName: \"kubernetes.io/projected/e7b17bf5-1086-48f3-a2a3-336e97deeec3-kube-api-access-vq6db\") pod \"console-59977679d8-kh5dp\" (UID: \"e7b17bf5-1086-48f3-a2a3-336e97deeec3\") " pod="openshift-console/console-59977679d8-kh5dp" Jul 15 12:00:42.721777 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:42.721702 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e7b17bf5-1086-48f3-a2a3-336e97deeec3-oauth-serving-cert\") pod \"console-59977679d8-kh5dp\" (UID: \"e7b17bf5-1086-48f3-a2a3-336e97deeec3\") " pod="openshift-console/console-59977679d8-kh5dp" Jul 15 12:00:42.721777 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:42.721761 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e7b17bf5-1086-48f3-a2a3-336e97deeec3-console-serving-cert\") pod \"console-59977679d8-kh5dp\" (UID: \"e7b17bf5-1086-48f3-a2a3-336e97deeec3\") " pod="openshift-console/console-59977679d8-kh5dp" Jul 15 12:00:42.722387 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:42.722343 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e7b17bf5-1086-48f3-a2a3-336e97deeec3-service-ca\") pod \"console-59977679d8-kh5dp\" (UID: \"e7b17bf5-1086-48f3-a2a3-336e97deeec3\") " pod="openshift-console/console-59977679d8-kh5dp" Jul 15 12:00:42.722784 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:42.722756 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e7b17bf5-1086-48f3-a2a3-336e97deeec3-oauth-serving-cert\") pod \"console-59977679d8-kh5dp\" (UID: \"e7b17bf5-1086-48f3-a2a3-336e97deeec3\") " pod="openshift-console/console-59977679d8-kh5dp" Jul 15 12:00:42.722784 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:42.722680 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7b17bf5-1086-48f3-a2a3-336e97deeec3-trusted-ca-bundle\") pod \"console-59977679d8-kh5dp\" (UID: \"e7b17bf5-1086-48f3-a2a3-336e97deeec3\") " pod="openshift-console/console-59977679d8-kh5dp" Jul 15 12:00:42.722980 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:42.722876 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e7b17bf5-1086-48f3-a2a3-336e97deeec3-console-config\") pod \"console-59977679d8-kh5dp\" (UID: \"e7b17bf5-1086-48f3-a2a3-336e97deeec3\") " pod="openshift-console/console-59977679d8-kh5dp" Jul 15 12:00:42.724627 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:42.724607 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e7b17bf5-1086-48f3-a2a3-336e97deeec3-console-oauth-config\") pod \"console-59977679d8-kh5dp\" (UID: \"e7b17bf5-1086-48f3-a2a3-336e97deeec3\") " pod="openshift-console/console-59977679d8-kh5dp" Jul 15 12:00:42.724981 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:42.724961 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e7b17bf5-1086-48f3-a2a3-336e97deeec3-console-serving-cert\") pod \"console-59977679d8-kh5dp\" (UID: \"e7b17bf5-1086-48f3-a2a3-336e97deeec3\") " pod="openshift-console/console-59977679d8-kh5dp" Jul 15 12:00:42.730694 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:42.730670 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vq6db\" (UniqueName: \"kubernetes.io/projected/e7b17bf5-1086-48f3-a2a3-336e97deeec3-kube-api-access-vq6db\") pod \"console-59977679d8-kh5dp\" (UID: \"e7b17bf5-1086-48f3-a2a3-336e97deeec3\") " pod="openshift-console/console-59977679d8-kh5dp" Jul 15 12:00:42.834514 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:42.834482 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59977679d8-kh5dp" Jul 15 12:00:44.361423 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:44.361385 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-b788d9845-z5cx7"] Jul 15 12:00:44.366384 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:44.366345 2587 patch_prober.go:28] interesting pod/image-registry-b788d9845-z5cx7 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 15 12:00:44.366525 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:44.366409 2587 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-b788d9845-z5cx7" podUID="f6e2e61d-a8c4-475d-a43a-fe32868b16ba" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 12:00:44.776626 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:00:44.776591 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcec409e8_1241_411e_834f_7d82377b1f8b.slice/crio-41a4116baf4deb452e55d4a944b83147a984f7585541c879709e658e2e6f0311 WatchSource:0}: Error finding container 41a4116baf4deb452e55d4a944b83147a984f7585541c879709e658e2e6f0311: Status 404 returned error can't find the container with id 41a4116baf4deb452e55d4a944b83147a984f7585541c879709e658e2e6f0311 Jul 15 12:00:44.909980 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:44.909952 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 15 12:00:44.911307 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:00:44.911278 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7db5bbc3_42f4_4da9_8b50_ef83c65b502d.slice/crio-8da722307349399eeea59b1cb398bc7ac443291aba011c04f184f9c3dcd2bf5f WatchSource:0}: Error finding container 8da722307349399eeea59b1cb398bc7ac443291aba011c04f184f9c3dcd2bf5f: Status 404 returned error can't find the container with id 8da722307349399eeea59b1cb398bc7ac443291aba011c04f184f9c3dcd2bf5f Jul 15 12:00:44.917599 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:44.917573 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59977679d8-kh5dp"] Jul 15 12:00:44.921131 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:00:44.921105 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7b17bf5_1086_48f3_a2a3_336e97deeec3.slice/crio-bb78e72bc3ec7407e8f8d0dff473ba781d00e6a2f38cf77ef1133f42a2f1e3d9 WatchSource:0}: Error finding container bb78e72bc3ec7407e8f8d0dff473ba781d00e6a2f38cf77ef1133f42a2f1e3d9: Status 404 returned error can't find the container with id bb78e72bc3ec7407e8f8d0dff473ba781d00e6a2f38cf77ef1133f42a2f1e3d9 Jul 15 12:00:45.698235 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:45.698107 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59977679d8-kh5dp" event={"ID":"e7b17bf5-1086-48f3-a2a3-336e97deeec3","Type":"ContainerStarted","Data":"bfb19eefc825d3b515c4fa25369897343cc6628f4e867ef6ba528aeb1088a938"} Jul 15 12:00:45.698235 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:45.698179 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59977679d8-kh5dp" event={"ID":"e7b17bf5-1086-48f3-a2a3-336e97deeec3","Type":"ContainerStarted","Data":"bb78e72bc3ec7407e8f8d0dff473ba781d00e6a2f38cf77ef1133f42a2f1e3d9"} Jul 15 12:00:45.699870 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:45.699811 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-7kt4q" event={"ID":"56b02112-6aa5-42dd-aeab-0a296a6314b7","Type":"ContainerStarted","Data":"316dfea804f5e7e4bfd608e78dd376621a3aeb259d91ead45a9f93aee5e37223"} Jul 15 12:00:45.699870 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:45.699870 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-7kt4q" Jul 15 12:00:45.701259 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:45.701231 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7db5bbc3-42f4-4da9-8b50-ef83c65b502d","Type":"ContainerStarted","Data":"8da722307349399eeea59b1cb398bc7ac443291aba011c04f184f9c3dcd2bf5f"} Jul 15 12:00:45.702990 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:45.702969 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7vz7j" event={"ID":"cec409e8-1241-411e-834f-7d82377b1f8b","Type":"ContainerStarted","Data":"41a4116baf4deb452e55d4a944b83147a984f7585541c879709e658e2e6f0311"} Jul 15 12:00:45.718562 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:45.718504 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-59977679d8-kh5dp" podStartSLOduration=3.718485711 podStartE2EDuration="3.718485711s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:00:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:00:45.717307907 +0000 UTC m=+117.174684836" watchObservedRunningTime="2026-07-15 12:00:45.718485711 +0000 UTC m=+117.175862638" Jul 15 12:00:45.721925 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:45.721898 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-7kt4q" Jul 15 12:00:45.735150 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:45.735100 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-7kt4q" podStartSLOduration=2.163542453 podStartE2EDuration="24.735081065s" totalImagesPullingTime="22.571538612s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:00:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:00:45.734236382 +0000 UTC m=+117.191613308" watchObservedRunningTime="2026-07-15 12:00:45.735081065 +0000 UTC m=+117.192457992" Jul 15 12:00:48.718674 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:48.718582 2587 generic.go:356] "Generic (PLEG): container finished" podID="7db5bbc3-42f4-4da9-8b50-ef83c65b502d" containerID="e55a6b3bf9229715df974464e4e863deca9568a324d2f2449bb9b9945cfe6f21" exitCode=0 Jul 15 12:00:48.719163 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:48.718668 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7db5bbc3-42f4-4da9-8b50-ef83c65b502d","Type":"ContainerDied","Data":"e55a6b3bf9229715df974464e4e863deca9568a324d2f2449bb9b9945cfe6f21"} Jul 15 12:00:48.720685 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:48.720657 2587 generic.go:356] "Generic (PLEG): container finished" podID="cec409e8-1241-411e-834f-7d82377b1f8b" containerID="70944003210a1d7b8515d601eee2d46d3310278a47abc46ee99648fa84e76a22" exitCode=0 Jul 15 12:00:48.720784 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:48.720712 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7vz7j" event={"ID":"cec409e8-1241-411e-834f-7d82377b1f8b","Type":"ContainerDied","Data":"70944003210a1d7b8515d601eee2d46d3310278a47abc46ee99648fa84e76a22"} Jul 15 12:00:49.732454 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:49.732342 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7vz7j" event={"ID":"cec409e8-1241-411e-834f-7d82377b1f8b","Type":"ContainerStarted","Data":"d323e993569020af4c87ab96206b3d60c3ccb084c0b028bf4fe859cf11d73415"} Jul 15 12:00:49.732454 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:49.732438 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7vz7j" event={"ID":"cec409e8-1241-411e-834f-7d82377b1f8b","Type":"ContainerStarted","Data":"f1dd9d9194ecf8e8ffe56887c0b93d48a0e031696a26cf60c6d86306beb14f30"} Jul 15 12:00:49.754526 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:49.754455 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-7vz7j" podStartSLOduration=9.268767876 podStartE2EDuration="12.754435458s" totalImagesPullingTime="3.381370626s" totalInitContainerRuntime="104.296956ms" isStatefulPod=true podCreationTimestamp="2026-07-15 12:00:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:00:49.752689739 +0000 UTC m=+121.210066707" watchObservedRunningTime="2026-07-15 12:00:49.754435458 +0000 UTC m=+121.211812383" Jul 15 12:00:51.002480 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:51.002445 2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7d9c8b4588-89wzq" Jul 15 12:00:51.018321 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:51.018295 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7d9c8b4588-89wzq" Jul 15 12:00:51.746892 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:51.746851 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7db5bbc3-42f4-4da9-8b50-ef83c65b502d","Type":"ContainerStarted","Data":"fcf412858e43c1962c07b5969a44bec1ef55c6d59529d7c841352969e0b4a332"} Jul 15 12:00:51.747097 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:51.746941 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7db5bbc3-42f4-4da9-8b50-ef83c65b502d","Type":"ContainerStarted","Data":"7234caa3449e3d5a2eb7a690b7e76ade122abe6965d06c98d718b562bcf9c6ff"} Jul 15 12:00:51.747097 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:51.746986 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7db5bbc3-42f4-4da9-8b50-ef83c65b502d","Type":"ContainerStarted","Data":"92dfda96204cc79c3b06028b831e29e898c10f2c00d793365319d910f9fd472e"} Jul 15 12:00:51.747097 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:51.747015 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7db5bbc3-42f4-4da9-8b50-ef83c65b502d","Type":"ContainerStarted","Data":"abb84544201d0a33733bc0f8c127b36ebd2b144a0ae133a42f67dcce181ff28b"} Jul 15 12:00:51.747097 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:51.747041 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7db5bbc3-42f4-4da9-8b50-ef83c65b502d","Type":"ContainerStarted","Data":"6118436ec0021880e86287ad9009efd9d909e401a78466d7a7d1d09a3ab3ebbc"} Jul 15 12:00:52.835605 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:52.835564 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-59977679d8-kh5dp" Jul 15 12:00:52.836060 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:52.835637 2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-59977679d8-kh5dp" Jul 15 12:00:52.841269 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:52.841245 2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-59977679d8-kh5dp" Jul 15 12:00:53.758920 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:53.758874 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7db5bbc3-42f4-4da9-8b50-ef83c65b502d","Type":"ContainerStarted","Data":"33ab04a13d082a31271ac01779f1272a987726a563d23001a12eaf515fc38f84"} Jul 15 12:00:53.763496 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:53.763473 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-59977679d8-kh5dp" Jul 15 12:00:53.790269 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:53.790208 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=9.083556764 podStartE2EDuration="15.790191536s" totalImagesPullingTime="6.701678539s" totalInitContainerRuntime="4.956233ms" isStatefulPod=false podCreationTimestamp="2026-07-15 12:00:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:00:53.789532985 +0000 UTC m=+125.246909935" watchObservedRunningTime="2026-07-15 12:00:53.790191536 +0000 UTC m=+125.247568460" Jul 15 12:00:53.845526 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:53.845490 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d9c8b4588-89wzq"] Jul 15 12:00:54.367235 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:54.367206 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-b788d9845-z5cx7" Jul 15 12:00:58.878662 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:58.878622 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7e5aa88-1817-4a28-8add-88cf29795c94-metrics-certs\") pod \"network-metrics-daemon-xtlxm\" (UID: \"e7e5aa88-1817-4a28-8add-88cf29795c94\") " pod="openshift-multus/network-metrics-daemon-xtlxm" Jul 15 12:00:58.881177 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:58.881145 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7e5aa88-1817-4a28-8add-88cf29795c94-metrics-certs\") pod \"network-metrics-daemon-xtlxm\" (UID: \"e7e5aa88-1817-4a28-8add-88cf29795c94\") " pod="openshift-multus/network-metrics-daemon-xtlxm" Jul 15 12:00:59.108549 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:59.108518 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-f876h\"" Jul 15 12:00:59.116656 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:59.116634 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtlxm" Jul 15 12:00:59.368002 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:59.367976 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xtlxm"] Jul 15 12:00:59.370466 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:00:59.370421 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7e5aa88_1817_4a28_8add_88cf29795c94.slice/crio-6e6053613b5ebf614a6e43968cae58cb30b426edad9dbfaa5241fc542711ca26 WatchSource:0}: Error finding container 6e6053613b5ebf614a6e43968cae58cb30b426edad9dbfaa5241fc542711ca26: Status 404 returned error can't find the container with id 6e6053613b5ebf614a6e43968cae58cb30b426edad9dbfaa5241fc542711ca26 Jul 15 12:00:59.785387 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:00:59.785351 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xtlxm" event={"ID":"e7e5aa88-1817-4a28-8add-88cf29795c94","Type":"ContainerStarted","Data":"6e6053613b5ebf614a6e43968cae58cb30b426edad9dbfaa5241fc542711ca26"} Jul 15 12:01:01.794528 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:01.794488 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xtlxm" event={"ID":"e7e5aa88-1817-4a28-8add-88cf29795c94","Type":"ContainerStarted","Data":"ac19d45528c6c8170362f6bcdc1f8f65c640e32ddbb0675d9fa5bf86595abb80"} Jul 15 12:01:01.794955 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:01.794556 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xtlxm" event={"ID":"e7e5aa88-1817-4a28-8add-88cf29795c94","Type":"ContainerStarted","Data":"d3e78760bb3637a7d35621ede19c914570338ba0797363e31871483ed32edb2a"} Jul 15 12:01:01.812853 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:01.812804 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-xtlxm" podStartSLOduration=131.169541353 podStartE2EDuration="2m12.812786994s" totalImagesPullingTime="1.643245641s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 11:58:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:01:01.812784033 +0000 UTC m=+133.270160958" watchObservedRunningTime="2026-07-15 12:01:01.812786994 +0000 UTC m=+133.270163919" Jul 15 12:01:09.383822 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:09.383754 2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-b788d9845-z5cx7" podUID="f6e2e61d-a8c4-475d-a43a-fe32868b16ba" containerName="registry" containerID="cri-o://220f3ca26f11d0b185463e3bc63aff11e5edc46a6c72d17b259e8f874dc26ef1" gracePeriod=30 Jul 15 12:01:09.648716 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:09.648641 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b788d9845-z5cx7" Jul 15 12:01:09.769443 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:09.769401 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-registry-tls\") pod \"f6e2e61d-a8c4-475d-a43a-fe32868b16ba\" (UID: \"f6e2e61d-a8c4-475d-a43a-fe32868b16ba\") " Jul 15 12:01:09.769641 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:09.769455 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-drr47\" (UniqueName: \"kubernetes.io/projected/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-kube-api-access-drr47\") pod \"f6e2e61d-a8c4-475d-a43a-fe32868b16ba\" (UID: \"f6e2e61d-a8c4-475d-a43a-fe32868b16ba\") " Jul 15 12:01:09.769641 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:09.769480 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-bound-sa-token\") pod \"f6e2e61d-a8c4-475d-a43a-fe32868b16ba\" (UID: \"f6e2e61d-a8c4-475d-a43a-fe32868b16ba\") " Jul 15 12:01:09.769641 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:09.769508 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-registry-certificates\") pod \"f6e2e61d-a8c4-475d-a43a-fe32868b16ba\" (UID: \"f6e2e61d-a8c4-475d-a43a-fe32868b16ba\") " Jul 15 12:01:09.769641 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:09.769553 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-image-registry-private-configuration\") pod \"f6e2e61d-a8c4-475d-a43a-fe32868b16ba\" (UID: \"f6e2e61d-a8c4-475d-a43a-fe32868b16ba\") " Jul 15 12:01:09.769940 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:09.769659 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-ca-trust-extracted\") pod \"f6e2e61d-a8c4-475d-a43a-fe32868b16ba\" (UID: \"f6e2e61d-a8c4-475d-a43a-fe32868b16ba\") " Jul 15 12:01:09.769940 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:09.769725 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-trusted-ca\") pod \"f6e2e61d-a8c4-475d-a43a-fe32868b16ba\" (UID: \"f6e2e61d-a8c4-475d-a43a-fe32868b16ba\") " Jul 15 12:01:09.769940 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:09.769770 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-installation-pull-secrets\") pod \"f6e2e61d-a8c4-475d-a43a-fe32868b16ba\" (UID: \"f6e2e61d-a8c4-475d-a43a-fe32868b16ba\") " Jul 15 12:01:09.770082 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:09.769999 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "f6e2e61d-a8c4-475d-a43a-fe32868b16ba" (UID: "f6e2e61d-a8c4-475d-a43a-fe32868b16ba"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:01:09.770383 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:09.770354 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "f6e2e61d-a8c4-475d-a43a-fe32868b16ba" (UID: "f6e2e61d-a8c4-475d-a43a-fe32868b16ba"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:01:09.772154 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:09.772102 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "f6e2e61d-a8c4-475d-a43a-fe32868b16ba" (UID: "f6e2e61d-a8c4-475d-a43a-fe32868b16ba"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 12:01:09.772312 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:09.772286 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "f6e2e61d-a8c4-475d-a43a-fe32868b16ba" (UID: "f6e2e61d-a8c4-475d-a43a-fe32868b16ba"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 12:01:09.772387 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:09.772348 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-kube-api-access-drr47" (OuterVolumeSpecName: "kube-api-access-drr47") pod "f6e2e61d-a8c4-475d-a43a-fe32868b16ba" (UID: "f6e2e61d-a8c4-475d-a43a-fe32868b16ba"). InnerVolumeSpecName "kube-api-access-drr47". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:01:09.772444 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:09.772398 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "f6e2e61d-a8c4-475d-a43a-fe32868b16ba" (UID: "f6e2e61d-a8c4-475d-a43a-fe32868b16ba"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:01:09.772444 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:09.772400 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "f6e2e61d-a8c4-475d-a43a-fe32868b16ba" (UID: "f6e2e61d-a8c4-475d-a43a-fe32868b16ba"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:01:09.778584 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:09.778549 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "f6e2e61d-a8c4-475d-a43a-fe32868b16ba" (UID: "f6e2e61d-a8c4-475d-a43a-fe32868b16ba"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:01:09.824675 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:09.824633 2587 generic.go:356] "Generic (PLEG): container finished" podID="f6e2e61d-a8c4-475d-a43a-fe32868b16ba" containerID="220f3ca26f11d0b185463e3bc63aff11e5edc46a6c72d17b259e8f874dc26ef1" exitCode=0 Jul 15 12:01:09.824846 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:09.824680 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b788d9845-z5cx7" Jul 15 12:01:09.824846 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:09.824716 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b788d9845-z5cx7" event={"ID":"f6e2e61d-a8c4-475d-a43a-fe32868b16ba","Type":"ContainerDied","Data":"220f3ca26f11d0b185463e3bc63aff11e5edc46a6c72d17b259e8f874dc26ef1"} Jul 15 12:01:09.824846 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:09.824784 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b788d9845-z5cx7" event={"ID":"f6e2e61d-a8c4-475d-a43a-fe32868b16ba","Type":"ContainerDied","Data":"226c25d142b27974be391371e4fbf4cdb9c0bbbbf41d0e057c661cd33a3af631"} Jul 15 12:01:09.824846 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:09.824806 2587 scope.go:117] "RemoveContainer" containerID="220f3ca26f11d0b185463e3bc63aff11e5edc46a6c72d17b259e8f874dc26ef1" Jul 15 12:01:09.838541 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:09.838516 2587 scope.go:117] "RemoveContainer" containerID="220f3ca26f11d0b185463e3bc63aff11e5edc46a6c72d17b259e8f874dc26ef1" Jul 15 12:01:09.838877 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:01:09.838850 2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"220f3ca26f11d0b185463e3bc63aff11e5edc46a6c72d17b259e8f874dc26ef1\": container with ID starting with 220f3ca26f11d0b185463e3bc63aff11e5edc46a6c72d17b259e8f874dc26ef1 not found: ID does not exist" containerID="220f3ca26f11d0b185463e3bc63aff11e5edc46a6c72d17b259e8f874dc26ef1" Jul 15 12:01:09.838929 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:09.838886 2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"220f3ca26f11d0b185463e3bc63aff11e5edc46a6c72d17b259e8f874dc26ef1"} err="failed to get container status \"220f3ca26f11d0b185463e3bc63aff11e5edc46a6c72d17b259e8f874dc26ef1\": rpc error: code = NotFound desc = could not find container \"220f3ca26f11d0b185463e3bc63aff11e5edc46a6c72d17b259e8f874dc26ef1\": container with ID starting with 220f3ca26f11d0b185463e3bc63aff11e5edc46a6c72d17b259e8f874dc26ef1 not found: ID does not exist" Jul 15 12:01:09.845620 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:09.845591 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-b788d9845-z5cx7"] Jul 15 12:01:09.849397 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:09.849372 2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-b788d9845-z5cx7"] Jul 15 12:01:09.870383 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:09.870353 2587 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-trusted-ca\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:01:09.870383 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:09.870382 2587 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-installation-pull-secrets\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:01:09.870540 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:09.870394 2587 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-registry-tls\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:01:09.870540 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:09.870403 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-drr47\" (UniqueName: \"kubernetes.io/projected/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-kube-api-access-drr47\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:01:09.870540 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:09.870412 2587 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-bound-sa-token\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:01:09.870540 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:09.870420 2587 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-registry-certificates\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:01:09.870540 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:09.870429 2587 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-image-registry-private-configuration\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:01:09.870540 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:09.870440 2587 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f6e2e61d-a8c4-475d-a43a-fe32868b16ba-ca-trust-extracted\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:01:11.184236 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:11.184206 2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6e2e61d-a8c4-475d-a43a-fe32868b16ba" path="/var/lib/kubelet/pods/f6e2e61d-a8c4-475d-a43a-fe32868b16ba/volumes" Jul 15 12:01:18.884680 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:18.884614 2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7d9c8b4588-89wzq" podUID="ea719dc7-8860-4933-bb40-609215906a91" containerName="console" containerID="cri-o://9ef327aa6cb0712377a8ebf8209b86c1f868e7ebadd0d28cc82644b9ebff4c68" gracePeriod=15 Jul 15 12:01:19.173434 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:19.173401 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d9c8b4588-89wzq" Jul 15 12:01:19.348593 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:19.348552 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ea719dc7-8860-4933-bb40-609215906a91-console-config\") pod \"ea719dc7-8860-4933-bb40-609215906a91\" (UID: \"ea719dc7-8860-4933-bb40-609215906a91\") " Jul 15 12:01:19.348811 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:19.348623 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ea719dc7-8860-4933-bb40-609215906a91-service-ca\") pod \"ea719dc7-8860-4933-bb40-609215906a91\" (UID: \"ea719dc7-8860-4933-bb40-609215906a91\") " Jul 15 12:01:19.348811 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:19.348656 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ea719dc7-8860-4933-bb40-609215906a91-console-serving-cert\") pod \"ea719dc7-8860-4933-bb40-609215906a91\" (UID: \"ea719dc7-8860-4933-bb40-609215906a91\") " Jul 15 12:01:19.348811 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:19.348683 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pc7zm\" (UniqueName: \"kubernetes.io/projected/ea719dc7-8860-4933-bb40-609215906a91-kube-api-access-pc7zm\") pod \"ea719dc7-8860-4933-bb40-609215906a91\" (UID: \"ea719dc7-8860-4933-bb40-609215906a91\") " Jul 15 12:01:19.348811 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:19.348713 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ea719dc7-8860-4933-bb40-609215906a91-oauth-serving-cert\") pod \"ea719dc7-8860-4933-bb40-609215906a91\" (UID: \"ea719dc7-8860-4933-bb40-609215906a91\") " Jul 15 12:01:19.348811 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:19.348769 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ea719dc7-8860-4933-bb40-609215906a91-console-oauth-config\") pod \"ea719dc7-8860-4933-bb40-609215906a91\" (UID: \"ea719dc7-8860-4933-bb40-609215906a91\") " Jul 15 12:01:19.348811 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:19.348797 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea719dc7-8860-4933-bb40-609215906a91-trusted-ca-bundle\") pod \"ea719dc7-8860-4933-bb40-609215906a91\" (UID: \"ea719dc7-8860-4933-bb40-609215906a91\") " Jul 15 12:01:19.349165 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:19.349135 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea719dc7-8860-4933-bb40-609215906a91-console-config" (OuterVolumeSpecName: "console-config") pod "ea719dc7-8860-4933-bb40-609215906a91" (UID: "ea719dc7-8860-4933-bb40-609215906a91"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:01:19.349165 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:19.349147 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea719dc7-8860-4933-bb40-609215906a91-service-ca" (OuterVolumeSpecName: "service-ca") pod "ea719dc7-8860-4933-bb40-609215906a91" (UID: "ea719dc7-8860-4933-bb40-609215906a91"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:01:19.349284 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:19.349226 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea719dc7-8860-4933-bb40-609215906a91-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ea719dc7-8860-4933-bb40-609215906a91" (UID: "ea719dc7-8860-4933-bb40-609215906a91"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:01:19.349344 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:19.349286 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea719dc7-8860-4933-bb40-609215906a91-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "ea719dc7-8860-4933-bb40-609215906a91" (UID: "ea719dc7-8860-4933-bb40-609215906a91"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:01:19.351101 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:19.351077 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea719dc7-8860-4933-bb40-609215906a91-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ea719dc7-8860-4933-bb40-609215906a91" (UID: "ea719dc7-8860-4933-bb40-609215906a91"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 12:01:19.351101 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:19.351093 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea719dc7-8860-4933-bb40-609215906a91-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ea719dc7-8860-4933-bb40-609215906a91" (UID: "ea719dc7-8860-4933-bb40-609215906a91"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 12:01:19.351239 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:19.351103 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea719dc7-8860-4933-bb40-609215906a91-kube-api-access-pc7zm" (OuterVolumeSpecName: "kube-api-access-pc7zm") pod "ea719dc7-8860-4933-bb40-609215906a91" (UID: "ea719dc7-8860-4933-bb40-609215906a91"). InnerVolumeSpecName "kube-api-access-pc7zm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:01:19.449712 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:19.449622 2587 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ea719dc7-8860-4933-bb40-609215906a91-console-config\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:01:19.449712 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:19.449655 2587 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ea719dc7-8860-4933-bb40-609215906a91-service-ca\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:01:19.449712 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:19.449663 2587 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ea719dc7-8860-4933-bb40-609215906a91-console-serving-cert\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:01:19.449712 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:19.449673 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pc7zm\" (UniqueName: \"kubernetes.io/projected/ea719dc7-8860-4933-bb40-609215906a91-kube-api-access-pc7zm\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:01:19.449712 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:19.449683 2587 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ea719dc7-8860-4933-bb40-609215906a91-oauth-serving-cert\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:01:19.449712 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:19.449691 2587 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ea719dc7-8860-4933-bb40-609215906a91-console-oauth-config\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:01:19.449712 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:19.449698 2587 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea719dc7-8860-4933-bb40-609215906a91-trusted-ca-bundle\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:01:19.868315 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:19.868273 2587 generic.go:356] "Generic (PLEG): container finished" podID="ea719dc7-8860-4933-bb40-609215906a91" containerID="9ef327aa6cb0712377a8ebf8209b86c1f868e7ebadd0d28cc82644b9ebff4c68" exitCode=0 Jul 15 12:01:19.868493 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:19.868321 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d9c8b4588-89wzq" Jul 15 12:01:19.868493 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:19.868363 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d9c8b4588-89wzq" event={"ID":"ea719dc7-8860-4933-bb40-609215906a91","Type":"ContainerDied","Data":"9ef327aa6cb0712377a8ebf8209b86c1f868e7ebadd0d28cc82644b9ebff4c68"} Jul 15 12:01:19.868493 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:19.868420 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d9c8b4588-89wzq" event={"ID":"ea719dc7-8860-4933-bb40-609215906a91","Type":"ContainerDied","Data":"047a4249a902bc4a166f373e33be395d4197f4d65b6b31f62b95bafdfc390ebb"} Jul 15 12:01:19.868493 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:19.868443 2587 scope.go:117] "RemoveContainer" containerID="9ef327aa6cb0712377a8ebf8209b86c1f868e7ebadd0d28cc82644b9ebff4c68" Jul 15 12:01:19.876979 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:19.876963 2587 scope.go:117] "RemoveContainer" containerID="9ef327aa6cb0712377a8ebf8209b86c1f868e7ebadd0d28cc82644b9ebff4c68" Jul 15 12:01:19.877249 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:01:19.877228 2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ef327aa6cb0712377a8ebf8209b86c1f868e7ebadd0d28cc82644b9ebff4c68\": container with ID starting with 9ef327aa6cb0712377a8ebf8209b86c1f868e7ebadd0d28cc82644b9ebff4c68 not found: ID does not exist" containerID="9ef327aa6cb0712377a8ebf8209b86c1f868e7ebadd0d28cc82644b9ebff4c68" Jul 15 12:01:19.877299 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:19.877258 2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ef327aa6cb0712377a8ebf8209b86c1f868e7ebadd0d28cc82644b9ebff4c68"} err="failed to get container status \"9ef327aa6cb0712377a8ebf8209b86c1f868e7ebadd0d28cc82644b9ebff4c68\": rpc error: code = NotFound desc = could not find container \"9ef327aa6cb0712377a8ebf8209b86c1f868e7ebadd0d28cc82644b9ebff4c68\": container with ID starting with 9ef327aa6cb0712377a8ebf8209b86c1f868e7ebadd0d28cc82644b9ebff4c68 not found: ID does not exist" Jul 15 12:01:19.888991 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:19.888965 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d9c8b4588-89wzq"] Jul 15 12:01:19.895233 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:19.895207 2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7d9c8b4588-89wzq"] Jul 15 12:01:21.183807 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:01:21.183775 2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea719dc7-8860-4933-bb40-609215906a91" path="/var/lib/kubelet/pods/ea719dc7-8860-4933-bb40-609215906a91/volumes" Jul 15 12:02:01.997600 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:01.997556 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7c9f77bc7-ltx55"] Jul 15 12:02:01.998162 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:01.998024 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6e2e61d-a8c4-475d-a43a-fe32868b16ba" containerName="registry" Jul 15 12:02:01.998162 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:01.998044 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6e2e61d-a8c4-475d-a43a-fe32868b16ba" containerName="registry" Jul 15 12:02:01.998162 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:01.998072 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea719dc7-8860-4933-bb40-609215906a91" containerName="console" Jul 15 12:02:01.998162 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:01.998079 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea719dc7-8860-4933-bb40-609215906a91" containerName="console" Jul 15 12:02:01.998162 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:01.998145 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6e2e61d-a8c4-475d-a43a-fe32868b16ba" containerName="registry" Jul 15 12:02:01.998162 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:01.998156 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea719dc7-8860-4933-bb40-609215906a91" containerName="console" Jul 15 12:02:02.001113 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:02.001091 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7c9f77bc7-ltx55" Jul 15 12:02:02.004245 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:02.004222 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 15 12:02:02.004365 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:02.004253 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 15 12:02:02.004365 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:02.004355 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 15 12:02:02.004477 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:02.004364 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 15 12:02:02.004477 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:02.004466 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-dp7fd\"" Jul 15 12:02:02.004581 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:02.004562 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 15 12:02:02.010915 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:02.010228 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 15 12:02:02.015432 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:02.015406 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7c9f77bc7-ltx55"] Jul 15 12:02:02.084276 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:02.084239 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1a4cf141-579e-47bd-b911-b19e20f0599c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7c9f77bc7-ltx55\" (UID: \"1a4cf141-579e-47bd-b911-b19e20f0599c\") " pod="openshift-monitoring/telemeter-client-7c9f77bc7-ltx55" Jul 15 12:02:02.084276 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:02.084282 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/1a4cf141-579e-47bd-b911-b19e20f0599c-federate-client-tls\") pod \"telemeter-client-7c9f77bc7-ltx55\" (UID: \"1a4cf141-579e-47bd-b911-b19e20f0599c\") " pod="openshift-monitoring/telemeter-client-7c9f77bc7-ltx55" Jul 15 12:02:02.084479 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:02.084313 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1a4cf141-579e-47bd-b911-b19e20f0599c-metrics-client-ca\") pod \"telemeter-client-7c9f77bc7-ltx55\" (UID: \"1a4cf141-579e-47bd-b911-b19e20f0599c\") " pod="openshift-monitoring/telemeter-client-7c9f77bc7-ltx55" Jul 15 12:02:02.084479 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:02.084389 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a4cf141-579e-47bd-b911-b19e20f0599c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7c9f77bc7-ltx55\" (UID: \"1a4cf141-579e-47bd-b911-b19e20f0599c\") " pod="openshift-monitoring/telemeter-client-7c9f77bc7-ltx55" Jul 15 12:02:02.084479 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:02.084422 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a4cf141-579e-47bd-b911-b19e20f0599c-serving-certs-ca-bundle\") pod \"telemeter-client-7c9f77bc7-ltx55\" (UID: \"1a4cf141-579e-47bd-b911-b19e20f0599c\") " pod="openshift-monitoring/telemeter-client-7c9f77bc7-ltx55" Jul 15 12:02:02.084479 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:02.084454 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmmhk\" (UniqueName: \"kubernetes.io/projected/1a4cf141-579e-47bd-b911-b19e20f0599c-kube-api-access-hmmhk\") pod \"telemeter-client-7c9f77bc7-ltx55\" (UID: \"1a4cf141-579e-47bd-b911-b19e20f0599c\") " pod="openshift-monitoring/telemeter-client-7c9f77bc7-ltx55" Jul 15 12:02:02.084602 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:02.084485 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/1a4cf141-579e-47bd-b911-b19e20f0599c-secret-telemeter-client\") pod \"telemeter-client-7c9f77bc7-ltx55\" (UID: \"1a4cf141-579e-47bd-b911-b19e20f0599c\") " pod="openshift-monitoring/telemeter-client-7c9f77bc7-ltx55" Jul 15 12:02:02.084602 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:02.084501 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/1a4cf141-579e-47bd-b911-b19e20f0599c-telemeter-client-tls\") pod \"telemeter-client-7c9f77bc7-ltx55\" (UID: \"1a4cf141-579e-47bd-b911-b19e20f0599c\") " pod="openshift-monitoring/telemeter-client-7c9f77bc7-ltx55" Jul 15 12:02:02.185441 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:02.185402 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1a4cf141-579e-47bd-b911-b19e20f0599c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7c9f77bc7-ltx55\" (UID: \"1a4cf141-579e-47bd-b911-b19e20f0599c\") " pod="openshift-monitoring/telemeter-client-7c9f77bc7-ltx55" Jul 15 12:02:02.185441 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:02.185447 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/1a4cf141-579e-47bd-b911-b19e20f0599c-federate-client-tls\") pod \"telemeter-client-7c9f77bc7-ltx55\" (UID: \"1a4cf141-579e-47bd-b911-b19e20f0599c\") " pod="openshift-monitoring/telemeter-client-7c9f77bc7-ltx55" Jul 15 12:02:02.185645 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:02.185515 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1a4cf141-579e-47bd-b911-b19e20f0599c-metrics-client-ca\") pod \"telemeter-client-7c9f77bc7-ltx55\" (UID: \"1a4cf141-579e-47bd-b911-b19e20f0599c\") " pod="openshift-monitoring/telemeter-client-7c9f77bc7-ltx55" Jul 15 12:02:02.185645 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:02.185578 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a4cf141-579e-47bd-b911-b19e20f0599c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7c9f77bc7-ltx55\" (UID: \"1a4cf141-579e-47bd-b911-b19e20f0599c\") " pod="openshift-monitoring/telemeter-client-7c9f77bc7-ltx55" Jul 15 12:02:02.185645 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:02.185610 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a4cf141-579e-47bd-b911-b19e20f0599c-serving-certs-ca-bundle\") pod \"telemeter-client-7c9f77bc7-ltx55\" (UID: \"1a4cf141-579e-47bd-b911-b19e20f0599c\") " pod="openshift-monitoring/telemeter-client-7c9f77bc7-ltx55" Jul 15 12:02:02.185645 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:02.185637 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmmhk\" (UniqueName: \"kubernetes.io/projected/1a4cf141-579e-47bd-b911-b19e20f0599c-kube-api-access-hmmhk\") pod \"telemeter-client-7c9f77bc7-ltx55\" (UID: \"1a4cf141-579e-47bd-b911-b19e20f0599c\") " pod="openshift-monitoring/telemeter-client-7c9f77bc7-ltx55" Jul 15 12:02:02.185872 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:02.185662 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/1a4cf141-579e-47bd-b911-b19e20f0599c-secret-telemeter-client\") pod \"telemeter-client-7c9f77bc7-ltx55\" (UID: \"1a4cf141-579e-47bd-b911-b19e20f0599c\") " pod="openshift-monitoring/telemeter-client-7c9f77bc7-ltx55" Jul 15 12:02:02.185872 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:02.185687 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/1a4cf141-579e-47bd-b911-b19e20f0599c-telemeter-client-tls\") pod \"telemeter-client-7c9f77bc7-ltx55\" (UID: \"1a4cf141-579e-47bd-b911-b19e20f0599c\") " pod="openshift-monitoring/telemeter-client-7c9f77bc7-ltx55" Jul 15 12:02:02.186403 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:02.186369 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a4cf141-579e-47bd-b911-b19e20f0599c-serving-certs-ca-bundle\") pod \"telemeter-client-7c9f77bc7-ltx55\" (UID: \"1a4cf141-579e-47bd-b911-b19e20f0599c\") " pod="openshift-monitoring/telemeter-client-7c9f77bc7-ltx55" Jul 15 12:02:02.186529 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:02.186479 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a4cf141-579e-47bd-b911-b19e20f0599c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7c9f77bc7-ltx55\" (UID: \"1a4cf141-579e-47bd-b911-b19e20f0599c\") " pod="openshift-monitoring/telemeter-client-7c9f77bc7-ltx55" Jul 15 12:02:02.186689 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:02.186669 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1a4cf141-579e-47bd-b911-b19e20f0599c-metrics-client-ca\") pod \"telemeter-client-7c9f77bc7-ltx55\" (UID: \"1a4cf141-579e-47bd-b911-b19e20f0599c\") " pod="openshift-monitoring/telemeter-client-7c9f77bc7-ltx55" Jul 15 12:02:02.188138 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:02.188111 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/1a4cf141-579e-47bd-b911-b19e20f0599c-federate-client-tls\") pod \"telemeter-client-7c9f77bc7-ltx55\" (UID: \"1a4cf141-579e-47bd-b911-b19e20f0599c\") " pod="openshift-monitoring/telemeter-client-7c9f77bc7-ltx55" Jul 15 12:02:02.188243 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:02.188142 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/1a4cf141-579e-47bd-b911-b19e20f0599c-secret-telemeter-client\") pod \"telemeter-client-7c9f77bc7-ltx55\" (UID: \"1a4cf141-579e-47bd-b911-b19e20f0599c\") " pod="openshift-monitoring/telemeter-client-7c9f77bc7-ltx55" Jul 15 12:02:02.188303 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:02.188269 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/1a4cf141-579e-47bd-b911-b19e20f0599c-telemeter-client-tls\") pod \"telemeter-client-7c9f77bc7-ltx55\" (UID: \"1a4cf141-579e-47bd-b911-b19e20f0599c\") " pod="openshift-monitoring/telemeter-client-7c9f77bc7-ltx55" Jul 15 12:02:02.188303 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:02.188282 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1a4cf141-579e-47bd-b911-b19e20f0599c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7c9f77bc7-ltx55\" (UID: \"1a4cf141-579e-47bd-b911-b19e20f0599c\") " pod="openshift-monitoring/telemeter-client-7c9f77bc7-ltx55" Jul 15 12:02:02.194428 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:02.194408 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmmhk\" (UniqueName: \"kubernetes.io/projected/1a4cf141-579e-47bd-b911-b19e20f0599c-kube-api-access-hmmhk\") pod \"telemeter-client-7c9f77bc7-ltx55\" (UID: \"1a4cf141-579e-47bd-b911-b19e20f0599c\") " pod="openshift-monitoring/telemeter-client-7c9f77bc7-ltx55" Jul 15 12:02:02.312658 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:02.312626 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7c9f77bc7-ltx55" Jul 15 12:02:02.452776 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:02.452750 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7c9f77bc7-ltx55"] Jul 15 12:02:02.454808 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:02:02.454779 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a4cf141_579e_47bd_b911_b19e20f0599c.slice/crio-7623442ceaedc71db0a3289111d411085649a3560f738f69e6ce9926dc25b62d WatchSource:0}: Error finding container 7623442ceaedc71db0a3289111d411085649a3560f738f69e6ce9926dc25b62d: Status 404 returned error can't find the container with id 7623442ceaedc71db0a3289111d411085649a3560f738f69e6ce9926dc25b62d Jul 15 12:02:03.041281 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:03.041239 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7c9f77bc7-ltx55" event={"ID":"1a4cf141-579e-47bd-b911-b19e20f0599c","Type":"ContainerStarted","Data":"7623442ceaedc71db0a3289111d411085649a3560f738f69e6ce9926dc25b62d"} Jul 15 12:02:05.051400 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:05.051367 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7c9f77bc7-ltx55" event={"ID":"1a4cf141-579e-47bd-b911-b19e20f0599c","Type":"ContainerStarted","Data":"7661ce8200b1f86772e5ae8070e88ca119dc947af23a12d66a21e1045f60ae9f"} Jul 15 12:02:05.051792 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:05.051429 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7c9f77bc7-ltx55" event={"ID":"1a4cf141-579e-47bd-b911-b19e20f0599c","Type":"ContainerStarted","Data":"c59ad335738710b4668852baad38a3100517649e16a6f8b260a7a1d2325d0fc8"} Jul 15 12:02:05.051792 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:05.051449 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7c9f77bc7-ltx55" event={"ID":"1a4cf141-579e-47bd-b911-b19e20f0599c","Type":"ContainerStarted","Data":"1c5ab79025f1a29e63ef4be22e3c462e6fbd74e1ccba01d375bba3e22a65e29a"} Jul 15 12:02:05.079822 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:05.079762 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7c9f77bc7-ltx55" podStartSLOduration=2.315906234 podStartE2EDuration="4.07972686s" totalImagesPullingTime="1.763820626s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:02:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:02:05.078493842 +0000 UTC m=+196.535870801" watchObservedRunningTime="2026-07-15 12:02:05.07972686 +0000 UTC m=+196.537103785" Jul 15 12:02:05.701793 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:05.701758 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5654cf4b9f-zpmzx"] Jul 15 12:02:05.704077 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:05.704054 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5654cf4b9f-zpmzx" Jul 15 12:02:05.728526 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:05.728495 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5654cf4b9f-zpmzx"] Jul 15 12:02:05.816303 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:05.816266 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/43bcae23-6450-45b1-a6e0-4175c0fefa80-service-ca\") pod \"console-5654cf4b9f-zpmzx\" (UID: \"43bcae23-6450-45b1-a6e0-4175c0fefa80\") " pod="openshift-console/console-5654cf4b9f-zpmzx" Jul 15 12:02:05.816303 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:05.816310 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/43bcae23-6450-45b1-a6e0-4175c0fefa80-console-serving-cert\") pod \"console-5654cf4b9f-zpmzx\" (UID: \"43bcae23-6450-45b1-a6e0-4175c0fefa80\") " pod="openshift-console/console-5654cf4b9f-zpmzx" Jul 15 12:02:05.816536 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:05.816332 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/43bcae23-6450-45b1-a6e0-4175c0fefa80-oauth-serving-cert\") pod \"console-5654cf4b9f-zpmzx\" (UID: \"43bcae23-6450-45b1-a6e0-4175c0fefa80\") " pod="openshift-console/console-5654cf4b9f-zpmzx" Jul 15 12:02:05.816536 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:05.816431 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43bcae23-6450-45b1-a6e0-4175c0fefa80-trusted-ca-bundle\") pod \"console-5654cf4b9f-zpmzx\" (UID: \"43bcae23-6450-45b1-a6e0-4175c0fefa80\") " pod="openshift-console/console-5654cf4b9f-zpmzx" Jul 15 12:02:05.816536 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:05.816476 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bs4vq\" (UniqueName: \"kubernetes.io/projected/43bcae23-6450-45b1-a6e0-4175c0fefa80-kube-api-access-bs4vq\") pod \"console-5654cf4b9f-zpmzx\" (UID: \"43bcae23-6450-45b1-a6e0-4175c0fefa80\") " pod="openshift-console/console-5654cf4b9f-zpmzx" Jul 15 12:02:05.816644 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:05.816541 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/43bcae23-6450-45b1-a6e0-4175c0fefa80-console-config\") pod \"console-5654cf4b9f-zpmzx\" (UID: \"43bcae23-6450-45b1-a6e0-4175c0fefa80\") " pod="openshift-console/console-5654cf4b9f-zpmzx" Jul 15 12:02:05.816644 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:05.816574 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/43bcae23-6450-45b1-a6e0-4175c0fefa80-console-oauth-config\") pod \"console-5654cf4b9f-zpmzx\" (UID: \"43bcae23-6450-45b1-a6e0-4175c0fefa80\") " pod="openshift-console/console-5654cf4b9f-zpmzx" Jul 15 12:02:05.917497 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:05.917453 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43bcae23-6450-45b1-a6e0-4175c0fefa80-trusted-ca-bundle\") pod \"console-5654cf4b9f-zpmzx\" (UID: \"43bcae23-6450-45b1-a6e0-4175c0fefa80\") " pod="openshift-console/console-5654cf4b9f-zpmzx" Jul 15 12:02:05.917497 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:05.917502 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bs4vq\" (UniqueName: \"kubernetes.io/projected/43bcae23-6450-45b1-a6e0-4175c0fefa80-kube-api-access-bs4vq\") pod \"console-5654cf4b9f-zpmzx\" (UID: \"43bcae23-6450-45b1-a6e0-4175c0fefa80\") " pod="openshift-console/console-5654cf4b9f-zpmzx" Jul 15 12:02:05.917778 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:05.917543 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/43bcae23-6450-45b1-a6e0-4175c0fefa80-console-config\") pod \"console-5654cf4b9f-zpmzx\" (UID: \"43bcae23-6450-45b1-a6e0-4175c0fefa80\") " pod="openshift-console/console-5654cf4b9f-zpmzx" Jul 15 12:02:05.917778 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:05.917576 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/43bcae23-6450-45b1-a6e0-4175c0fefa80-console-oauth-config\") pod \"console-5654cf4b9f-zpmzx\" (UID: \"43bcae23-6450-45b1-a6e0-4175c0fefa80\") " pod="openshift-console/console-5654cf4b9f-zpmzx" Jul 15 12:02:05.917778 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:05.917631 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/43bcae23-6450-45b1-a6e0-4175c0fefa80-service-ca\") pod \"console-5654cf4b9f-zpmzx\" (UID: \"43bcae23-6450-45b1-a6e0-4175c0fefa80\") " pod="openshift-console/console-5654cf4b9f-zpmzx" Jul 15 12:02:05.917926 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:05.917804 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/43bcae23-6450-45b1-a6e0-4175c0fefa80-console-serving-cert\") pod \"console-5654cf4b9f-zpmzx\" (UID: \"43bcae23-6450-45b1-a6e0-4175c0fefa80\") " pod="openshift-console/console-5654cf4b9f-zpmzx" Jul 15 12:02:05.917926 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:05.917854 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/43bcae23-6450-45b1-a6e0-4175c0fefa80-oauth-serving-cert\") pod \"console-5654cf4b9f-zpmzx\" (UID: \"43bcae23-6450-45b1-a6e0-4175c0fefa80\") " pod="openshift-console/console-5654cf4b9f-zpmzx" Jul 15 12:02:05.918431 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:05.918399 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/43bcae23-6450-45b1-a6e0-4175c0fefa80-service-ca\") pod \"console-5654cf4b9f-zpmzx\" (UID: \"43bcae23-6450-45b1-a6e0-4175c0fefa80\") " pod="openshift-console/console-5654cf4b9f-zpmzx" Jul 15 12:02:05.918559 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:05.918403 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/43bcae23-6450-45b1-a6e0-4175c0fefa80-console-config\") pod \"console-5654cf4b9f-zpmzx\" (UID: \"43bcae23-6450-45b1-a6e0-4175c0fefa80\") " pod="openshift-console/console-5654cf4b9f-zpmzx" Jul 15 12:02:05.918559 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:05.918465 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43bcae23-6450-45b1-a6e0-4175c0fefa80-trusted-ca-bundle\") pod \"console-5654cf4b9f-zpmzx\" (UID: \"43bcae23-6450-45b1-a6e0-4175c0fefa80\") " pod="openshift-console/console-5654cf4b9f-zpmzx" Jul 15 12:02:05.918559 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:05.918546 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/43bcae23-6450-45b1-a6e0-4175c0fefa80-oauth-serving-cert\") pod \"console-5654cf4b9f-zpmzx\" (UID: \"43bcae23-6450-45b1-a6e0-4175c0fefa80\") " pod="openshift-console/console-5654cf4b9f-zpmzx" Jul 15 12:02:05.920270 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:05.920249 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/43bcae23-6450-45b1-a6e0-4175c0fefa80-console-serving-cert\") pod \"console-5654cf4b9f-zpmzx\" (UID: \"43bcae23-6450-45b1-a6e0-4175c0fefa80\") " pod="openshift-console/console-5654cf4b9f-zpmzx" Jul 15 12:02:05.920332 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:05.920273 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/43bcae23-6450-45b1-a6e0-4175c0fefa80-console-oauth-config\") pod \"console-5654cf4b9f-zpmzx\" (UID: \"43bcae23-6450-45b1-a6e0-4175c0fefa80\") " pod="openshift-console/console-5654cf4b9f-zpmzx" Jul 15 12:02:05.927663 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:05.927636 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bs4vq\" (UniqueName: \"kubernetes.io/projected/43bcae23-6450-45b1-a6e0-4175c0fefa80-kube-api-access-bs4vq\") pod \"console-5654cf4b9f-zpmzx\" (UID: \"43bcae23-6450-45b1-a6e0-4175c0fefa80\") " pod="openshift-console/console-5654cf4b9f-zpmzx" Jul 15 12:02:06.013656 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:06.013548 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5654cf4b9f-zpmzx" Jul 15 12:02:06.173001 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:06.172960 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5654cf4b9f-zpmzx"] Jul 15 12:02:06.177215 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:02:06.177189 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43bcae23_6450_45b1_a6e0_4175c0fefa80.slice/crio-16fbb5b2efc19c25808dfd346dab25d7cbc409ee55c7b7d425b428c2b8bd4e9e WatchSource:0}: Error finding container 16fbb5b2efc19c25808dfd346dab25d7cbc409ee55c7b7d425b428c2b8bd4e9e: Status 404 returned error can't find the container with id 16fbb5b2efc19c25808dfd346dab25d7cbc409ee55c7b7d425b428c2b8bd4e9e Jul 15 12:02:07.062419 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:07.062383 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5654cf4b9f-zpmzx" event={"ID":"43bcae23-6450-45b1-a6e0-4175c0fefa80","Type":"ContainerStarted","Data":"aa1f11b94c0fd42ebf77a71af297b11039e855d71f521dd0b70f8959b9d2fcd8"} Jul 15 12:02:07.062572 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:07.062435 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5654cf4b9f-zpmzx" event={"ID":"43bcae23-6450-45b1-a6e0-4175c0fefa80","Type":"ContainerStarted","Data":"16fbb5b2efc19c25808dfd346dab25d7cbc409ee55c7b7d425b428c2b8bd4e9e"} Jul 15 12:02:07.082185 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:07.082132 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5654cf4b9f-zpmzx" podStartSLOduration=2.082104167 podStartE2EDuration="2.082104167s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:02:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:02:07.081574513 +0000 UTC m=+198.538951451" watchObservedRunningTime="2026-07-15 12:02:07.082104167 +0000 UTC m=+198.539481091" Jul 15 12:02:16.014094 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:16.014051 2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5654cf4b9f-zpmzx" Jul 15 12:02:16.014516 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:16.014123 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5654cf4b9f-zpmzx" Jul 15 12:02:16.018884 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:16.018858 2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5654cf4b9f-zpmzx" Jul 15 12:02:16.100507 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:16.100476 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5654cf4b9f-zpmzx" Jul 15 12:02:16.148455 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:16.148421 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59977679d8-kh5dp"] Jul 15 12:02:16.242564 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:16.242530 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-75cdfcc548-8bjw6"] Jul 15 12:02:16.245763 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:16.245742 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75cdfcc548-8bjw6" Jul 15 12:02:16.257026 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:16.256999 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75cdfcc548-8bjw6"] Jul 15 12:02:16.408831 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:16.408789 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d958cc3b-aa17-48e1-84b6-9bfa9bfd4307-console-serving-cert\") pod \"console-75cdfcc548-8bjw6\" (UID: \"d958cc3b-aa17-48e1-84b6-9bfa9bfd4307\") " pod="openshift-console/console-75cdfcc548-8bjw6" Jul 15 12:02:16.408831 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:16.408833 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d958cc3b-aa17-48e1-84b6-9bfa9bfd4307-oauth-serving-cert\") pod \"console-75cdfcc548-8bjw6\" (UID: \"d958cc3b-aa17-48e1-84b6-9bfa9bfd4307\") " pod="openshift-console/console-75cdfcc548-8bjw6" Jul 15 12:02:16.409066 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:16.408869 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d958cc3b-aa17-48e1-84b6-9bfa9bfd4307-console-oauth-config\") pod \"console-75cdfcc548-8bjw6\" (UID: \"d958cc3b-aa17-48e1-84b6-9bfa9bfd4307\") " pod="openshift-console/console-75cdfcc548-8bjw6" Jul 15 12:02:16.409066 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:16.408891 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d958cc3b-aa17-48e1-84b6-9bfa9bfd4307-console-config\") pod \"console-75cdfcc548-8bjw6\" (UID: \"d958cc3b-aa17-48e1-84b6-9bfa9bfd4307\") " pod="openshift-console/console-75cdfcc548-8bjw6" Jul 15 12:02:16.409066 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:16.408909 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d958cc3b-aa17-48e1-84b6-9bfa9bfd4307-service-ca\") pod \"console-75cdfcc548-8bjw6\" (UID: \"d958cc3b-aa17-48e1-84b6-9bfa9bfd4307\") " pod="openshift-console/console-75cdfcc548-8bjw6" Jul 15 12:02:16.409066 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:16.408926 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d958cc3b-aa17-48e1-84b6-9bfa9bfd4307-trusted-ca-bundle\") pod \"console-75cdfcc548-8bjw6\" (UID: \"d958cc3b-aa17-48e1-84b6-9bfa9bfd4307\") " pod="openshift-console/console-75cdfcc548-8bjw6" Jul 15 12:02:16.409066 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:16.408942 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfqst\" (UniqueName: \"kubernetes.io/projected/d958cc3b-aa17-48e1-84b6-9bfa9bfd4307-kube-api-access-hfqst\") pod \"console-75cdfcc548-8bjw6\" (UID: \"d958cc3b-aa17-48e1-84b6-9bfa9bfd4307\") " pod="openshift-console/console-75cdfcc548-8bjw6" Jul 15 12:02:16.510364 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:16.510325 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d958cc3b-aa17-48e1-84b6-9bfa9bfd4307-console-serving-cert\") pod \"console-75cdfcc548-8bjw6\" (UID: \"d958cc3b-aa17-48e1-84b6-9bfa9bfd4307\") " pod="openshift-console/console-75cdfcc548-8bjw6" Jul 15 12:02:16.510364 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:16.510368 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d958cc3b-aa17-48e1-84b6-9bfa9bfd4307-oauth-serving-cert\") pod \"console-75cdfcc548-8bjw6\" (UID: \"d958cc3b-aa17-48e1-84b6-9bfa9bfd4307\") " pod="openshift-console/console-75cdfcc548-8bjw6" Jul 15 12:02:16.510578 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:16.510419 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d958cc3b-aa17-48e1-84b6-9bfa9bfd4307-console-oauth-config\") pod \"console-75cdfcc548-8bjw6\" (UID: \"d958cc3b-aa17-48e1-84b6-9bfa9bfd4307\") " pod="openshift-console/console-75cdfcc548-8bjw6" Jul 15 12:02:16.510578 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:16.510448 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d958cc3b-aa17-48e1-84b6-9bfa9bfd4307-console-config\") pod \"console-75cdfcc548-8bjw6\" (UID: \"d958cc3b-aa17-48e1-84b6-9bfa9bfd4307\") " pod="openshift-console/console-75cdfcc548-8bjw6" Jul 15 12:02:16.510578 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:16.510471 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d958cc3b-aa17-48e1-84b6-9bfa9bfd4307-service-ca\") pod \"console-75cdfcc548-8bjw6\" (UID: \"d958cc3b-aa17-48e1-84b6-9bfa9bfd4307\") " pod="openshift-console/console-75cdfcc548-8bjw6" Jul 15 12:02:16.510578 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:16.510492 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d958cc3b-aa17-48e1-84b6-9bfa9bfd4307-trusted-ca-bundle\") pod \"console-75cdfcc548-8bjw6\" (UID: \"d958cc3b-aa17-48e1-84b6-9bfa9bfd4307\") " pod="openshift-console/console-75cdfcc548-8bjw6" Jul 15 12:02:16.510578 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:16.510514 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfqst\" (UniqueName: \"kubernetes.io/projected/d958cc3b-aa17-48e1-84b6-9bfa9bfd4307-kube-api-access-hfqst\") pod \"console-75cdfcc548-8bjw6\" (UID: \"d958cc3b-aa17-48e1-84b6-9bfa9bfd4307\") " pod="openshift-console/console-75cdfcc548-8bjw6" Jul 15 12:02:16.511340 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:16.511312 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d958cc3b-aa17-48e1-84b6-9bfa9bfd4307-oauth-serving-cert\") pod \"console-75cdfcc548-8bjw6\" (UID: \"d958cc3b-aa17-48e1-84b6-9bfa9bfd4307\") " pod="openshift-console/console-75cdfcc548-8bjw6" Jul 15 12:02:16.511464 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:16.511350 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d958cc3b-aa17-48e1-84b6-9bfa9bfd4307-console-config\") pod \"console-75cdfcc548-8bjw6\" (UID: \"d958cc3b-aa17-48e1-84b6-9bfa9bfd4307\") " pod="openshift-console/console-75cdfcc548-8bjw6" Jul 15 12:02:16.511464 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:16.511312 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d958cc3b-aa17-48e1-84b6-9bfa9bfd4307-service-ca\") pod \"console-75cdfcc548-8bjw6\" (UID: \"d958cc3b-aa17-48e1-84b6-9bfa9bfd4307\") " pod="openshift-console/console-75cdfcc548-8bjw6" Jul 15 12:02:16.511654 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:16.511631 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d958cc3b-aa17-48e1-84b6-9bfa9bfd4307-trusted-ca-bundle\") pod \"console-75cdfcc548-8bjw6\" (UID: \"d958cc3b-aa17-48e1-84b6-9bfa9bfd4307\") " pod="openshift-console/console-75cdfcc548-8bjw6" Jul 15 12:02:16.512949 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:16.512917 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d958cc3b-aa17-48e1-84b6-9bfa9bfd4307-console-oauth-config\") pod \"console-75cdfcc548-8bjw6\" (UID: \"d958cc3b-aa17-48e1-84b6-9bfa9bfd4307\") " pod="openshift-console/console-75cdfcc548-8bjw6" Jul 15 12:02:16.513067 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:16.513049 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d958cc3b-aa17-48e1-84b6-9bfa9bfd4307-console-serving-cert\") pod \"console-75cdfcc548-8bjw6\" (UID: \"d958cc3b-aa17-48e1-84b6-9bfa9bfd4307\") " pod="openshift-console/console-75cdfcc548-8bjw6" Jul 15 12:02:16.520641 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:16.520618 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfqst\" (UniqueName: \"kubernetes.io/projected/d958cc3b-aa17-48e1-84b6-9bfa9bfd4307-kube-api-access-hfqst\") pod \"console-75cdfcc548-8bjw6\" (UID: \"d958cc3b-aa17-48e1-84b6-9bfa9bfd4307\") " pod="openshift-console/console-75cdfcc548-8bjw6" Jul 15 12:02:16.555619 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:16.555582 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75cdfcc548-8bjw6" Jul 15 12:02:16.688773 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:16.688617 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75cdfcc548-8bjw6"] Jul 15 12:02:16.691414 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:02:16.691384 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd958cc3b_aa17_48e1_84b6_9bfa9bfd4307.slice/crio-4ef0ccbfc1b8d92b6451a6072e564cf4bede2432ad2861ba7fa8675ca7e468ac WatchSource:0}: Error finding container 4ef0ccbfc1b8d92b6451a6072e564cf4bede2432ad2861ba7fa8675ca7e468ac: Status 404 returned error can't find the container with id 4ef0ccbfc1b8d92b6451a6072e564cf4bede2432ad2861ba7fa8675ca7e468ac Jul 15 12:02:17.101018 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:17.100982 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75cdfcc548-8bjw6" event={"ID":"d958cc3b-aa17-48e1-84b6-9bfa9bfd4307","Type":"ContainerStarted","Data":"2f974203c065d3ba0bb2a8110b8b9e2c5a43b5354eb0961a62cf3a533804bd2b"} Jul 15 12:02:17.101383 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:17.101027 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75cdfcc548-8bjw6" event={"ID":"d958cc3b-aa17-48e1-84b6-9bfa9bfd4307","Type":"ContainerStarted","Data":"4ef0ccbfc1b8d92b6451a6072e564cf4bede2432ad2861ba7fa8675ca7e468ac"} Jul 15 12:02:17.120155 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:17.120107 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-75cdfcc548-8bjw6" podStartSLOduration=1.12009145 podStartE2EDuration="1.12009145s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:02:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:02:17.119299852 +0000 UTC m=+208.576676802" watchObservedRunningTime="2026-07-15 12:02:17.12009145 +0000 UTC m=+208.577468374" Jul 15 12:02:26.556708 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:26.556653 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-75cdfcc548-8bjw6" Jul 15 12:02:26.557203 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:26.556752 2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-75cdfcc548-8bjw6" Jul 15 12:02:26.561587 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:26.561561 2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-75cdfcc548-8bjw6" Jul 15 12:02:27.144798 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:27.144769 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-75cdfcc548-8bjw6" Jul 15 12:02:27.193578 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:27.193545 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5654cf4b9f-zpmzx"] Jul 15 12:02:41.169303 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:41.169263 2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-59977679d8-kh5dp" podUID="e7b17bf5-1086-48f3-a2a3-336e97deeec3" containerName="console" containerID="cri-o://bfb19eefc825d3b515c4fa25369897343cc6628f4e867ef6ba528aeb1088a938" gracePeriod=15 Jul 15 12:02:41.410319 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:41.410293 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59977679d8-kh5dp" Jul 15 12:02:41.522180 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:41.522079 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e7b17bf5-1086-48f3-a2a3-336e97deeec3-service-ca\") pod \"e7b17bf5-1086-48f3-a2a3-336e97deeec3\" (UID: \"e7b17bf5-1086-48f3-a2a3-336e97deeec3\") " Jul 15 12:02:41.522180 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:41.522186 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e7b17bf5-1086-48f3-a2a3-336e97deeec3-oauth-serving-cert\") pod \"e7b17bf5-1086-48f3-a2a3-336e97deeec3\" (UID: \"e7b17bf5-1086-48f3-a2a3-336e97deeec3\") " Jul 15 12:02:41.522394 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:41.522216 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e7b17bf5-1086-48f3-a2a3-336e97deeec3-console-config\") pod \"e7b17bf5-1086-48f3-a2a3-336e97deeec3\" (UID: \"e7b17bf5-1086-48f3-a2a3-336e97deeec3\") " Jul 15 12:02:41.522394 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:41.522232 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vq6db\" (UniqueName: \"kubernetes.io/projected/e7b17bf5-1086-48f3-a2a3-336e97deeec3-kube-api-access-vq6db\") pod \"e7b17bf5-1086-48f3-a2a3-336e97deeec3\" (UID: \"e7b17bf5-1086-48f3-a2a3-336e97deeec3\") " Jul 15 12:02:41.522394 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:41.522266 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e7b17bf5-1086-48f3-a2a3-336e97deeec3-console-serving-cert\") pod \"e7b17bf5-1086-48f3-a2a3-336e97deeec3\" (UID: \"e7b17bf5-1086-48f3-a2a3-336e97deeec3\") " Jul 15 12:02:41.522394 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:41.522353 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e7b17bf5-1086-48f3-a2a3-336e97deeec3-console-oauth-config\") pod \"e7b17bf5-1086-48f3-a2a3-336e97deeec3\" (UID: \"e7b17bf5-1086-48f3-a2a3-336e97deeec3\") " Jul 15 12:02:41.522589 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:41.522459 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7b17bf5-1086-48f3-a2a3-336e97deeec3-trusted-ca-bundle\") pod \"e7b17bf5-1086-48f3-a2a3-336e97deeec3\" (UID: \"e7b17bf5-1086-48f3-a2a3-336e97deeec3\") " Jul 15 12:02:41.522589 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:41.522574 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7b17bf5-1086-48f3-a2a3-336e97deeec3-service-ca" (OuterVolumeSpecName: "service-ca") pod "e7b17bf5-1086-48f3-a2a3-336e97deeec3" (UID: "e7b17bf5-1086-48f3-a2a3-336e97deeec3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:02:41.522712 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:41.522672 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7b17bf5-1086-48f3-a2a3-336e97deeec3-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e7b17bf5-1086-48f3-a2a3-336e97deeec3" (UID: "e7b17bf5-1086-48f3-a2a3-336e97deeec3"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:02:41.522802 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:41.522680 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7b17bf5-1086-48f3-a2a3-336e97deeec3-console-config" (OuterVolumeSpecName: "console-config") pod "e7b17bf5-1086-48f3-a2a3-336e97deeec3" (UID: "e7b17bf5-1086-48f3-a2a3-336e97deeec3"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:02:41.522869 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:41.522804 2587 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e7b17bf5-1086-48f3-a2a3-336e97deeec3-service-ca\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:02:41.522937 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:41.522922 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7b17bf5-1086-48f3-a2a3-336e97deeec3-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e7b17bf5-1086-48f3-a2a3-336e97deeec3" (UID: "e7b17bf5-1086-48f3-a2a3-336e97deeec3"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:02:41.524533 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:41.524503 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7b17bf5-1086-48f3-a2a3-336e97deeec3-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e7b17bf5-1086-48f3-a2a3-336e97deeec3" (UID: "e7b17bf5-1086-48f3-a2a3-336e97deeec3"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 12:02:41.524646 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:41.524550 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7b17bf5-1086-48f3-a2a3-336e97deeec3-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e7b17bf5-1086-48f3-a2a3-336e97deeec3" (UID: "e7b17bf5-1086-48f3-a2a3-336e97deeec3"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 12:02:41.524646 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:41.524586 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7b17bf5-1086-48f3-a2a3-336e97deeec3-kube-api-access-vq6db" (OuterVolumeSpecName: "kube-api-access-vq6db") pod "e7b17bf5-1086-48f3-a2a3-336e97deeec3" (UID: "e7b17bf5-1086-48f3-a2a3-336e97deeec3"). InnerVolumeSpecName "kube-api-access-vq6db". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:02:41.623528 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:41.623472 2587 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e7b17bf5-1086-48f3-a2a3-336e97deeec3-oauth-serving-cert\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:02:41.623528 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:41.623518 2587 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e7b17bf5-1086-48f3-a2a3-336e97deeec3-console-config\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:02:41.623528 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:41.623529 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vq6db\" (UniqueName: \"kubernetes.io/projected/e7b17bf5-1086-48f3-a2a3-336e97deeec3-kube-api-access-vq6db\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:02:41.623528 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:41.623538 2587 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e7b17bf5-1086-48f3-a2a3-336e97deeec3-console-serving-cert\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:02:41.623528 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:41.623547 2587 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e7b17bf5-1086-48f3-a2a3-336e97deeec3-console-oauth-config\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:02:41.623851 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:41.623555 2587 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7b17bf5-1086-48f3-a2a3-336e97deeec3-trusted-ca-bundle\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:02:42.205276 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:42.205240 2587 generic.go:356] "Generic (PLEG): container finished" podID="e7b17bf5-1086-48f3-a2a3-336e97deeec3" containerID="bfb19eefc825d3b515c4fa25369897343cc6628f4e867ef6ba528aeb1088a938" exitCode=0 Jul 15 12:02:42.205651 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:42.205291 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59977679d8-kh5dp" Jul 15 12:02:42.205651 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:42.205294 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59977679d8-kh5dp" event={"ID":"e7b17bf5-1086-48f3-a2a3-336e97deeec3","Type":"ContainerDied","Data":"bfb19eefc825d3b515c4fa25369897343cc6628f4e867ef6ba528aeb1088a938"} Jul 15 12:02:42.205651 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:42.205418 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59977679d8-kh5dp" event={"ID":"e7b17bf5-1086-48f3-a2a3-336e97deeec3","Type":"ContainerDied","Data":"bb78e72bc3ec7407e8f8d0dff473ba781d00e6a2f38cf77ef1133f42a2f1e3d9"} Jul 15 12:02:42.205651 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:42.205450 2587 scope.go:117] "RemoveContainer" containerID="bfb19eefc825d3b515c4fa25369897343cc6628f4e867ef6ba528aeb1088a938" Jul 15 12:02:42.214333 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:42.214316 2587 scope.go:117] "RemoveContainer" containerID="bfb19eefc825d3b515c4fa25369897343cc6628f4e867ef6ba528aeb1088a938" Jul 15 12:02:42.214583 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:02:42.214564 2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfb19eefc825d3b515c4fa25369897343cc6628f4e867ef6ba528aeb1088a938\": container with ID starting with bfb19eefc825d3b515c4fa25369897343cc6628f4e867ef6ba528aeb1088a938 not found: ID does not exist" containerID="bfb19eefc825d3b515c4fa25369897343cc6628f4e867ef6ba528aeb1088a938" Jul 15 12:02:42.214624 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:42.214591 2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfb19eefc825d3b515c4fa25369897343cc6628f4e867ef6ba528aeb1088a938"} err="failed to get container status \"bfb19eefc825d3b515c4fa25369897343cc6628f4e867ef6ba528aeb1088a938\": rpc error: code = NotFound desc = could not find container \"bfb19eefc825d3b515c4fa25369897343cc6628f4e867ef6ba528aeb1088a938\": container with ID starting with bfb19eefc825d3b515c4fa25369897343cc6628f4e867ef6ba528aeb1088a938 not found: ID does not exist" Jul 15 12:02:42.226761 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:42.226712 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59977679d8-kh5dp"] Jul 15 12:02:42.227981 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:42.227962 2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-59977679d8-kh5dp"] Jul 15 12:02:43.183570 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:43.183529 2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7b17bf5-1086-48f3-a2a3-336e97deeec3" path="/var/lib/kubelet/pods/e7b17bf5-1086-48f3-a2a3-336e97deeec3/volumes" Jul 15 12:02:52.217862 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:52.217800 2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5654cf4b9f-zpmzx" podUID="43bcae23-6450-45b1-a6e0-4175c0fefa80" containerName="console" containerID="cri-o://aa1f11b94c0fd42ebf77a71af297b11039e855d71f521dd0b70f8959b9d2fcd8" gracePeriod=15 Jul 15 12:02:52.459343 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:52.459319 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5654cf4b9f-zpmzx" Jul 15 12:02:52.616393 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:52.616357 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/43bcae23-6450-45b1-a6e0-4175c0fefa80-oauth-serving-cert\") pod \"43bcae23-6450-45b1-a6e0-4175c0fefa80\" (UID: \"43bcae23-6450-45b1-a6e0-4175c0fefa80\") " Jul 15 12:02:52.616546 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:52.616424 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/43bcae23-6450-45b1-a6e0-4175c0fefa80-console-oauth-config\") pod \"43bcae23-6450-45b1-a6e0-4175c0fefa80\" (UID: \"43bcae23-6450-45b1-a6e0-4175c0fefa80\") " Jul 15 12:02:52.616546 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:52.616449 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43bcae23-6450-45b1-a6e0-4175c0fefa80-trusted-ca-bundle\") pod \"43bcae23-6450-45b1-a6e0-4175c0fefa80\" (UID: \"43bcae23-6450-45b1-a6e0-4175c0fefa80\") " Jul 15 12:02:52.616546 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:52.616472 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/43bcae23-6450-45b1-a6e0-4175c0fefa80-service-ca\") pod \"43bcae23-6450-45b1-a6e0-4175c0fefa80\" (UID: \"43bcae23-6450-45b1-a6e0-4175c0fefa80\") " Jul 15 12:02:52.616546 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:52.616496 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/43bcae23-6450-45b1-a6e0-4175c0fefa80-console-serving-cert\") pod \"43bcae23-6450-45b1-a6e0-4175c0fefa80\" (UID: \"43bcae23-6450-45b1-a6e0-4175c0fefa80\") " Jul 15 12:02:52.616546 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:52.616536 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bs4vq\" (UniqueName: \"kubernetes.io/projected/43bcae23-6450-45b1-a6e0-4175c0fefa80-kube-api-access-bs4vq\") pod \"43bcae23-6450-45b1-a6e0-4175c0fefa80\" (UID: \"43bcae23-6450-45b1-a6e0-4175c0fefa80\") " Jul 15 12:02:52.616811 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:52.616580 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/43bcae23-6450-45b1-a6e0-4175c0fefa80-console-config\") pod \"43bcae23-6450-45b1-a6e0-4175c0fefa80\" (UID: \"43bcae23-6450-45b1-a6e0-4175c0fefa80\") " Jul 15 12:02:52.617003 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:52.616975 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43bcae23-6450-45b1-a6e0-4175c0fefa80-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "43bcae23-6450-45b1-a6e0-4175c0fefa80" (UID: "43bcae23-6450-45b1-a6e0-4175c0fefa80"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:02:52.617096 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:52.617008 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43bcae23-6450-45b1-a6e0-4175c0fefa80-service-ca" (OuterVolumeSpecName: "service-ca") pod "43bcae23-6450-45b1-a6e0-4175c0fefa80" (UID: "43bcae23-6450-45b1-a6e0-4175c0fefa80"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:02:52.617096 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:52.617026 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43bcae23-6450-45b1-a6e0-4175c0fefa80-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "43bcae23-6450-45b1-a6e0-4175c0fefa80" (UID: "43bcae23-6450-45b1-a6e0-4175c0fefa80"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:02:52.617196 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:52.617118 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43bcae23-6450-45b1-a6e0-4175c0fefa80-console-config" (OuterVolumeSpecName: "console-config") pod "43bcae23-6450-45b1-a6e0-4175c0fefa80" (UID: "43bcae23-6450-45b1-a6e0-4175c0fefa80"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:02:52.618784 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:52.618746 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43bcae23-6450-45b1-a6e0-4175c0fefa80-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "43bcae23-6450-45b1-a6e0-4175c0fefa80" (UID: "43bcae23-6450-45b1-a6e0-4175c0fefa80"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 12:02:52.618784 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:52.618771 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43bcae23-6450-45b1-a6e0-4175c0fefa80-kube-api-access-bs4vq" (OuterVolumeSpecName: "kube-api-access-bs4vq") pod "43bcae23-6450-45b1-a6e0-4175c0fefa80" (UID: "43bcae23-6450-45b1-a6e0-4175c0fefa80"). InnerVolumeSpecName "kube-api-access-bs4vq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:02:52.618954 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:52.618822 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43bcae23-6450-45b1-a6e0-4175c0fefa80-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "43bcae23-6450-45b1-a6e0-4175c0fefa80" (UID: "43bcae23-6450-45b1-a6e0-4175c0fefa80"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 12:02:52.718062 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:52.718023 2587 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/43bcae23-6450-45b1-a6e0-4175c0fefa80-console-oauth-config\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:02:52.718062 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:52.718055 2587 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43bcae23-6450-45b1-a6e0-4175c0fefa80-trusted-ca-bundle\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:02:52.718062 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:52.718070 2587 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/43bcae23-6450-45b1-a6e0-4175c0fefa80-service-ca\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:02:52.718306 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:52.718083 2587 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/43bcae23-6450-45b1-a6e0-4175c0fefa80-console-serving-cert\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:02:52.718306 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:52.718096 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bs4vq\" (UniqueName: \"kubernetes.io/projected/43bcae23-6450-45b1-a6e0-4175c0fefa80-kube-api-access-bs4vq\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:02:52.718306 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:52.718108 2587 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/43bcae23-6450-45b1-a6e0-4175c0fefa80-console-config\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:02:52.718306 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:52.718119 2587 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/43bcae23-6450-45b1-a6e0-4175c0fefa80-oauth-serving-cert\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:02:53.245852 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:53.245817 2587 generic.go:356] "Generic (PLEG): container finished" podID="43bcae23-6450-45b1-a6e0-4175c0fefa80" containerID="aa1f11b94c0fd42ebf77a71af297b11039e855d71f521dd0b70f8959b9d2fcd8" exitCode=0 Jul 15 12:02:53.246197 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:53.245865 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5654cf4b9f-zpmzx" Jul 15 12:02:53.246197 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:53.245869 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5654cf4b9f-zpmzx" event={"ID":"43bcae23-6450-45b1-a6e0-4175c0fefa80","Type":"ContainerDied","Data":"aa1f11b94c0fd42ebf77a71af297b11039e855d71f521dd0b70f8959b9d2fcd8"} Jul 15 12:02:53.246197 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:53.245918 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5654cf4b9f-zpmzx" event={"ID":"43bcae23-6450-45b1-a6e0-4175c0fefa80","Type":"ContainerDied","Data":"16fbb5b2efc19c25808dfd346dab25d7cbc409ee55c7b7d425b428c2b8bd4e9e"} Jul 15 12:02:53.246197 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:53.245944 2587 scope.go:117] "RemoveContainer" containerID="aa1f11b94c0fd42ebf77a71af297b11039e855d71f521dd0b70f8959b9d2fcd8" Jul 15 12:02:53.253966 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:53.253948 2587 scope.go:117] "RemoveContainer" containerID="aa1f11b94c0fd42ebf77a71af297b11039e855d71f521dd0b70f8959b9d2fcd8" Jul 15 12:02:53.254207 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:02:53.254185 2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa1f11b94c0fd42ebf77a71af297b11039e855d71f521dd0b70f8959b9d2fcd8\": container with ID starting with aa1f11b94c0fd42ebf77a71af297b11039e855d71f521dd0b70f8959b9d2fcd8 not found: ID does not exist" containerID="aa1f11b94c0fd42ebf77a71af297b11039e855d71f521dd0b70f8959b9d2fcd8" Jul 15 12:02:53.254260 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:53.254215 2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa1f11b94c0fd42ebf77a71af297b11039e855d71f521dd0b70f8959b9d2fcd8"} err="failed to get container status \"aa1f11b94c0fd42ebf77a71af297b11039e855d71f521dd0b70f8959b9d2fcd8\": rpc error: code = NotFound desc = could not find container \"aa1f11b94c0fd42ebf77a71af297b11039e855d71f521dd0b70f8959b9d2fcd8\": container with ID starting with aa1f11b94c0fd42ebf77a71af297b11039e855d71f521dd0b70f8959b9d2fcd8 not found: ID does not exist" Jul 15 12:02:53.262699 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:53.262678 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5654cf4b9f-zpmzx"] Jul 15 12:02:53.267156 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:53.267134 2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5654cf4b9f-zpmzx"] Jul 15 12:02:55.184109 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:02:55.184077 2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43bcae23-6450-45b1-a6e0-4175c0fefa80" path="/var/lib/kubelet/pods/43bcae23-6450-45b1-a6e0-4175c0fefa80/volumes" Jul 15 12:03:31.578775 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:31.578724 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v46jx"] Jul 15 12:03:31.579188 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:31.579018 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43bcae23-6450-45b1-a6e0-4175c0fefa80" containerName="console" Jul 15 12:03:31.579188 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:31.579028 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="43bcae23-6450-45b1-a6e0-4175c0fefa80" containerName="console" Jul 15 12:03:31.579188 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:31.579049 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7b17bf5-1086-48f3-a2a3-336e97deeec3" containerName="console" Jul 15 12:03:31.579188 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:31.579054 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7b17bf5-1086-48f3-a2a3-336e97deeec3" containerName="console" Jul 15 12:03:31.579188 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:31.579120 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7b17bf5-1086-48f3-a2a3-336e97deeec3" containerName="console" Jul 15 12:03:31.579188 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:31.579128 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="43bcae23-6450-45b1-a6e0-4175c0fefa80" containerName="console" Jul 15 12:03:31.582154 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:31.582137 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v46jx" Jul 15 12:03:31.584836 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:31.584815 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-qn677\"" Jul 15 12:03:31.584949 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:31.584868 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 15 12:03:31.585910 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:31.585895 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 15 12:03:31.592576 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:31.592554 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v46jx"] Jul 15 12:03:31.621636 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:31.621604 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f181207-f4c4-4db5-b1c5-15bcb21bf1b1-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v46jx\" (UID: \"2f181207-f4c4-4db5-b1c5-15bcb21bf1b1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v46jx" Jul 15 12:03:31.621848 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:31.621644 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f181207-f4c4-4db5-b1c5-15bcb21bf1b1-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v46jx\" (UID: \"2f181207-f4c4-4db5-b1c5-15bcb21bf1b1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v46jx" Jul 15 12:03:31.621848 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:31.621807 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67wbn\" (UniqueName: \"kubernetes.io/projected/2f181207-f4c4-4db5-b1c5-15bcb21bf1b1-kube-api-access-67wbn\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v46jx\" (UID: \"2f181207-f4c4-4db5-b1c5-15bcb21bf1b1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v46jx" Jul 15 12:03:31.723055 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:31.723011 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67wbn\" (UniqueName: \"kubernetes.io/projected/2f181207-f4c4-4db5-b1c5-15bcb21bf1b1-kube-api-access-67wbn\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v46jx\" (UID: \"2f181207-f4c4-4db5-b1c5-15bcb21bf1b1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v46jx" Jul 15 12:03:31.723055 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:31.723063 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f181207-f4c4-4db5-b1c5-15bcb21bf1b1-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v46jx\" (UID: \"2f181207-f4c4-4db5-b1c5-15bcb21bf1b1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v46jx" Jul 15 12:03:31.723322 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:31.723177 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f181207-f4c4-4db5-b1c5-15bcb21bf1b1-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v46jx\" (UID: \"2f181207-f4c4-4db5-b1c5-15bcb21bf1b1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v46jx" Jul 15 12:03:31.723525 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:31.723501 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f181207-f4c4-4db5-b1c5-15bcb21bf1b1-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v46jx\" (UID: \"2f181207-f4c4-4db5-b1c5-15bcb21bf1b1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v46jx" Jul 15 12:03:31.723584 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:31.723527 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f181207-f4c4-4db5-b1c5-15bcb21bf1b1-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v46jx\" (UID: \"2f181207-f4c4-4db5-b1c5-15bcb21bf1b1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v46jx" Jul 15 12:03:31.731682 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:31.731652 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-67wbn\" (UniqueName: \"kubernetes.io/projected/2f181207-f4c4-4db5-b1c5-15bcb21bf1b1-kube-api-access-67wbn\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v46jx\" (UID: \"2f181207-f4c4-4db5-b1c5-15bcb21bf1b1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v46jx" Jul 15 12:03:31.891469 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:31.891375 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v46jx" Jul 15 12:03:32.022001 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:32.021969 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v46jx"] Jul 15 12:03:32.024890 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:03:32.024850 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f181207_f4c4_4db5_b1c5_15bcb21bf1b1.slice/crio-45d26ef15000057a017e720a12a0ef1b680ae68799b10ece20d7f7d058c18caa WatchSource:0}: Error finding container 45d26ef15000057a017e720a12a0ef1b680ae68799b10ece20d7f7d058c18caa: Status 404 returned error can't find the container with id 45d26ef15000057a017e720a12a0ef1b680ae68799b10ece20d7f7d058c18caa Jul 15 12:03:32.393391 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:32.393352 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v46jx" event={"ID":"2f181207-f4c4-4db5-b1c5-15bcb21bf1b1","Type":"ContainerStarted","Data":"45d26ef15000057a017e720a12a0ef1b680ae68799b10ece20d7f7d058c18caa"} Jul 15 12:03:39.425183 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:39.425145 2587 generic.go:356] "Generic (PLEG): container finished" podID="2f181207-f4c4-4db5-b1c5-15bcb21bf1b1" containerID="7ffce98d2b2f7c44106fabd76cce79865734514e82a87b5d14876949688af0ab" exitCode=0 Jul 15 12:03:39.425563 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:39.425229 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v46jx" event={"ID":"2f181207-f4c4-4db5-b1c5-15bcb21bf1b1","Type":"ContainerDied","Data":"7ffce98d2b2f7c44106fabd76cce79865734514e82a87b5d14876949688af0ab"} Jul 15 12:03:41.435779 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:41.435745 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v46jx" event={"ID":"2f181207-f4c4-4db5-b1c5-15bcb21bf1b1","Type":"ContainerStarted","Data":"f3a9cd8032144117f732c7103efb58a4d3608094f9286d88e788a6e11c71ee06"} Jul 15 12:03:42.441706 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:42.441668 2587 generic.go:356] "Generic (PLEG): container finished" podID="2f181207-f4c4-4db5-b1c5-15bcb21bf1b1" containerID="f3a9cd8032144117f732c7103efb58a4d3608094f9286d88e788a6e11c71ee06" exitCode=0 Jul 15 12:03:42.442097 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:42.441753 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v46jx" event={"ID":"2f181207-f4c4-4db5-b1c5-15bcb21bf1b1","Type":"ContainerDied","Data":"f3a9cd8032144117f732c7103efb58a4d3608094f9286d88e788a6e11c71ee06"} Jul 15 12:03:49.012535 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:49.012501 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-4xrn6_85ce5adf-d361-4258-b4ca-7f13d35fe13d/console-operator/1.log" Jul 15 12:03:49.013149 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:49.013093 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-4xrn6_85ce5adf-d361-4258-b4ca-7f13d35fe13d/console-operator/1.log" Jul 15 12:03:49.021168 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:49.021146 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mcrkx_ea415947-4742-4cf7-8b71-e22b418a131f/ovn-acl-logging/0.log" Jul 15 12:03:49.021364 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:49.021351 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mcrkx_ea415947-4742-4cf7-8b71-e22b418a131f/ovn-acl-logging/0.log" Jul 15 12:03:49.025171 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:49.025147 2587 kubelet.go:1648] "Image garbage collection succeeded" Jul 15 12:03:52.490212 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:52.490172 2587 generic.go:356] "Generic (PLEG): container finished" podID="2f181207-f4c4-4db5-b1c5-15bcb21bf1b1" containerID="04169c0bc10e01c491d93d4dfca70f75487a09a79969c1518879c9660c1bc49a" exitCode=0 Jul 15 12:03:52.490212 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:52.490216 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v46jx" event={"ID":"2f181207-f4c4-4db5-b1c5-15bcb21bf1b1","Type":"ContainerDied","Data":"04169c0bc10e01c491d93d4dfca70f75487a09a79969c1518879c9660c1bc49a"} Jul 15 12:03:53.659890 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:53.659854 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v46jx" Jul 15 12:03:53.808614 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:53.808580 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f181207-f4c4-4db5-b1c5-15bcb21bf1b1-bundle\") pod \"2f181207-f4c4-4db5-b1c5-15bcb21bf1b1\" (UID: \"2f181207-f4c4-4db5-b1c5-15bcb21bf1b1\") " Jul 15 12:03:53.808835 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:53.808650 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-67wbn\" (UniqueName: \"kubernetes.io/projected/2f181207-f4c4-4db5-b1c5-15bcb21bf1b1-kube-api-access-67wbn\") pod \"2f181207-f4c4-4db5-b1c5-15bcb21bf1b1\" (UID: \"2f181207-f4c4-4db5-b1c5-15bcb21bf1b1\") " Jul 15 12:03:53.808835 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:53.808681 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f181207-f4c4-4db5-b1c5-15bcb21bf1b1-util\") pod \"2f181207-f4c4-4db5-b1c5-15bcb21bf1b1\" (UID: \"2f181207-f4c4-4db5-b1c5-15bcb21bf1b1\") " Jul 15 12:03:53.809323 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:53.809293 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f181207-f4c4-4db5-b1c5-15bcb21bf1b1-bundle" (OuterVolumeSpecName: "bundle") pod "2f181207-f4c4-4db5-b1c5-15bcb21bf1b1" (UID: "2f181207-f4c4-4db5-b1c5-15bcb21bf1b1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:03:53.810881 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:53.810850 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f181207-f4c4-4db5-b1c5-15bcb21bf1b1-kube-api-access-67wbn" (OuterVolumeSpecName: "kube-api-access-67wbn") pod "2f181207-f4c4-4db5-b1c5-15bcb21bf1b1" (UID: "2f181207-f4c4-4db5-b1c5-15bcb21bf1b1"). InnerVolumeSpecName "kube-api-access-67wbn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:03:53.812915 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:53.812895 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f181207-f4c4-4db5-b1c5-15bcb21bf1b1-util" (OuterVolumeSpecName: "util") pod "2f181207-f4c4-4db5-b1c5-15bcb21bf1b1" (UID: "2f181207-f4c4-4db5-b1c5-15bcb21bf1b1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:03:53.910163 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:53.910126 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-67wbn\" (UniqueName: \"kubernetes.io/projected/2f181207-f4c4-4db5-b1c5-15bcb21bf1b1-kube-api-access-67wbn\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:03:53.910163 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:53.910156 2587 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f181207-f4c4-4db5-b1c5-15bcb21bf1b1-util\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:03:53.910163 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:53.910167 2587 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f181207-f4c4-4db5-b1c5-15bcb21bf1b1-bundle\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:03:54.499907 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:54.499870 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v46jx" event={"ID":"2f181207-f4c4-4db5-b1c5-15bcb21bf1b1","Type":"ContainerDied","Data":"45d26ef15000057a017e720a12a0ef1b680ae68799b10ece20d7f7d058c18caa"} Jul 15 12:03:54.500108 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:54.499922 2587 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="45d26ef15000057a017e720a12a0ef1b680ae68799b10ece20d7f7d058c18caa" Jul 15 12:03:54.500108 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:54.499879 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v46jx" Jul 15 12:03:59.639017 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:59.638979 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-nw4pp"] Jul 15 12:03:59.639500 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:59.639405 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f181207-f4c4-4db5-b1c5-15bcb21bf1b1" containerName="util" Jul 15 12:03:59.639500 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:59.639424 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f181207-f4c4-4db5-b1c5-15bcb21bf1b1" containerName="util" Jul 15 12:03:59.639500 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:59.639441 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f181207-f4c4-4db5-b1c5-15bcb21bf1b1" containerName="extract" Jul 15 12:03:59.639500 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:59.639449 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f181207-f4c4-4db5-b1c5-15bcb21bf1b1" containerName="extract" Jul 15 12:03:59.639500 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:59.639466 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f181207-f4c4-4db5-b1c5-15bcb21bf1b1" containerName="pull" Jul 15 12:03:59.639500 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:59.639473 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f181207-f4c4-4db5-b1c5-15bcb21bf1b1" containerName="pull" Jul 15 12:03:59.639813 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:59.639549 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f181207-f4c4-4db5-b1c5-15bcb21bf1b1" containerName="extract" Jul 15 12:03:59.641536 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:59.641515 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-nw4pp" Jul 15 12:03:59.644311 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:59.644291 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 15 12:03:59.644547 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:59.644532 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 15 12:03:59.644636 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:59.644550 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-9nk78\"" Jul 15 12:03:59.655801 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:59.655781 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-nw4pp"] Jul 15 12:03:59.756179 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:59.756148 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/993d3718-79e7-4d82-bb2a-c2c9fdeca007-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-nw4pp\" (UID: \"993d3718-79e7-4d82-bb2a-c2c9fdeca007\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-nw4pp" Jul 15 12:03:59.756179 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:59.756181 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljcqg\" (UniqueName: \"kubernetes.io/projected/993d3718-79e7-4d82-bb2a-c2c9fdeca007-kube-api-access-ljcqg\") pod \"cert-manager-operator-controller-manager-55c444bf86-nw4pp\" (UID: \"993d3718-79e7-4d82-bb2a-c2c9fdeca007\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-nw4pp" Jul 15 12:03:59.857504 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:59.857454 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/993d3718-79e7-4d82-bb2a-c2c9fdeca007-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-nw4pp\" (UID: \"993d3718-79e7-4d82-bb2a-c2c9fdeca007\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-nw4pp" Jul 15 12:03:59.857641 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:59.857514 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljcqg\" (UniqueName: \"kubernetes.io/projected/993d3718-79e7-4d82-bb2a-c2c9fdeca007-kube-api-access-ljcqg\") pod \"cert-manager-operator-controller-manager-55c444bf86-nw4pp\" (UID: \"993d3718-79e7-4d82-bb2a-c2c9fdeca007\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-nw4pp" Jul 15 12:03:59.857958 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:59.857940 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/993d3718-79e7-4d82-bb2a-c2c9fdeca007-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-nw4pp\" (UID: \"993d3718-79e7-4d82-bb2a-c2c9fdeca007\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-nw4pp" Jul 15 12:03:59.867032 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:59.867000 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljcqg\" (UniqueName: \"kubernetes.io/projected/993d3718-79e7-4d82-bb2a-c2c9fdeca007-kube-api-access-ljcqg\") pod \"cert-manager-operator-controller-manager-55c444bf86-nw4pp\" (UID: \"993d3718-79e7-4d82-bb2a-c2c9fdeca007\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-nw4pp" Jul 15 12:03:59.950676 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:03:59.950595 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-nw4pp" Jul 15 12:04:00.080440 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:00.080417 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-nw4pp"] Jul 15 12:04:00.082920 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:04:00.082893 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod993d3718_79e7_4d82_bb2a_c2c9fdeca007.slice/crio-c1504e814602c7fa7a95a755f292e0d0e5e25e6ab10c154518790f00c21dc7d0 WatchSource:0}: Error finding container c1504e814602c7fa7a95a755f292e0d0e5e25e6ab10c154518790f00c21dc7d0: Status 404 returned error can't find the container with id c1504e814602c7fa7a95a755f292e0d0e5e25e6ab10c154518790f00c21dc7d0 Jul 15 12:04:00.085679 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:00.085661 2587 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 12:04:00.523973 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:00.523940 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-nw4pp" event={"ID":"993d3718-79e7-4d82-bb2a-c2c9fdeca007","Type":"ContainerStarted","Data":"c1504e814602c7fa7a95a755f292e0d0e5e25e6ab10c154518790f00c21dc7d0"} Jul 15 12:04:05.548714 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:05.548679 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-nw4pp" event={"ID":"993d3718-79e7-4d82-bb2a-c2c9fdeca007","Type":"ContainerStarted","Data":"10c8a8293bdfbc8c2974dfbf5e2c5333f3f15ef26bc72710dc67aa28214b9871"} Jul 15 12:04:05.571206 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:05.571154 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-nw4pp" podStartSLOduration=1.693115707 podStartE2EDuration="6.571138777s" totalImagesPullingTime="4.87802307s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:03:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:04:05.569007569 +0000 UTC m=+317.026384494" watchObservedRunningTime="2026-07-15 12:04:05.571138777 +0000 UTC m=+317.028515702" Jul 15 12:04:06.696229 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:06.696199 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-z2zkw"] Jul 15 12:04:06.699032 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:06.699016 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-z2zkw" Jul 15 12:04:06.702148 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:06.702127 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 15 12:04:06.702253 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:06.702182 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 15 12:04:06.702253 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:06.702192 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-qbr76\"" Jul 15 12:04:06.708444 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:06.708422 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-z2zkw"] Jul 15 12:04:06.816377 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:06.816348 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d4c8d56f-beaf-49d7-ac14-ccf1d03a636a-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-z2zkw\" (UID: \"d4c8d56f-beaf-49d7-ac14-ccf1d03a636a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-z2zkw" Jul 15 12:04:06.816556 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:06.816382 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rlwr\" (UniqueName: \"kubernetes.io/projected/d4c8d56f-beaf-49d7-ac14-ccf1d03a636a-kube-api-access-7rlwr\") pod \"cert-manager-webhook-6486ffd997-z2zkw\" (UID: \"d4c8d56f-beaf-49d7-ac14-ccf1d03a636a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-z2zkw" Jul 15 12:04:06.917509 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:06.917457 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d4c8d56f-beaf-49d7-ac14-ccf1d03a636a-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-z2zkw\" (UID: \"d4c8d56f-beaf-49d7-ac14-ccf1d03a636a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-z2zkw" Jul 15 12:04:06.917509 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:06.917513 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rlwr\" (UniqueName: \"kubernetes.io/projected/d4c8d56f-beaf-49d7-ac14-ccf1d03a636a-kube-api-access-7rlwr\") pod \"cert-manager-webhook-6486ffd997-z2zkw\" (UID: \"d4c8d56f-beaf-49d7-ac14-ccf1d03a636a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-z2zkw" Jul 15 12:04:06.925654 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:06.925628 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d4c8d56f-beaf-49d7-ac14-ccf1d03a636a-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-z2zkw\" (UID: \"d4c8d56f-beaf-49d7-ac14-ccf1d03a636a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-z2zkw" Jul 15 12:04:06.925827 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:06.925808 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rlwr\" (UniqueName: \"kubernetes.io/projected/d4c8d56f-beaf-49d7-ac14-ccf1d03a636a-kube-api-access-7rlwr\") pod \"cert-manager-webhook-6486ffd997-z2zkw\" (UID: \"d4c8d56f-beaf-49d7-ac14-ccf1d03a636a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-z2zkw" Jul 15 12:04:07.019064 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:07.018984 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-z2zkw" Jul 15 12:04:07.150172 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:07.150143 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-z2zkw"] Jul 15 12:04:07.153409 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:04:07.153378 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd4c8d56f_beaf_49d7_ac14_ccf1d03a636a.slice/crio-c42e98c5a1176048d1de78f431d77588959118495aa5c5ef1c73ccde071ed635 WatchSource:0}: Error finding container c42e98c5a1176048d1de78f431d77588959118495aa5c5ef1c73ccde071ed635: Status 404 returned error can't find the container with id c42e98c5a1176048d1de78f431d77588959118495aa5c5ef1c73ccde071ed635 Jul 15 12:04:07.558292 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:07.558252 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-z2zkw" event={"ID":"d4c8d56f-beaf-49d7-ac14-ccf1d03a636a","Type":"ContainerStarted","Data":"c42e98c5a1176048d1de78f431d77588959118495aa5c5ef1c73ccde071ed635"} Jul 15 12:04:07.673223 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:07.673191 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvq72b"] Jul 15 12:04:07.677245 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:07.677229 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvq72b" Jul 15 12:04:07.679888 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:07.679854 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 15 12:04:07.680027 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:07.679922 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 15 12:04:07.680027 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:07.679967 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-qn677\"" Jul 15 12:04:07.685688 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:07.685664 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvq72b"] Jul 15 12:04:07.824422 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:07.824341 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/223cfb0e-0c88-4934-afff-5daf8b6e83fb-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvq72b\" (UID: \"223cfb0e-0c88-4934-afff-5daf8b6e83fb\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvq72b" Jul 15 12:04:07.824422 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:07.824406 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pfhb\" (UniqueName: \"kubernetes.io/projected/223cfb0e-0c88-4934-afff-5daf8b6e83fb-kube-api-access-2pfhb\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvq72b\" (UID: \"223cfb0e-0c88-4934-afff-5daf8b6e83fb\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvq72b" Jul 15 12:04:07.824823 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:07.824436 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/223cfb0e-0c88-4934-afff-5daf8b6e83fb-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvq72b\" (UID: \"223cfb0e-0c88-4934-afff-5daf8b6e83fb\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvq72b" Jul 15 12:04:07.925517 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:07.925476 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pfhb\" (UniqueName: \"kubernetes.io/projected/223cfb0e-0c88-4934-afff-5daf8b6e83fb-kube-api-access-2pfhb\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvq72b\" (UID: \"223cfb0e-0c88-4934-afff-5daf8b6e83fb\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvq72b" Jul 15 12:04:07.925690 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:07.925533 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/223cfb0e-0c88-4934-afff-5daf8b6e83fb-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvq72b\" (UID: \"223cfb0e-0c88-4934-afff-5daf8b6e83fb\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvq72b" Jul 15 12:04:07.925690 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:07.925591 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/223cfb0e-0c88-4934-afff-5daf8b6e83fb-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvq72b\" (UID: \"223cfb0e-0c88-4934-afff-5daf8b6e83fb\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvq72b" Jul 15 12:04:07.926007 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:07.925982 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/223cfb0e-0c88-4934-afff-5daf8b6e83fb-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvq72b\" (UID: \"223cfb0e-0c88-4934-afff-5daf8b6e83fb\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvq72b" Jul 15 12:04:07.926105 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:07.926077 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/223cfb0e-0c88-4934-afff-5daf8b6e83fb-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvq72b\" (UID: \"223cfb0e-0c88-4934-afff-5daf8b6e83fb\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvq72b" Jul 15 12:04:07.935287 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:07.935259 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pfhb\" (UniqueName: \"kubernetes.io/projected/223cfb0e-0c88-4934-afff-5daf8b6e83fb-kube-api-access-2pfhb\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvq72b\" (UID: \"223cfb0e-0c88-4934-afff-5daf8b6e83fb\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvq72b" Jul 15 12:04:07.966559 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:07.966528 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-gh8w9"] Jul 15 12:04:07.969890 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:07.969868 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-gh8w9" Jul 15 12:04:07.972688 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:07.972665 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-xf4dh\"" Jul 15 12:04:07.982305 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:07.982281 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-gh8w9"] Jul 15 12:04:07.987887 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:07.987861 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvq72b" Jul 15 12:04:08.126969 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:08.126926 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d8896f14-80ad-478c-8b32-f2dc4c7e41ff-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-gh8w9\" (UID: \"d8896f14-80ad-478c-8b32-f2dc4c7e41ff\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-gh8w9" Jul 15 12:04:08.126969 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:08.126967 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svbmh\" (UniqueName: \"kubernetes.io/projected/d8896f14-80ad-478c-8b32-f2dc4c7e41ff-kube-api-access-svbmh\") pod \"cert-manager-cainjector-559cdd8c87-gh8w9\" (UID: \"d8896f14-80ad-478c-8b32-f2dc4c7e41ff\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-gh8w9" Jul 15 12:04:08.151229 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:08.151204 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvq72b"] Jul 15 12:04:08.154526 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:04:08.154493 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod223cfb0e_0c88_4934_afff_5daf8b6e83fb.slice/crio-cbbbdc2085664bb2925d0ca82b6c505cb83216051d209e6096c6847125eb2bee WatchSource:0}: Error finding container cbbbdc2085664bb2925d0ca82b6c505cb83216051d209e6096c6847125eb2bee: Status 404 returned error can't find the container with id cbbbdc2085664bb2925d0ca82b6c505cb83216051d209e6096c6847125eb2bee Jul 15 12:04:08.228365 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:08.228338 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d8896f14-80ad-478c-8b32-f2dc4c7e41ff-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-gh8w9\" (UID: \"d8896f14-80ad-478c-8b32-f2dc4c7e41ff\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-gh8w9" Jul 15 12:04:08.228491 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:08.228380 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svbmh\" (UniqueName: \"kubernetes.io/projected/d8896f14-80ad-478c-8b32-f2dc4c7e41ff-kube-api-access-svbmh\") pod \"cert-manager-cainjector-559cdd8c87-gh8w9\" (UID: \"d8896f14-80ad-478c-8b32-f2dc4c7e41ff\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-gh8w9" Jul 15 12:04:08.236750 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:08.236703 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d8896f14-80ad-478c-8b32-f2dc4c7e41ff-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-gh8w9\" (UID: \"d8896f14-80ad-478c-8b32-f2dc4c7e41ff\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-gh8w9" Jul 15 12:04:08.237379 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:08.237358 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-svbmh\" (UniqueName: \"kubernetes.io/projected/d8896f14-80ad-478c-8b32-f2dc4c7e41ff-kube-api-access-svbmh\") pod \"cert-manager-cainjector-559cdd8c87-gh8w9\" (UID: \"d8896f14-80ad-478c-8b32-f2dc4c7e41ff\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-gh8w9" Jul 15 12:04:08.281425 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:08.281396 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-gh8w9" Jul 15 12:04:08.429148 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:08.429006 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-gh8w9"] Jul 15 12:04:08.431414 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:04:08.431384 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd8896f14_80ad_478c_8b32_f2dc4c7e41ff.slice/crio-7fa66db2ff919c8990cecc9a9fcaba415933dcad0cc468bb64b3dd33a06c2848 WatchSource:0}: Error finding container 7fa66db2ff919c8990cecc9a9fcaba415933dcad0cc468bb64b3dd33a06c2848: Status 404 returned error can't find the container with id 7fa66db2ff919c8990cecc9a9fcaba415933dcad0cc468bb64b3dd33a06c2848 Jul 15 12:04:08.563771 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:08.563717 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-gh8w9" event={"ID":"d8896f14-80ad-478c-8b32-f2dc4c7e41ff","Type":"ContainerStarted","Data":"7fa66db2ff919c8990cecc9a9fcaba415933dcad0cc468bb64b3dd33a06c2848"} Jul 15 12:04:08.565280 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:08.565251 2587 generic.go:356] "Generic (PLEG): container finished" podID="223cfb0e-0c88-4934-afff-5daf8b6e83fb" containerID="005d5e584fba877d284313a27dde8dcf6a214543547631386e1dc47bab070260" exitCode=0 Jul 15 12:04:08.565414 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:08.565357 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvq72b" event={"ID":"223cfb0e-0c88-4934-afff-5daf8b6e83fb","Type":"ContainerDied","Data":"005d5e584fba877d284313a27dde8dcf6a214543547631386e1dc47bab070260"} Jul 15 12:04:08.565414 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:08.565411 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvq72b" event={"ID":"223cfb0e-0c88-4934-afff-5daf8b6e83fb","Type":"ContainerStarted","Data":"cbbbdc2085664bb2925d0ca82b6c505cb83216051d209e6096c6847125eb2bee"} Jul 15 12:04:10.577201 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:10.577164 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-gh8w9" event={"ID":"d8896f14-80ad-478c-8b32-f2dc4c7e41ff","Type":"ContainerStarted","Data":"f02d984d8210587ad30e90049ef1c2baae99a2912f54fa46044b607888d359db"} Jul 15 12:04:10.578648 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:10.578625 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-z2zkw" event={"ID":"d4c8d56f-beaf-49d7-ac14-ccf1d03a636a","Type":"ContainerStarted","Data":"e13f38a9df5a168dea1b532c0c61d9c6146e3e3d469ea9690acd85e92930333c"} Jul 15 12:04:10.578774 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:10.578757 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-z2zkw" Jul 15 12:04:10.597788 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:10.597721 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-gh8w9" podStartSLOduration=1.919664934 podStartE2EDuration="3.597707786s" totalImagesPullingTime="1.678042852s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:04:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:04:10.596400194 +0000 UTC m=+322.053777116" watchObservedRunningTime="2026-07-15 12:04:10.597707786 +0000 UTC m=+322.055084711" Jul 15 12:04:10.620929 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:10.620880 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-z2zkw" podStartSLOduration=1.671082988 podStartE2EDuration="4.620864947s" totalImagesPullingTime="2.949781959s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:04:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:04:10.619543804 +0000 UTC m=+322.076920774" watchObservedRunningTime="2026-07-15 12:04:10.620864947 +0000 UTC m=+322.078241872" Jul 15 12:04:16.155665 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:16.155626 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-jszww"] Jul 15 12:04:16.160205 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:16.160189 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-jszww" Jul 15 12:04:16.163054 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:16.163017 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-kh86d\"" Jul 15 12:04:16.170676 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:16.170654 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-jszww"] Jul 15 12:04:16.188384 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:16.188360 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbvrc\" (UniqueName: \"kubernetes.io/projected/bbbc25ef-f95e-475d-874e-f256f911fb16-kube-api-access-jbvrc\") pod \"cert-manager-55b4d954d5-jszww\" (UID: \"bbbc25ef-f95e-475d-874e-f256f911fb16\") " pod="cert-manager/cert-manager-55b4d954d5-jszww" Jul 15 12:04:16.188506 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:16.188449 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bbbc25ef-f95e-475d-874e-f256f911fb16-bound-sa-token\") pod \"cert-manager-55b4d954d5-jszww\" (UID: \"bbbc25ef-f95e-475d-874e-f256f911fb16\") " pod="cert-manager/cert-manager-55b4d954d5-jszww" Jul 15 12:04:16.289438 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:16.289402 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbvrc\" (UniqueName: \"kubernetes.io/projected/bbbc25ef-f95e-475d-874e-f256f911fb16-kube-api-access-jbvrc\") pod \"cert-manager-55b4d954d5-jszww\" (UID: \"bbbc25ef-f95e-475d-874e-f256f911fb16\") " pod="cert-manager/cert-manager-55b4d954d5-jszww" Jul 15 12:04:16.289602 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:16.289450 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bbbc25ef-f95e-475d-874e-f256f911fb16-bound-sa-token\") pod \"cert-manager-55b4d954d5-jszww\" (UID: \"bbbc25ef-f95e-475d-874e-f256f911fb16\") " pod="cert-manager/cert-manager-55b4d954d5-jszww" Jul 15 12:04:16.297936 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:16.297904 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bbbc25ef-f95e-475d-874e-f256f911fb16-bound-sa-token\") pod \"cert-manager-55b4d954d5-jszww\" (UID: \"bbbc25ef-f95e-475d-874e-f256f911fb16\") " pod="cert-manager/cert-manager-55b4d954d5-jszww" Jul 15 12:04:16.298051 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:16.298020 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbvrc\" (UniqueName: \"kubernetes.io/projected/bbbc25ef-f95e-475d-874e-f256f911fb16-kube-api-access-jbvrc\") pod \"cert-manager-55b4d954d5-jszww\" (UID: \"bbbc25ef-f95e-475d-874e-f256f911fb16\") " pod="cert-manager/cert-manager-55b4d954d5-jszww" Jul 15 12:04:16.469151 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:16.469056 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-jszww" Jul 15 12:04:16.585083 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:16.585057 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-z2zkw" Jul 15 12:04:16.608327 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:16.608290 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-jszww"] Jul 15 12:04:16.612878 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:04:16.612847 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbbbc25ef_f95e_475d_874e_f256f911fb16.slice/crio-689bd8b2837280541f4c54e0fb23c25880ec16ef754ede736d84c434b8ca00ed WatchSource:0}: Error finding container 689bd8b2837280541f4c54e0fb23c25880ec16ef754ede736d84c434b8ca00ed: Status 404 returned error can't find the container with id 689bd8b2837280541f4c54e0fb23c25880ec16ef754ede736d84c434b8ca00ed Jul 15 12:04:17.618425 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:17.618387 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-jszww" event={"ID":"bbbc25ef-f95e-475d-874e-f256f911fb16","Type":"ContainerStarted","Data":"938abf2d06def28103a366365b3afd5ea4905ee10604335674348d61b5ac4c8d"} Jul 15 12:04:17.618812 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:17.618435 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-jszww" event={"ID":"bbbc25ef-f95e-475d-874e-f256f911fb16","Type":"ContainerStarted","Data":"689bd8b2837280541f4c54e0fb23c25880ec16ef754ede736d84c434b8ca00ed"} Jul 15 12:04:17.638565 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:17.638512 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-jszww" podStartSLOduration=1.638498304 podStartE2EDuration="1.638498304s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:04:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:04:17.636137246 +0000 UTC m=+329.093514189" watchObservedRunningTime="2026-07-15 12:04:17.638498304 +0000 UTC m=+329.095875229" Jul 15 12:04:28.667744 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:28.667699 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvq72b" event={"ID":"223cfb0e-0c88-4934-afff-5daf8b6e83fb","Type":"ContainerStarted","Data":"237cfbd8c75b6460bfc8172033641095fd7164116abc29fca8e96f951df432f6"} Jul 15 12:04:29.673581 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:29.673543 2587 generic.go:356] "Generic (PLEG): container finished" podID="223cfb0e-0c88-4934-afff-5daf8b6e83fb" containerID="237cfbd8c75b6460bfc8172033641095fd7164116abc29fca8e96f951df432f6" exitCode=0 Jul 15 12:04:29.674089 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:29.673620 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvq72b" event={"ID":"223cfb0e-0c88-4934-afff-5daf8b6e83fb","Type":"ContainerDied","Data":"237cfbd8c75b6460bfc8172033641095fd7164116abc29fca8e96f951df432f6"} Jul 15 12:04:29.823032 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:04:29.822999 2587 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod223cfb0e_0c88_4934_afff_5daf8b6e83fb.slice/crio-conmon-fb18eb8bbfce54179c55564f851f99c6fc407dc0ca1aa06e70445343b283f7d9.scope\": RecentStats: unable to find data in memory cache]" Jul 15 12:04:30.679748 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:30.679698 2587 generic.go:356] "Generic (PLEG): container finished" podID="223cfb0e-0c88-4934-afff-5daf8b6e83fb" containerID="fb18eb8bbfce54179c55564f851f99c6fc407dc0ca1aa06e70445343b283f7d9" exitCode=0 Jul 15 12:04:30.680146 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:30.679767 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvq72b" event={"ID":"223cfb0e-0c88-4934-afff-5daf8b6e83fb","Type":"ContainerDied","Data":"fb18eb8bbfce54179c55564f851f99c6fc407dc0ca1aa06e70445343b283f7d9"} Jul 15 12:04:31.815795 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:31.815772 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvq72b" Jul 15 12:04:31.905493 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:31.905461 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2pfhb\" (UniqueName: \"kubernetes.io/projected/223cfb0e-0c88-4934-afff-5daf8b6e83fb-kube-api-access-2pfhb\") pod \"223cfb0e-0c88-4934-afff-5daf8b6e83fb\" (UID: \"223cfb0e-0c88-4934-afff-5daf8b6e83fb\") " Jul 15 12:04:31.905652 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:31.905513 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/223cfb0e-0c88-4934-afff-5daf8b6e83fb-bundle\") pod \"223cfb0e-0c88-4934-afff-5daf8b6e83fb\" (UID: \"223cfb0e-0c88-4934-afff-5daf8b6e83fb\") " Jul 15 12:04:31.905652 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:31.905583 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/223cfb0e-0c88-4934-afff-5daf8b6e83fb-util\") pod \"223cfb0e-0c88-4934-afff-5daf8b6e83fb\" (UID: \"223cfb0e-0c88-4934-afff-5daf8b6e83fb\") " Jul 15 12:04:31.905961 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:31.905934 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/223cfb0e-0c88-4934-afff-5daf8b6e83fb-bundle" (OuterVolumeSpecName: "bundle") pod "223cfb0e-0c88-4934-afff-5daf8b6e83fb" (UID: "223cfb0e-0c88-4934-afff-5daf8b6e83fb"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:04:31.907546 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:31.907518 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/223cfb0e-0c88-4934-afff-5daf8b6e83fb-kube-api-access-2pfhb" (OuterVolumeSpecName: "kube-api-access-2pfhb") pod "223cfb0e-0c88-4934-afff-5daf8b6e83fb" (UID: "223cfb0e-0c88-4934-afff-5daf8b6e83fb"). InnerVolumeSpecName "kube-api-access-2pfhb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:04:31.909831 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:31.909812 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/223cfb0e-0c88-4934-afff-5daf8b6e83fb-util" (OuterVolumeSpecName: "util") pod "223cfb0e-0c88-4934-afff-5daf8b6e83fb" (UID: "223cfb0e-0c88-4934-afff-5daf8b6e83fb"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:04:32.006718 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:32.006647 2587 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/223cfb0e-0c88-4934-afff-5daf8b6e83fb-bundle\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:04:32.006718 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:32.006677 2587 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/223cfb0e-0c88-4934-afff-5daf8b6e83fb-util\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:04:32.006718 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:32.006690 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2pfhb\" (UniqueName: \"kubernetes.io/projected/223cfb0e-0c88-4934-afff-5daf8b6e83fb-kube-api-access-2pfhb\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:04:32.690968 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:32.690940 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvq72b" Jul 15 12:04:32.691129 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:32.690939 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvq72b" event={"ID":"223cfb0e-0c88-4934-afff-5daf8b6e83fb","Type":"ContainerDied","Data":"cbbbdc2085664bb2925d0ca82b6c505cb83216051d209e6096c6847125eb2bee"} Jul 15 12:04:32.691129 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:32.691054 2587 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cbbbdc2085664bb2925d0ca82b6c505cb83216051d209e6096c6847125eb2bee" Jul 15 12:04:36.609601 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:36.609562 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kwdh4"] Jul 15 12:04:36.610063 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:36.609915 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="223cfb0e-0c88-4934-afff-5daf8b6e83fb" containerName="util" Jul 15 12:04:36.610063 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:36.609928 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="223cfb0e-0c88-4934-afff-5daf8b6e83fb" containerName="util" Jul 15 12:04:36.610063 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:36.609936 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="223cfb0e-0c88-4934-afff-5daf8b6e83fb" containerName="extract" Jul 15 12:04:36.610063 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:36.609942 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="223cfb0e-0c88-4934-afff-5daf8b6e83fb" containerName="extract" Jul 15 12:04:36.610063 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:36.609956 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="223cfb0e-0c88-4934-afff-5daf8b6e83fb" containerName="pull" Jul 15 12:04:36.610063 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:36.609962 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="223cfb0e-0c88-4934-afff-5daf8b6e83fb" containerName="pull" Jul 15 12:04:36.610063 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:36.610030 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="223cfb0e-0c88-4934-afff-5daf8b6e83fb" containerName="extract" Jul 15 12:04:36.613076 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:36.613058 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kwdh4" Jul 15 12:04:36.616526 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:36.616507 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 15 12:04:36.617575 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:36.617556 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 15 12:04:36.617661 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:36.617639 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-f9rt2\"" Jul 15 12:04:36.632489 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:36.632462 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kwdh4"] Jul 15 12:04:36.643913 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:36.643884 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xq9ht\" (UniqueName: \"kubernetes.io/projected/3ddb7416-fddb-4328-9b6d-d076a13f940a-kube-api-access-xq9ht\") pod \"openshift-lws-operator-fd8ccff4c-kwdh4\" (UID: \"3ddb7416-fddb-4328-9b6d-d076a13f940a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kwdh4" Jul 15 12:04:36.644023 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:36.643927 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3ddb7416-fddb-4328-9b6d-d076a13f940a-tmp\") pod \"openshift-lws-operator-fd8ccff4c-kwdh4\" (UID: \"3ddb7416-fddb-4328-9b6d-d076a13f940a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kwdh4" Jul 15 12:04:36.745041 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:36.745009 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xq9ht\" (UniqueName: \"kubernetes.io/projected/3ddb7416-fddb-4328-9b6d-d076a13f940a-kube-api-access-xq9ht\") pod \"openshift-lws-operator-fd8ccff4c-kwdh4\" (UID: \"3ddb7416-fddb-4328-9b6d-d076a13f940a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kwdh4" Jul 15 12:04:36.745041 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:36.745047 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3ddb7416-fddb-4328-9b6d-d076a13f940a-tmp\") pod \"openshift-lws-operator-fd8ccff4c-kwdh4\" (UID: \"3ddb7416-fddb-4328-9b6d-d076a13f940a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kwdh4" Jul 15 12:04:36.745413 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:36.745397 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3ddb7416-fddb-4328-9b6d-d076a13f940a-tmp\") pod \"openshift-lws-operator-fd8ccff4c-kwdh4\" (UID: \"3ddb7416-fddb-4328-9b6d-d076a13f940a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kwdh4" Jul 15 12:04:36.757499 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:36.757469 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xq9ht\" (UniqueName: \"kubernetes.io/projected/3ddb7416-fddb-4328-9b6d-d076a13f940a-kube-api-access-xq9ht\") pod \"openshift-lws-operator-fd8ccff4c-kwdh4\" (UID: \"3ddb7416-fddb-4328-9b6d-d076a13f940a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kwdh4" Jul 15 12:04:36.922128 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:36.922036 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kwdh4" Jul 15 12:04:37.076570 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:37.076548 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kwdh4"] Jul 15 12:04:37.078999 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:04:37.078963 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3ddb7416_fddb_4328_9b6d_d076a13f940a.slice/crio-d7938091bb4ddefb5ec78dbff2beb847e62f429a467bb3b377e57fa902cbb84f WatchSource:0}: Error finding container d7938091bb4ddefb5ec78dbff2beb847e62f429a467bb3b377e57fa902cbb84f: Status 404 returned error can't find the container with id d7938091bb4ddefb5ec78dbff2beb847e62f429a467bb3b377e57fa902cbb84f Jul 15 12:04:37.713637 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:37.713598 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kwdh4" event={"ID":"3ddb7416-fddb-4328-9b6d-d076a13f940a","Type":"ContainerStarted","Data":"d7938091bb4ddefb5ec78dbff2beb847e62f429a467bb3b377e57fa902cbb84f"} Jul 15 12:04:39.724843 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:39.724807 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kwdh4" event={"ID":"3ddb7416-fddb-4328-9b6d-d076a13f940a","Type":"ContainerStarted","Data":"d837ca79187f0ba991d0650f3a272d44e7006502316a67ceb37f049479486a29"} Jul 15 12:04:39.744683 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:39.744636 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kwdh4" podStartSLOduration=2.126369749 podStartE2EDuration="3.744622989s" totalImagesPullingTime="1.61825324s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:04:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:04:39.743333262 +0000 UTC m=+351.200710187" watchObservedRunningTime="2026-07-15 12:04:39.744622989 +0000 UTC m=+351.201999914" Jul 15 12:04:42.214982 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:42.214900 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqh5t2"] Jul 15 12:04:42.218245 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:42.218228 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqh5t2" Jul 15 12:04:42.220901 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:42.220878 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 15 12:04:42.222171 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:42.222143 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 15 12:04:42.222283 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:42.222214 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-qn677\"" Jul 15 12:04:42.227151 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:42.227129 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqh5t2"] Jul 15 12:04:42.298807 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:42.298770 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7c5c\" (UniqueName: \"kubernetes.io/projected/73c0d744-7017-49cc-b995-026cb3d438e3-kube-api-access-p7c5c\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqh5t2\" (UID: \"73c0d744-7017-49cc-b995-026cb3d438e3\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqh5t2" Jul 15 12:04:42.298973 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:42.298899 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/73c0d744-7017-49cc-b995-026cb3d438e3-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqh5t2\" (UID: \"73c0d744-7017-49cc-b995-026cb3d438e3\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqh5t2" Jul 15 12:04:42.298973 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:42.298930 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/73c0d744-7017-49cc-b995-026cb3d438e3-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqh5t2\" (UID: \"73c0d744-7017-49cc-b995-026cb3d438e3\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqh5t2" Jul 15 12:04:42.400144 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:42.400107 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/73c0d744-7017-49cc-b995-026cb3d438e3-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqh5t2\" (UID: \"73c0d744-7017-49cc-b995-026cb3d438e3\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqh5t2" Jul 15 12:04:42.400144 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:42.400145 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/73c0d744-7017-49cc-b995-026cb3d438e3-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqh5t2\" (UID: \"73c0d744-7017-49cc-b995-026cb3d438e3\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqh5t2" Jul 15 12:04:42.400390 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:42.400280 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7c5c\" (UniqueName: \"kubernetes.io/projected/73c0d744-7017-49cc-b995-026cb3d438e3-kube-api-access-p7c5c\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqh5t2\" (UID: \"73c0d744-7017-49cc-b995-026cb3d438e3\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqh5t2" Jul 15 12:04:42.400569 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:42.400548 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/73c0d744-7017-49cc-b995-026cb3d438e3-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqh5t2\" (UID: \"73c0d744-7017-49cc-b995-026cb3d438e3\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqh5t2" Jul 15 12:04:42.400638 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:42.400571 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/73c0d744-7017-49cc-b995-026cb3d438e3-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqh5t2\" (UID: \"73c0d744-7017-49cc-b995-026cb3d438e3\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqh5t2" Jul 15 12:04:42.409311 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:42.409287 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7c5c\" (UniqueName: \"kubernetes.io/projected/73c0d744-7017-49cc-b995-026cb3d438e3-kube-api-access-p7c5c\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqh5t2\" (UID: \"73c0d744-7017-49cc-b995-026cb3d438e3\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqh5t2" Jul 15 12:04:42.528308 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:42.528224 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqh5t2" Jul 15 12:04:42.660185 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:42.660160 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqh5t2"] Jul 15 12:04:42.662280 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:04:42.662254 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73c0d744_7017_49cc_b995_026cb3d438e3.slice/crio-11a6ca25948b289cc7ddfed99e27fc51a25c3d7db4483243ccf4628b26860e9e WatchSource:0}: Error finding container 11a6ca25948b289cc7ddfed99e27fc51a25c3d7db4483243ccf4628b26860e9e: Status 404 returned error can't find the container with id 11a6ca25948b289cc7ddfed99e27fc51a25c3d7db4483243ccf4628b26860e9e Jul 15 12:04:42.740413 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:42.740371 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqh5t2" event={"ID":"73c0d744-7017-49cc-b995-026cb3d438e3","Type":"ContainerStarted","Data":"c79f6de2dce3122dce33a4a54ebe12a921ca6907c1bf84cafa9b397545a5a3ae"} Jul 15 12:04:42.740554 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:42.740437 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqh5t2" event={"ID":"73c0d744-7017-49cc-b995-026cb3d438e3","Type":"ContainerStarted","Data":"11a6ca25948b289cc7ddfed99e27fc51a25c3d7db4483243ccf4628b26860e9e"} Jul 15 12:04:43.747194 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:43.747100 2587 generic.go:356] "Generic (PLEG): container finished" podID="73c0d744-7017-49cc-b995-026cb3d438e3" containerID="c79f6de2dce3122dce33a4a54ebe12a921ca6907c1bf84cafa9b397545a5a3ae" exitCode=0 Jul 15 12:04:43.747194 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:43.747141 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqh5t2" event={"ID":"73c0d744-7017-49cc-b995-026cb3d438e3","Type":"ContainerDied","Data":"c79f6de2dce3122dce33a4a54ebe12a921ca6907c1bf84cafa9b397545a5a3ae"} Jul 15 12:04:44.754151 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:44.754114 2587 generic.go:356] "Generic (PLEG): container finished" podID="73c0d744-7017-49cc-b995-026cb3d438e3" containerID="96a538e6f22b453fac190bcfaefc8237a8840b9ae51760b2f20631bd8aa7bf6b" exitCode=0 Jul 15 12:04:44.754507 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:44.754197 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqh5t2" event={"ID":"73c0d744-7017-49cc-b995-026cb3d438e3","Type":"ContainerDied","Data":"96a538e6f22b453fac190bcfaefc8237a8840b9ae51760b2f20631bd8aa7bf6b"} Jul 15 12:04:45.761787 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:45.761726 2587 generic.go:356] "Generic (PLEG): container finished" podID="73c0d744-7017-49cc-b995-026cb3d438e3" containerID="3c6b4ed649d0244520ffe8cad8e3d66106907ccca0a8230899118c5b2fe56c9f" exitCode=0 Jul 15 12:04:45.762157 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:45.761825 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqh5t2" event={"ID":"73c0d744-7017-49cc-b995-026cb3d438e3","Type":"ContainerDied","Data":"3c6b4ed649d0244520ffe8cad8e3d66106907ccca0a8230899118c5b2fe56c9f"} Jul 15 12:04:46.898094 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:46.898067 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqh5t2" Jul 15 12:04:47.044861 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:47.044829 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/73c0d744-7017-49cc-b995-026cb3d438e3-util\") pod \"73c0d744-7017-49cc-b995-026cb3d438e3\" (UID: \"73c0d744-7017-49cc-b995-026cb3d438e3\") " Jul 15 12:04:47.045020 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:47.044899 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/73c0d744-7017-49cc-b995-026cb3d438e3-bundle\") pod \"73c0d744-7017-49cc-b995-026cb3d438e3\" (UID: \"73c0d744-7017-49cc-b995-026cb3d438e3\") " Jul 15 12:04:47.045020 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:47.044951 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p7c5c\" (UniqueName: \"kubernetes.io/projected/73c0d744-7017-49cc-b995-026cb3d438e3-kube-api-access-p7c5c\") pod \"73c0d744-7017-49cc-b995-026cb3d438e3\" (UID: \"73c0d744-7017-49cc-b995-026cb3d438e3\") " Jul 15 12:04:47.045669 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:47.045638 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73c0d744-7017-49cc-b995-026cb3d438e3-bundle" (OuterVolumeSpecName: "bundle") pod "73c0d744-7017-49cc-b995-026cb3d438e3" (UID: "73c0d744-7017-49cc-b995-026cb3d438e3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:04:47.047071 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:47.047037 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73c0d744-7017-49cc-b995-026cb3d438e3-kube-api-access-p7c5c" (OuterVolumeSpecName: "kube-api-access-p7c5c") pod "73c0d744-7017-49cc-b995-026cb3d438e3" (UID: "73c0d744-7017-49cc-b995-026cb3d438e3"). InnerVolumeSpecName "kube-api-access-p7c5c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:04:47.050489 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:47.050464 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73c0d744-7017-49cc-b995-026cb3d438e3-util" (OuterVolumeSpecName: "util") pod "73c0d744-7017-49cc-b995-026cb3d438e3" (UID: "73c0d744-7017-49cc-b995-026cb3d438e3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:04:47.146459 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:47.146422 2587 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/73c0d744-7017-49cc-b995-026cb3d438e3-util\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:04:47.146459 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:47.146452 2587 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/73c0d744-7017-49cc-b995-026cb3d438e3-bundle\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:04:47.146459 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:47.146465 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7c5c\" (UniqueName: \"kubernetes.io/projected/73c0d744-7017-49cc-b995-026cb3d438e3-kube-api-access-p7c5c\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:04:47.773400 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:47.773366 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqh5t2" event={"ID":"73c0d744-7017-49cc-b995-026cb3d438e3","Type":"ContainerDied","Data":"11a6ca25948b289cc7ddfed99e27fc51a25c3d7db4483243ccf4628b26860e9e"} Jul 15 12:04:47.773574 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:47.773412 2587 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="11a6ca25948b289cc7ddfed99e27fc51a25c3d7db4483243ccf4628b26860e9e" Jul 15 12:04:47.773574 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:47.773379 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqh5t2" Jul 15 12:04:58.235338 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:58.235305 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-694c4fcbc8-mdm8d"] Jul 15 12:04:58.235709 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:58.235664 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73c0d744-7017-49cc-b995-026cb3d438e3" containerName="pull" Jul 15 12:04:58.235709 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:58.235675 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="73c0d744-7017-49cc-b995-026cb3d438e3" containerName="pull" Jul 15 12:04:58.235709 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:58.235696 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73c0d744-7017-49cc-b995-026cb3d438e3" containerName="util" Jul 15 12:04:58.235709 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:58.235704 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="73c0d744-7017-49cc-b995-026cb3d438e3" containerName="util" Jul 15 12:04:58.235867 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:58.235714 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73c0d744-7017-49cc-b995-026cb3d438e3" containerName="extract" Jul 15 12:04:58.235867 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:58.235719 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="73c0d744-7017-49cc-b995-026cb3d438e3" containerName="extract" Jul 15 12:04:58.235867 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:58.235800 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="73c0d744-7017-49cc-b995-026cb3d438e3" containerName="extract" Jul 15 12:04:58.240836 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:58.240815 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-694c4fcbc8-mdm8d" Jul 15 12:04:58.243712 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:58.243692 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 15 12:04:58.244239 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:58.244220 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-vrmsp\"" Jul 15 12:04:58.244494 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:58.244468 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 15 12:04:58.244561 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:58.244514 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 15 12:04:58.244561 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:58.244526 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 15 12:04:58.267054 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:58.267027 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-694c4fcbc8-mdm8d"] Jul 15 12:04:58.348505 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:58.348464 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/04eb66ea-bd75-40be-8683-2072ee07eb79-apiservice-cert\") pod \"opendatahub-operator-controller-manager-694c4fcbc8-mdm8d\" (UID: \"04eb66ea-bd75-40be-8683-2072ee07eb79\") " pod="opendatahub/opendatahub-operator-controller-manager-694c4fcbc8-mdm8d" Jul 15 12:04:58.348673 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:58.348517 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/04eb66ea-bd75-40be-8683-2072ee07eb79-tmp\") pod \"opendatahub-operator-controller-manager-694c4fcbc8-mdm8d\" (UID: \"04eb66ea-bd75-40be-8683-2072ee07eb79\") " pod="opendatahub/opendatahub-operator-controller-manager-694c4fcbc8-mdm8d" Jul 15 12:04:58.348673 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:58.348631 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/04eb66ea-bd75-40be-8683-2072ee07eb79-manifests\") pod \"opendatahub-operator-controller-manager-694c4fcbc8-mdm8d\" (UID: \"04eb66ea-bd75-40be-8683-2072ee07eb79\") " pod="opendatahub/opendatahub-operator-controller-manager-694c4fcbc8-mdm8d" Jul 15 12:04:58.348770 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:58.348678 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/04eb66ea-bd75-40be-8683-2072ee07eb79-webhook-cert\") pod \"opendatahub-operator-controller-manager-694c4fcbc8-mdm8d\" (UID: \"04eb66ea-bd75-40be-8683-2072ee07eb79\") " pod="opendatahub/opendatahub-operator-controller-manager-694c4fcbc8-mdm8d" Jul 15 12:04:58.348770 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:58.348714 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwm69\" (UniqueName: \"kubernetes.io/projected/04eb66ea-bd75-40be-8683-2072ee07eb79-kube-api-access-jwm69\") pod \"opendatahub-operator-controller-manager-694c4fcbc8-mdm8d\" (UID: \"04eb66ea-bd75-40be-8683-2072ee07eb79\") " pod="opendatahub/opendatahub-operator-controller-manager-694c4fcbc8-mdm8d" Jul 15 12:04:58.449285 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:58.449244 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwm69\" (UniqueName: \"kubernetes.io/projected/04eb66ea-bd75-40be-8683-2072ee07eb79-kube-api-access-jwm69\") pod \"opendatahub-operator-controller-manager-694c4fcbc8-mdm8d\" (UID: \"04eb66ea-bd75-40be-8683-2072ee07eb79\") " pod="opendatahub/opendatahub-operator-controller-manager-694c4fcbc8-mdm8d" Jul 15 12:04:58.449285 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:58.449291 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/04eb66ea-bd75-40be-8683-2072ee07eb79-apiservice-cert\") pod \"opendatahub-operator-controller-manager-694c4fcbc8-mdm8d\" (UID: \"04eb66ea-bd75-40be-8683-2072ee07eb79\") " pod="opendatahub/opendatahub-operator-controller-manager-694c4fcbc8-mdm8d" Jul 15 12:04:58.449555 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:58.449323 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/04eb66ea-bd75-40be-8683-2072ee07eb79-tmp\") pod \"opendatahub-operator-controller-manager-694c4fcbc8-mdm8d\" (UID: \"04eb66ea-bd75-40be-8683-2072ee07eb79\") " pod="opendatahub/opendatahub-operator-controller-manager-694c4fcbc8-mdm8d" Jul 15 12:04:58.449555 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:58.449356 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/04eb66ea-bd75-40be-8683-2072ee07eb79-manifests\") pod \"opendatahub-operator-controller-manager-694c4fcbc8-mdm8d\" (UID: \"04eb66ea-bd75-40be-8683-2072ee07eb79\") " pod="opendatahub/opendatahub-operator-controller-manager-694c4fcbc8-mdm8d" Jul 15 12:04:58.449555 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:58.449382 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/04eb66ea-bd75-40be-8683-2072ee07eb79-webhook-cert\") pod \"opendatahub-operator-controller-manager-694c4fcbc8-mdm8d\" (UID: \"04eb66ea-bd75-40be-8683-2072ee07eb79\") " pod="opendatahub/opendatahub-operator-controller-manager-694c4fcbc8-mdm8d" Jul 15 12:04:58.449884 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:58.449861 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/04eb66ea-bd75-40be-8683-2072ee07eb79-manifests\") pod \"opendatahub-operator-controller-manager-694c4fcbc8-mdm8d\" (UID: \"04eb66ea-bd75-40be-8683-2072ee07eb79\") " pod="opendatahub/opendatahub-operator-controller-manager-694c4fcbc8-mdm8d" Jul 15 12:04:58.451714 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:58.451685 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/04eb66ea-bd75-40be-8683-2072ee07eb79-tmp\") pod \"opendatahub-operator-controller-manager-694c4fcbc8-mdm8d\" (UID: \"04eb66ea-bd75-40be-8683-2072ee07eb79\") " pod="opendatahub/opendatahub-operator-controller-manager-694c4fcbc8-mdm8d" Jul 15 12:04:58.451974 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:58.451958 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/04eb66ea-bd75-40be-8683-2072ee07eb79-apiservice-cert\") pod \"opendatahub-operator-controller-manager-694c4fcbc8-mdm8d\" (UID: \"04eb66ea-bd75-40be-8683-2072ee07eb79\") " pod="opendatahub/opendatahub-operator-controller-manager-694c4fcbc8-mdm8d" Jul 15 12:04:58.452056 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:58.452035 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/04eb66ea-bd75-40be-8683-2072ee07eb79-webhook-cert\") pod \"opendatahub-operator-controller-manager-694c4fcbc8-mdm8d\" (UID: \"04eb66ea-bd75-40be-8683-2072ee07eb79\") " pod="opendatahub/opendatahub-operator-controller-manager-694c4fcbc8-mdm8d" Jul 15 12:04:58.457975 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:58.457951 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwm69\" (UniqueName: \"kubernetes.io/projected/04eb66ea-bd75-40be-8683-2072ee07eb79-kube-api-access-jwm69\") pod \"opendatahub-operator-controller-manager-694c4fcbc8-mdm8d\" (UID: \"04eb66ea-bd75-40be-8683-2072ee07eb79\") " pod="opendatahub/opendatahub-operator-controller-manager-694c4fcbc8-mdm8d" Jul 15 12:04:58.551066 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:58.551041 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-694c4fcbc8-mdm8d" Jul 15 12:04:58.697981 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:58.697943 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-694c4fcbc8-mdm8d"] Jul 15 12:04:58.701925 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:04:58.701893 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04eb66ea_bd75_40be_8683_2072ee07eb79.slice/crio-a5c3932749d2a48516991f1c85f7978ad66e63c713434f633f1e5fde4938bfda WatchSource:0}: Error finding container a5c3932749d2a48516991f1c85f7978ad66e63c713434f633f1e5fde4938bfda: Status 404 returned error can't find the container with id a5c3932749d2a48516991f1c85f7978ad66e63c713434f633f1e5fde4938bfda Jul 15 12:04:58.823992 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:04:58.823901 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-694c4fcbc8-mdm8d" event={"ID":"04eb66ea-bd75-40be-8683-2072ee07eb79","Type":"ContainerStarted","Data":"a5c3932749d2a48516991f1c85f7978ad66e63c713434f633f1e5fde4938bfda"} Jul 15 12:05:02.846486 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:02.846449 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-694c4fcbc8-mdm8d" event={"ID":"04eb66ea-bd75-40be-8683-2072ee07eb79","Type":"ContainerStarted","Data":"4a4c873be018163517a5c843bbc29222928aca6eca1c467e9dc5adf28efa8102"} Jul 15 12:05:03.852966 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:03.852928 2587 generic.go:356] "Generic (PLEG): container finished" podID="04eb66ea-bd75-40be-8683-2072ee07eb79" containerID="4a4c873be018163517a5c843bbc29222928aca6eca1c467e9dc5adf28efa8102" exitCode=0 Jul 15 12:05:03.853362 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:03.852997 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-694c4fcbc8-mdm8d" event={"ID":"04eb66ea-bd75-40be-8683-2072ee07eb79","Type":"ContainerDied","Data":"4a4c873be018163517a5c843bbc29222928aca6eca1c467e9dc5adf28efa8102"} Jul 15 12:05:04.859430 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:04.859392 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-694c4fcbc8-mdm8d" event={"ID":"04eb66ea-bd75-40be-8683-2072ee07eb79","Type":"ContainerStarted","Data":"5caec3456c941dee83741ab7573ac0cbf9a88c4a0458ff2d30962f9dc5652fef"} Jul 15 12:05:04.859883 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:04.859488 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-694c4fcbc8-mdm8d" Jul 15 12:05:04.889361 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:04.889310 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-694c4fcbc8-mdm8d" podStartSLOduration=2.288917755 podStartE2EDuration="6.889291319s" totalImagesPullingTime="3.543527268s" totalInitContainerRuntime="1.056846296s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:04:58 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:05:04.888869071 +0000 UTC m=+376.346246017" watchObservedRunningTime="2026-07-15 12:05:04.889291319 +0000 UTC m=+376.346668244" Jul 15 12:05:15.865348 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:15.865321 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-694c4fcbc8-mdm8d" Jul 15 12:05:19.909149 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:19.909105 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7d7f886b65-qd67k"] Jul 15 12:05:19.912853 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:19.912830 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7d7f886b65-qd67k" Jul 15 12:05:19.916845 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:19.916826 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 15 12:05:19.917055 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:19.917037 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 15 12:05:19.917308 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:19.917294 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 15 12:05:19.917371 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:19.917335 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 15 12:05:19.921028 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:19.921010 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-g5tqd\"" Jul 15 12:05:19.922409 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:19.922393 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a347f0d8-4a4b-4f5d-83b8-082a96f4aa7a-tmp\") pod \"kube-auth-proxy-7d7f886b65-qd67k\" (UID: \"a347f0d8-4a4b-4f5d-83b8-082a96f4aa7a\") " pod="openshift-ingress/kube-auth-proxy-7d7f886b65-qd67k" Jul 15 12:05:19.922514 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:19.922419 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a347f0d8-4a4b-4f5d-83b8-082a96f4aa7a-tls-certs\") pod \"kube-auth-proxy-7d7f886b65-qd67k\" (UID: \"a347f0d8-4a4b-4f5d-83b8-082a96f4aa7a\") " pod="openshift-ingress/kube-auth-proxy-7d7f886b65-qd67k" Jul 15 12:05:19.922514 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:19.922455 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkrln\" (UniqueName: \"kubernetes.io/projected/a347f0d8-4a4b-4f5d-83b8-082a96f4aa7a-kube-api-access-fkrln\") pod \"kube-auth-proxy-7d7f886b65-qd67k\" (UID: \"a347f0d8-4a4b-4f5d-83b8-082a96f4aa7a\") " pod="openshift-ingress/kube-auth-proxy-7d7f886b65-qd67k" Jul 15 12:05:19.929350 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:19.929324 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7d7f886b65-qd67k"] Jul 15 12:05:20.023317 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:20.023289 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkrln\" (UniqueName: \"kubernetes.io/projected/a347f0d8-4a4b-4f5d-83b8-082a96f4aa7a-kube-api-access-fkrln\") pod \"kube-auth-proxy-7d7f886b65-qd67k\" (UID: \"a347f0d8-4a4b-4f5d-83b8-082a96f4aa7a\") " pod="openshift-ingress/kube-auth-proxy-7d7f886b65-qd67k" Jul 15 12:05:20.023509 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:20.023376 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a347f0d8-4a4b-4f5d-83b8-082a96f4aa7a-tmp\") pod \"kube-auth-proxy-7d7f886b65-qd67k\" (UID: \"a347f0d8-4a4b-4f5d-83b8-082a96f4aa7a\") " pod="openshift-ingress/kube-auth-proxy-7d7f886b65-qd67k" Jul 15 12:05:20.023509 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:20.023408 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a347f0d8-4a4b-4f5d-83b8-082a96f4aa7a-tls-certs\") pod \"kube-auth-proxy-7d7f886b65-qd67k\" (UID: \"a347f0d8-4a4b-4f5d-83b8-082a96f4aa7a\") " pod="openshift-ingress/kube-auth-proxy-7d7f886b65-qd67k" Jul 15 12:05:20.023639 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:05:20.023512 2587 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 15 12:05:20.023639 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:05:20.023588 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a347f0d8-4a4b-4f5d-83b8-082a96f4aa7a-tls-certs podName:a347f0d8-4a4b-4f5d-83b8-082a96f4aa7a nodeName:}" failed. No retries permitted until 2026-07-15 12:05:20.52356566 +0000 UTC m=+391.980942577 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/a347f0d8-4a4b-4f5d-83b8-082a96f4aa7a-tls-certs") pod "kube-auth-proxy-7d7f886b65-qd67k" (UID: "a347f0d8-4a4b-4f5d-83b8-082a96f4aa7a") : secret "kube-auth-proxy-tls" not found Jul 15 12:05:20.025665 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:20.025643 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a347f0d8-4a4b-4f5d-83b8-082a96f4aa7a-tmp\") pod \"kube-auth-proxy-7d7f886b65-qd67k\" (UID: \"a347f0d8-4a4b-4f5d-83b8-082a96f4aa7a\") " pod="openshift-ingress/kube-auth-proxy-7d7f886b65-qd67k" Jul 15 12:05:20.036457 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:20.036439 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkrln\" (UniqueName: \"kubernetes.io/projected/a347f0d8-4a4b-4f5d-83b8-082a96f4aa7a-kube-api-access-fkrln\") pod \"kube-auth-proxy-7d7f886b65-qd67k\" (UID: \"a347f0d8-4a4b-4f5d-83b8-082a96f4aa7a\") " pod="openshift-ingress/kube-auth-proxy-7d7f886b65-qd67k" Jul 15 12:05:20.141386 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:20.141352 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-p2pr6"] Jul 15 12:05:20.144581 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:20.144565 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-p2pr6" Jul 15 12:05:20.147055 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:20.147036 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 15 12:05:20.147138 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:20.147074 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-4n4vh\"" Jul 15 12:05:20.154332 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:20.154312 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-p2pr6"] Jul 15 12:05:20.225810 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:20.225718 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/14ba6a32-b02c-4afd-9adb-4547d982708f-cert\") pod \"odh-model-controller-7998c67dc4-p2pr6\" (UID: \"14ba6a32-b02c-4afd-9adb-4547d982708f\") " pod="opendatahub/odh-model-controller-7998c67dc4-p2pr6" Jul 15 12:05:20.225951 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:20.225890 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzzgg\" (UniqueName: \"kubernetes.io/projected/14ba6a32-b02c-4afd-9adb-4547d982708f-kube-api-access-wzzgg\") pod \"odh-model-controller-7998c67dc4-p2pr6\" (UID: \"14ba6a32-b02c-4afd-9adb-4547d982708f\") " pod="opendatahub/odh-model-controller-7998c67dc4-p2pr6" Jul 15 12:05:20.327238 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:20.327189 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzzgg\" (UniqueName: \"kubernetes.io/projected/14ba6a32-b02c-4afd-9adb-4547d982708f-kube-api-access-wzzgg\") pod \"odh-model-controller-7998c67dc4-p2pr6\" (UID: \"14ba6a32-b02c-4afd-9adb-4547d982708f\") " pod="opendatahub/odh-model-controller-7998c67dc4-p2pr6" Jul 15 12:05:20.327238 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:20.327241 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/14ba6a32-b02c-4afd-9adb-4547d982708f-cert\") pod \"odh-model-controller-7998c67dc4-p2pr6\" (UID: \"14ba6a32-b02c-4afd-9adb-4547d982708f\") " pod="opendatahub/odh-model-controller-7998c67dc4-p2pr6" Jul 15 12:05:20.327440 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:05:20.327394 2587 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 15 12:05:20.327479 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:05:20.327461 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14ba6a32-b02c-4afd-9adb-4547d982708f-cert podName:14ba6a32-b02c-4afd-9adb-4547d982708f nodeName:}" failed. No retries permitted until 2026-07-15 12:05:20.8274445 +0000 UTC m=+392.284821406 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/14ba6a32-b02c-4afd-9adb-4547d982708f-cert") pod "odh-model-controller-7998c67dc4-p2pr6" (UID: "14ba6a32-b02c-4afd-9adb-4547d982708f") : secret "odh-model-controller-webhook-cert" not found Jul 15 12:05:20.336612 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:20.336588 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzzgg\" (UniqueName: \"kubernetes.io/projected/14ba6a32-b02c-4afd-9adb-4547d982708f-kube-api-access-wzzgg\") pod \"odh-model-controller-7998c67dc4-p2pr6\" (UID: \"14ba6a32-b02c-4afd-9adb-4547d982708f\") " pod="opendatahub/odh-model-controller-7998c67dc4-p2pr6" Jul 15 12:05:20.529051 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:20.528963 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a347f0d8-4a4b-4f5d-83b8-082a96f4aa7a-tls-certs\") pod \"kube-auth-proxy-7d7f886b65-qd67k\" (UID: \"a347f0d8-4a4b-4f5d-83b8-082a96f4aa7a\") " pod="openshift-ingress/kube-auth-proxy-7d7f886b65-qd67k" Jul 15 12:05:20.531316 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:20.531297 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a347f0d8-4a4b-4f5d-83b8-082a96f4aa7a-tls-certs\") pod \"kube-auth-proxy-7d7f886b65-qd67k\" (UID: \"a347f0d8-4a4b-4f5d-83b8-082a96f4aa7a\") " pod="openshift-ingress/kube-auth-proxy-7d7f886b65-qd67k" Jul 15 12:05:20.822254 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:20.822222 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7d7f886b65-qd67k" Jul 15 12:05:20.832722 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:20.832693 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/14ba6a32-b02c-4afd-9adb-4547d982708f-cert\") pod \"odh-model-controller-7998c67dc4-p2pr6\" (UID: \"14ba6a32-b02c-4afd-9adb-4547d982708f\") " pod="opendatahub/odh-model-controller-7998c67dc4-p2pr6" Jul 15 12:05:20.835208 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:20.835181 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/14ba6a32-b02c-4afd-9adb-4547d982708f-cert\") pod \"odh-model-controller-7998c67dc4-p2pr6\" (UID: \"14ba6a32-b02c-4afd-9adb-4547d982708f\") " pod="opendatahub/odh-model-controller-7998c67dc4-p2pr6" Jul 15 12:05:20.972079 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:20.972047 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7d7f886b65-qd67k"] Jul 15 12:05:20.974694 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:05:20.974666 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda347f0d8_4a4b_4f5d_83b8_082a96f4aa7a.slice/crio-738850fc4f9f5ca7b30db12b69790bba0ae16c9e0b4429d95a9834bdf908ee05 WatchSource:0}: Error finding container 738850fc4f9f5ca7b30db12b69790bba0ae16c9e0b4429d95a9834bdf908ee05: Status 404 returned error can't find the container with id 738850fc4f9f5ca7b30db12b69790bba0ae16c9e0b4429d95a9834bdf908ee05 Jul 15 12:05:21.055588 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:21.055559 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-p2pr6" Jul 15 12:05:21.127329 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:21.124034 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-j4h5d"] Jul 15 12:05:21.130802 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:21.130774 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-j4h5d" Jul 15 12:05:21.134755 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:21.134668 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-2zbds\"" Jul 15 12:05:21.134755 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:21.134713 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 15 12:05:21.136167 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:21.136144 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-j4h5d"] Jul 15 12:05:21.203451 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:21.203426 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-p2pr6"] Jul 15 12:05:21.205217 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:05:21.205183 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14ba6a32_b02c_4afd_9adb_4547d982708f.slice/crio-f766f0a90a44fdde7f1d9c36b486192578967026c587d8e2b96dce7943a8d7d4 WatchSource:0}: Error finding container f766f0a90a44fdde7f1d9c36b486192578967026c587d8e2b96dce7943a8d7d4: Status 404 returned error can't find the container with id f766f0a90a44fdde7f1d9c36b486192578967026c587d8e2b96dce7943a8d7d4 Jul 15 12:05:21.235649 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:21.235620 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pwcg\" (UniqueName: \"kubernetes.io/projected/15f67716-be1c-4a3f-9dc9-d35980060c82-kube-api-access-6pwcg\") pod \"model-serving-api-54d89bd5fc-j4h5d\" (UID: \"15f67716-be1c-4a3f-9dc9-d35980060c82\") " pod="opendatahub/model-serving-api-54d89bd5fc-j4h5d" Jul 15 12:05:21.235822 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:21.235663 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/15f67716-be1c-4a3f-9dc9-d35980060c82-tls-certs\") pod \"model-serving-api-54d89bd5fc-j4h5d\" (UID: \"15f67716-be1c-4a3f-9dc9-d35980060c82\") " pod="opendatahub/model-serving-api-54d89bd5fc-j4h5d" Jul 15 12:05:21.333937 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:21.333908 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-hwk8s"] Jul 15 12:05:21.336279 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:21.336255 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pwcg\" (UniqueName: \"kubernetes.io/projected/15f67716-be1c-4a3f-9dc9-d35980060c82-kube-api-access-6pwcg\") pod \"model-serving-api-54d89bd5fc-j4h5d\" (UID: \"15f67716-be1c-4a3f-9dc9-d35980060c82\") " pod="opendatahub/model-serving-api-54d89bd5fc-j4h5d" Jul 15 12:05:21.336350 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:21.336299 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/15f67716-be1c-4a3f-9dc9-d35980060c82-tls-certs\") pod \"model-serving-api-54d89bd5fc-j4h5d\" (UID: \"15f67716-be1c-4a3f-9dc9-d35980060c82\") " pod="opendatahub/model-serving-api-54d89bd5fc-j4h5d" Jul 15 12:05:21.337342 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:21.337326 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-hwk8s" Jul 15 12:05:21.338758 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:21.338724 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/15f67716-be1c-4a3f-9dc9-d35980060c82-tls-certs\") pod \"model-serving-api-54d89bd5fc-j4h5d\" (UID: \"15f67716-be1c-4a3f-9dc9-d35980060c82\") " pod="opendatahub/model-serving-api-54d89bd5fc-j4h5d" Jul 15 12:05:21.339704 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:21.339684 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 15 12:05:21.339886 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:21.339868 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 15 12:05:21.339956 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:21.339884 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-ln982\"" Jul 15 12:05:21.340103 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:21.340089 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 15 12:05:21.340161 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:21.340094 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 15 12:05:21.344578 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:21.344555 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pwcg\" (UniqueName: \"kubernetes.io/projected/15f67716-be1c-4a3f-9dc9-d35980060c82-kube-api-access-6pwcg\") pod \"model-serving-api-54d89bd5fc-j4h5d\" (UID: \"15f67716-be1c-4a3f-9dc9-d35980060c82\") " pod="opendatahub/model-serving-api-54d89bd5fc-j4h5d" Jul 15 12:05:21.351124 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:21.351103 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-hwk8s"] Jul 15 12:05:21.437525 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:21.437455 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b1c6e24e-3b7a-4fe2-9970-343f84e238b8-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-hwk8s\" (UID: \"b1c6e24e-3b7a-4fe2-9970-343f84e238b8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hwk8s" Jul 15 12:05:21.437525 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:21.437491 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b1c6e24e-3b7a-4fe2-9970-343f84e238b8-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-hwk8s\" (UID: \"b1c6e24e-3b7a-4fe2-9970-343f84e238b8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hwk8s" Jul 15 12:05:21.437706 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:21.437539 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1c6e24e-3b7a-4fe2-9970-343f84e238b8-local-certs\") pod \"istiod-openshift-gateway-7574459656-hwk8s\" (UID: \"b1c6e24e-3b7a-4fe2-9970-343f84e238b8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hwk8s" Jul 15 12:05:21.437706 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:21.437571 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhpnz\" (UniqueName: \"kubernetes.io/projected/b1c6e24e-3b7a-4fe2-9970-343f84e238b8-kube-api-access-hhpnz\") pod \"istiod-openshift-gateway-7574459656-hwk8s\" (UID: \"b1c6e24e-3b7a-4fe2-9970-343f84e238b8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hwk8s" Jul 15 12:05:21.437706 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:21.437598 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1c6e24e-3b7a-4fe2-9970-343f84e238b8-istio-token\") pod \"istiod-openshift-gateway-7574459656-hwk8s\" (UID: \"b1c6e24e-3b7a-4fe2-9970-343f84e238b8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hwk8s" Jul 15 12:05:21.437706 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:21.437623 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b1c6e24e-3b7a-4fe2-9970-343f84e238b8-cacerts\") pod \"istiod-openshift-gateway-7574459656-hwk8s\" (UID: \"b1c6e24e-3b7a-4fe2-9970-343f84e238b8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hwk8s" Jul 15 12:05:21.437706 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:21.437668 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b1c6e24e-3b7a-4fe2-9970-343f84e238b8-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-hwk8s\" (UID: \"b1c6e24e-3b7a-4fe2-9970-343f84e238b8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hwk8s" Jul 15 12:05:21.442405 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:21.442387 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-j4h5d" Jul 15 12:05:21.538947 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:21.538913 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhpnz\" (UniqueName: \"kubernetes.io/projected/b1c6e24e-3b7a-4fe2-9970-343f84e238b8-kube-api-access-hhpnz\") pod \"istiod-openshift-gateway-7574459656-hwk8s\" (UID: \"b1c6e24e-3b7a-4fe2-9970-343f84e238b8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hwk8s" Jul 15 12:05:21.540318 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:21.538964 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1c6e24e-3b7a-4fe2-9970-343f84e238b8-istio-token\") pod \"istiod-openshift-gateway-7574459656-hwk8s\" (UID: \"b1c6e24e-3b7a-4fe2-9970-343f84e238b8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hwk8s" Jul 15 12:05:21.540318 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:21.538999 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b1c6e24e-3b7a-4fe2-9970-343f84e238b8-cacerts\") pod \"istiod-openshift-gateway-7574459656-hwk8s\" (UID: \"b1c6e24e-3b7a-4fe2-9970-343f84e238b8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hwk8s" Jul 15 12:05:21.540318 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:21.539051 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b1c6e24e-3b7a-4fe2-9970-343f84e238b8-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-hwk8s\" (UID: \"b1c6e24e-3b7a-4fe2-9970-343f84e238b8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hwk8s" Jul 15 12:05:21.540318 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:21.539114 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b1c6e24e-3b7a-4fe2-9970-343f84e238b8-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-hwk8s\" (UID: \"b1c6e24e-3b7a-4fe2-9970-343f84e238b8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hwk8s" Jul 15 12:05:21.540318 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:21.539143 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b1c6e24e-3b7a-4fe2-9970-343f84e238b8-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-hwk8s\" (UID: \"b1c6e24e-3b7a-4fe2-9970-343f84e238b8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hwk8s" Jul 15 12:05:21.540318 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:21.539226 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1c6e24e-3b7a-4fe2-9970-343f84e238b8-local-certs\") pod \"istiod-openshift-gateway-7574459656-hwk8s\" (UID: \"b1c6e24e-3b7a-4fe2-9970-343f84e238b8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hwk8s" Jul 15 12:05:21.540652 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:21.540409 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b1c6e24e-3b7a-4fe2-9970-343f84e238b8-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-hwk8s\" (UID: \"b1c6e24e-3b7a-4fe2-9970-343f84e238b8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hwk8s" Jul 15 12:05:21.542033 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:21.541985 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1c6e24e-3b7a-4fe2-9970-343f84e238b8-local-certs\") pod \"istiod-openshift-gateway-7574459656-hwk8s\" (UID: \"b1c6e24e-3b7a-4fe2-9970-343f84e238b8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hwk8s" Jul 15 12:05:21.542434 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:21.542388 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b1c6e24e-3b7a-4fe2-9970-343f84e238b8-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-hwk8s\" (UID: \"b1c6e24e-3b7a-4fe2-9970-343f84e238b8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hwk8s" Jul 15 12:05:21.543411 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:21.543383 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b1c6e24e-3b7a-4fe2-9970-343f84e238b8-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-hwk8s\" (UID: \"b1c6e24e-3b7a-4fe2-9970-343f84e238b8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hwk8s" Jul 15 12:05:21.544007 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:21.543969 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b1c6e24e-3b7a-4fe2-9970-343f84e238b8-cacerts\") pod \"istiod-openshift-gateway-7574459656-hwk8s\" (UID: \"b1c6e24e-3b7a-4fe2-9970-343f84e238b8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hwk8s" Jul 15 12:05:21.547851 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:21.547805 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1c6e24e-3b7a-4fe2-9970-343f84e238b8-istio-token\") pod \"istiod-openshift-gateway-7574459656-hwk8s\" (UID: \"b1c6e24e-3b7a-4fe2-9970-343f84e238b8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hwk8s" Jul 15 12:05:21.548173 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:21.548144 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhpnz\" (UniqueName: \"kubernetes.io/projected/b1c6e24e-3b7a-4fe2-9970-343f84e238b8-kube-api-access-hhpnz\") pod \"istiod-openshift-gateway-7574459656-hwk8s\" (UID: \"b1c6e24e-3b7a-4fe2-9970-343f84e238b8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hwk8s" Jul 15 12:05:21.589845 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:21.589810 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-j4h5d"] Jul 15 12:05:21.591991 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:05:21.591958 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15f67716_be1c_4a3f_9dc9_d35980060c82.slice/crio-84958d86d7f00da14a1026578600d9189f3183d127e184fbddd062134ade3858 WatchSource:0}: Error finding container 84958d86d7f00da14a1026578600d9189f3183d127e184fbddd062134ade3858: Status 404 returned error can't find the container with id 84958d86d7f00da14a1026578600d9189f3183d127e184fbddd062134ade3858 Jul 15 12:05:21.655395 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:21.655362 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-hwk8s" Jul 15 12:05:21.811604 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:21.811568 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-hwk8s"] Jul 15 12:05:21.814934 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:05:21.814903 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1c6e24e_3b7a_4fe2_9970_343f84e238b8.slice/crio-1f2c03d50155cbb2618ad8e24d87425b9e8a190af2b78c26037fd19a3976e279 WatchSource:0}: Error finding container 1f2c03d50155cbb2618ad8e24d87425b9e8a190af2b78c26037fd19a3976e279: Status 404 returned error can't find the container with id 1f2c03d50155cbb2618ad8e24d87425b9e8a190af2b78c26037fd19a3976e279 Jul 15 12:05:21.941879 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:21.941841 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-j4h5d" event={"ID":"15f67716-be1c-4a3f-9dc9-d35980060c82","Type":"ContainerStarted","Data":"84958d86d7f00da14a1026578600d9189f3183d127e184fbddd062134ade3858"} Jul 15 12:05:21.943251 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:21.943225 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-hwk8s" event={"ID":"b1c6e24e-3b7a-4fe2-9970-343f84e238b8","Type":"ContainerStarted","Data":"1f2c03d50155cbb2618ad8e24d87425b9e8a190af2b78c26037fd19a3976e279"} Jul 15 12:05:21.944494 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:21.944467 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-p2pr6" event={"ID":"14ba6a32-b02c-4afd-9adb-4547d982708f","Type":"ContainerStarted","Data":"f766f0a90a44fdde7f1d9c36b486192578967026c587d8e2b96dce7943a8d7d4"} Jul 15 12:05:21.945945 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:21.945910 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7d7f886b65-qd67k" event={"ID":"a347f0d8-4a4b-4f5d-83b8-082a96f4aa7a","Type":"ContainerStarted","Data":"738850fc4f9f5ca7b30db12b69790bba0ae16c9e0b4429d95a9834bdf908ee05"} Jul 15 12:05:26.249540 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:26.249511 2587 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 12:05:26.249815 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:26.249568 2587 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 12:05:26.976954 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:26.976902 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-hwk8s" event={"ID":"b1c6e24e-3b7a-4fe2-9970-343f84e238b8","Type":"ContainerStarted","Data":"824436874a175bfa9218eeb601ee4d717c804037c896d515f710b4ef0608c3b7"} Jul 15 12:05:26.977182 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:26.977023 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-hwk8s" Jul 15 12:05:26.978771 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:26.978707 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-p2pr6" event={"ID":"14ba6a32-b02c-4afd-9adb-4547d982708f","Type":"ContainerStarted","Data":"d323430910b952a9f1c557499aae35de810f01fdf1e7b555f054af8b671bfe72"} Jul 15 12:05:26.978940 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:26.978780 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-p2pr6" Jul 15 12:05:26.980400 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:26.980370 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7d7f886b65-qd67k" event={"ID":"a347f0d8-4a4b-4f5d-83b8-082a96f4aa7a","Type":"ContainerStarted","Data":"e11f582ea478a0032d5863190f12483d0b5ebbc41073be2bc1a939988277e398"} Jul 15 12:05:26.982165 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:26.982136 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-j4h5d" event={"ID":"15f67716-be1c-4a3f-9dc9-d35980060c82","Type":"ContainerStarted","Data":"49f705aa4e2d2efbb3b111cb8a25fa311170553cbb582d8a9cb76731e31b2541"} Jul 15 12:05:26.982280 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:26.982206 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-j4h5d" Jul 15 12:05:27.007505 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:27.007449 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-hwk8s" podStartSLOduration=1.575364513 podStartE2EDuration="6.007435209s" totalImagesPullingTime="4.432070696s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:05:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:05:27.006137838 +0000 UTC m=+398.463514836" watchObservedRunningTime="2026-07-15 12:05:27.007435209 +0000 UTC m=+398.464812133" Jul 15 12:05:27.029765 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:27.029699 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-j4h5d" podStartSLOduration=1.3810496780000001 podStartE2EDuration="6.029682815s" totalImagesPullingTime="4.648633137s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:05:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:05:27.028490106 +0000 UTC m=+398.485867031" watchObservedRunningTime="2026-07-15 12:05:27.029682815 +0000 UTC m=+398.487059739" Jul 15 12:05:27.065029 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:27.064955 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-p2pr6" podStartSLOduration=2.023012026 podStartE2EDuration="7.064932178s" totalImagesPullingTime="5.041920152s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:05:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:05:27.060707201 +0000 UTC m=+398.518084146" watchObservedRunningTime="2026-07-15 12:05:27.064932178 +0000 UTC m=+398.522309119" Jul 15 12:05:27.122301 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:27.122246 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7d7f886b65-qd67k" podStartSLOduration=2.8560790689999997 podStartE2EDuration="8.122228884s" totalImagesPullingTime="5.266149815s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:05:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:05:27.118459691 +0000 UTC m=+398.575836662" watchObservedRunningTime="2026-07-15 12:05:27.122228884 +0000 UTC m=+398.579605808" Jul 15 12:05:27.647581 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:27.647548 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4lxxm"] Jul 15 12:05:27.651110 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:27.651093 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4lxxm" Jul 15 12:05:27.669589 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:27.669552 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4lxxm"] Jul 15 12:05:27.697180 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:27.697140 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6dd415bd-a802-4b9d-8a67-b81b97863da4-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-4lxxm\" (UID: \"6dd415bd-a802-4b9d-8a67-b81b97863da4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4lxxm" Jul 15 12:05:27.697361 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:27.697194 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6dd415bd-a802-4b9d-8a67-b81b97863da4-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-4lxxm\" (UID: \"6dd415bd-a802-4b9d-8a67-b81b97863da4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4lxxm" Jul 15 12:05:27.697361 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:27.697221 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6dd415bd-a802-4b9d-8a67-b81b97863da4-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-4lxxm\" (UID: \"6dd415bd-a802-4b9d-8a67-b81b97863da4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4lxxm" Jul 15 12:05:27.697361 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:27.697241 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6dd415bd-a802-4b9d-8a67-b81b97863da4-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-4lxxm\" (UID: \"6dd415bd-a802-4b9d-8a67-b81b97863da4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4lxxm" Jul 15 12:05:27.697361 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:27.697272 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6dd415bd-a802-4b9d-8a67-b81b97863da4-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-4lxxm\" (UID: \"6dd415bd-a802-4b9d-8a67-b81b97863da4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4lxxm" Jul 15 12:05:27.697513 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:27.697366 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6dd415bd-a802-4b9d-8a67-b81b97863da4-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-4lxxm\" (UID: \"6dd415bd-a802-4b9d-8a67-b81b97863da4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4lxxm" Jul 15 12:05:27.697513 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:27.697425 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2pjn\" (UniqueName: \"kubernetes.io/projected/6dd415bd-a802-4b9d-8a67-b81b97863da4-kube-api-access-k2pjn\") pod \"istiod-openshift-gateway-94bb8fbfd-4lxxm\" (UID: \"6dd415bd-a802-4b9d-8a67-b81b97863da4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4lxxm" Jul 15 12:05:27.798235 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:27.798196 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2pjn\" (UniqueName: \"kubernetes.io/projected/6dd415bd-a802-4b9d-8a67-b81b97863da4-kube-api-access-k2pjn\") pod \"istiod-openshift-gateway-94bb8fbfd-4lxxm\" (UID: \"6dd415bd-a802-4b9d-8a67-b81b97863da4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4lxxm" Jul 15 12:05:27.798390 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:27.798247 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6dd415bd-a802-4b9d-8a67-b81b97863da4-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-4lxxm\" (UID: \"6dd415bd-a802-4b9d-8a67-b81b97863da4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4lxxm" Jul 15 12:05:27.798390 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:27.798274 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6dd415bd-a802-4b9d-8a67-b81b97863da4-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-4lxxm\" (UID: \"6dd415bd-a802-4b9d-8a67-b81b97863da4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4lxxm" Jul 15 12:05:27.798390 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:27.798289 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6dd415bd-a802-4b9d-8a67-b81b97863da4-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-4lxxm\" (UID: \"6dd415bd-a802-4b9d-8a67-b81b97863da4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4lxxm" Jul 15 12:05:27.798390 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:27.798305 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6dd415bd-a802-4b9d-8a67-b81b97863da4-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-4lxxm\" (UID: \"6dd415bd-a802-4b9d-8a67-b81b97863da4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4lxxm" Jul 15 12:05:27.798597 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:27.798489 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6dd415bd-a802-4b9d-8a67-b81b97863da4-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-4lxxm\" (UID: \"6dd415bd-a802-4b9d-8a67-b81b97863da4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4lxxm" Jul 15 12:05:27.798597 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:27.798573 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6dd415bd-a802-4b9d-8a67-b81b97863da4-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-4lxxm\" (UID: \"6dd415bd-a802-4b9d-8a67-b81b97863da4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4lxxm" Jul 15 12:05:27.799362 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:27.799335 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6dd415bd-a802-4b9d-8a67-b81b97863da4-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-4lxxm\" (UID: \"6dd415bd-a802-4b9d-8a67-b81b97863da4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4lxxm" Jul 15 12:05:27.801101 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:27.801076 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6dd415bd-a802-4b9d-8a67-b81b97863da4-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-4lxxm\" (UID: \"6dd415bd-a802-4b9d-8a67-b81b97863da4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4lxxm" Jul 15 12:05:27.801203 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:27.801165 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6dd415bd-a802-4b9d-8a67-b81b97863da4-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-4lxxm\" (UID: \"6dd415bd-a802-4b9d-8a67-b81b97863da4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4lxxm" Jul 15 12:05:27.801263 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:27.801246 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6dd415bd-a802-4b9d-8a67-b81b97863da4-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-4lxxm\" (UID: \"6dd415bd-a802-4b9d-8a67-b81b97863da4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4lxxm" Jul 15 12:05:27.801439 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:27.801419 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6dd415bd-a802-4b9d-8a67-b81b97863da4-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-4lxxm\" (UID: \"6dd415bd-a802-4b9d-8a67-b81b97863da4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4lxxm" Jul 15 12:05:27.810796 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:27.810773 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2pjn\" (UniqueName: \"kubernetes.io/projected/6dd415bd-a802-4b9d-8a67-b81b97863da4-kube-api-access-k2pjn\") pod \"istiod-openshift-gateway-94bb8fbfd-4lxxm\" (UID: \"6dd415bd-a802-4b9d-8a67-b81b97863da4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4lxxm" Jul 15 12:05:27.820839 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:27.820811 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6dd415bd-a802-4b9d-8a67-b81b97863da4-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-4lxxm\" (UID: \"6dd415bd-a802-4b9d-8a67-b81b97863da4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4lxxm" Jul 15 12:05:27.960486 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:27.960403 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4lxxm" Jul 15 12:05:27.990239 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:27.990210 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-hwk8s" Jul 15 12:05:28.115529 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:28.115507 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4lxxm"] Jul 15 12:05:28.117107 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:05:28.117081 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dd415bd_a802_4b9d_8a67_b81b97863da4.slice/crio-e950d25ef32438bcbabceeff1ed2199c5eef6b09b9c2462e395dfaf9e528b958 WatchSource:0}: Error finding container e950d25ef32438bcbabceeff1ed2199c5eef6b09b9c2462e395dfaf9e528b958: Status 404 returned error can't find the container with id e950d25ef32438bcbabceeff1ed2199c5eef6b09b9c2462e395dfaf9e528b958 Jul 15 12:05:28.120037 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:28.120006 2587 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 12:05:28.120119 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:28.120059 2587 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 12:05:28.994548 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:28.994454 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4lxxm" event={"ID":"6dd415bd-a802-4b9d-8a67-b81b97863da4","Type":"ContainerStarted","Data":"ddb2b0ad6b2d6d4c7b9990598fa9c513a9baed35a63b97f4621a8f0f9341eb7f"} Jul 15 12:05:28.994914 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:28.994642 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4lxxm" Jul 15 12:05:28.994914 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:28.994672 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4lxxm" event={"ID":"6dd415bd-a802-4b9d-8a67-b81b97863da4","Type":"ContainerStarted","Data":"e950d25ef32438bcbabceeff1ed2199c5eef6b09b9c2462e395dfaf9e528b958"} Jul 15 12:05:29.023800 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:29.023747 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4lxxm" podStartSLOduration=2.023717587 podStartE2EDuration="2.023717587s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:05:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:05:29.021648624 +0000 UTC m=+400.479025569" watchObservedRunningTime="2026-07-15 12:05:29.023717587 +0000 UTC m=+400.481094512" Jul 15 12:05:30.001799 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:30.001765 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4lxxm" Jul 15 12:05:30.099980 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:30.099932 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-hwk8s"] Jul 15 12:05:30.100138 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:30.100116 2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-hwk8s" podUID="b1c6e24e-3b7a-4fe2-9970-343f84e238b8" containerName="discovery" containerID="cri-o://824436874a175bfa9218eeb601ee4d717c804037c896d515f710b4ef0608c3b7" gracePeriod=30 Jul 15 12:05:30.349761 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:30.349722 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-hwk8s" Jul 15 12:05:30.421066 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:30.421035 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1c6e24e-3b7a-4fe2-9970-343f84e238b8-istio-token\") pod \"b1c6e24e-3b7a-4fe2-9970-343f84e238b8\" (UID: \"b1c6e24e-3b7a-4fe2-9970-343f84e238b8\") " Jul 15 12:05:30.421237 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:30.421109 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hhpnz\" (UniqueName: \"kubernetes.io/projected/b1c6e24e-3b7a-4fe2-9970-343f84e238b8-kube-api-access-hhpnz\") pod \"b1c6e24e-3b7a-4fe2-9970-343f84e238b8\" (UID: \"b1c6e24e-3b7a-4fe2-9970-343f84e238b8\") " Jul 15 12:05:30.421237 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:30.421171 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b1c6e24e-3b7a-4fe2-9970-343f84e238b8-istio-csr-dns-cert\") pod \"b1c6e24e-3b7a-4fe2-9970-343f84e238b8\" (UID: \"b1c6e24e-3b7a-4fe2-9970-343f84e238b8\") " Jul 15 12:05:30.421358 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:30.421270 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b1c6e24e-3b7a-4fe2-9970-343f84e238b8-istio-kubeconfig\") pod \"b1c6e24e-3b7a-4fe2-9970-343f84e238b8\" (UID: \"b1c6e24e-3b7a-4fe2-9970-343f84e238b8\") " Jul 15 12:05:30.421358 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:30.421333 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1c6e24e-3b7a-4fe2-9970-343f84e238b8-local-certs\") pod \"b1c6e24e-3b7a-4fe2-9970-343f84e238b8\" (UID: \"b1c6e24e-3b7a-4fe2-9970-343f84e238b8\") " Jul 15 12:05:30.421458 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:30.421378 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b1c6e24e-3b7a-4fe2-9970-343f84e238b8-cacerts\") pod \"b1c6e24e-3b7a-4fe2-9970-343f84e238b8\" (UID: \"b1c6e24e-3b7a-4fe2-9970-343f84e238b8\") " Jul 15 12:05:30.421458 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:30.421434 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b1c6e24e-3b7a-4fe2-9970-343f84e238b8-istio-csr-ca-configmap\") pod \"b1c6e24e-3b7a-4fe2-9970-343f84e238b8\" (UID: \"b1c6e24e-3b7a-4fe2-9970-343f84e238b8\") " Jul 15 12:05:30.422111 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:30.422052 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1c6e24e-3b7a-4fe2-9970-343f84e238b8-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "b1c6e24e-3b7a-4fe2-9970-343f84e238b8" (UID: "b1c6e24e-3b7a-4fe2-9970-343f84e238b8"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:05:30.424084 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:30.424055 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1c6e24e-3b7a-4fe2-9970-343f84e238b8-kube-api-access-hhpnz" (OuterVolumeSpecName: "kube-api-access-hhpnz") pod "b1c6e24e-3b7a-4fe2-9970-343f84e238b8" (UID: "b1c6e24e-3b7a-4fe2-9970-343f84e238b8"). InnerVolumeSpecName "kube-api-access-hhpnz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:05:30.424298 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:30.424276 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1c6e24e-3b7a-4fe2-9970-343f84e238b8-cacerts" (OuterVolumeSpecName: "cacerts") pod "b1c6e24e-3b7a-4fe2-9970-343f84e238b8" (UID: "b1c6e24e-3b7a-4fe2-9970-343f84e238b8"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 12:05:30.424478 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:30.424458 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1c6e24e-3b7a-4fe2-9970-343f84e238b8-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "b1c6e24e-3b7a-4fe2-9970-343f84e238b8" (UID: "b1c6e24e-3b7a-4fe2-9970-343f84e238b8"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 12:05:30.424611 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:30.424595 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1c6e24e-3b7a-4fe2-9970-343f84e238b8-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "b1c6e24e-3b7a-4fe2-9970-343f84e238b8" (UID: "b1c6e24e-3b7a-4fe2-9970-343f84e238b8"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 12:05:30.424971 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:30.424945 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1c6e24e-3b7a-4fe2-9970-343f84e238b8-istio-token" (OuterVolumeSpecName: "istio-token") pod "b1c6e24e-3b7a-4fe2-9970-343f84e238b8" (UID: "b1c6e24e-3b7a-4fe2-9970-343f84e238b8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:05:30.425395 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:30.425361 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1c6e24e-3b7a-4fe2-9970-343f84e238b8-local-certs" (OuterVolumeSpecName: "local-certs") pod "b1c6e24e-3b7a-4fe2-9970-343f84e238b8" (UID: "b1c6e24e-3b7a-4fe2-9970-343f84e238b8"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:05:30.522699 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:30.522606 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhpnz\" (UniqueName: \"kubernetes.io/projected/b1c6e24e-3b7a-4fe2-9970-343f84e238b8-kube-api-access-hhpnz\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:05:30.522699 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:30.522639 2587 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b1c6e24e-3b7a-4fe2-9970-343f84e238b8-istio-csr-dns-cert\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:05:30.522699 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:30.522651 2587 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b1c6e24e-3b7a-4fe2-9970-343f84e238b8-istio-kubeconfig\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:05:30.522699 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:30.522662 2587 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1c6e24e-3b7a-4fe2-9970-343f84e238b8-local-certs\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:05:30.522699 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:30.522673 2587 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b1c6e24e-3b7a-4fe2-9970-343f84e238b8-cacerts\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:05:30.522699 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:30.522683 2587 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b1c6e24e-3b7a-4fe2-9970-343f84e238b8-istio-csr-ca-configmap\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:05:30.522699 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:30.522694 2587 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1c6e24e-3b7a-4fe2-9970-343f84e238b8-istio-token\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:05:31.005875 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:31.005832 2587 generic.go:356] "Generic (PLEG): container finished" podID="b1c6e24e-3b7a-4fe2-9970-343f84e238b8" containerID="824436874a175bfa9218eeb601ee4d717c804037c896d515f710b4ef0608c3b7" exitCode=0 Jul 15 12:05:31.006317 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:31.005899 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-hwk8s" Jul 15 12:05:31.006317 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:31.005938 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-hwk8s" event={"ID":"b1c6e24e-3b7a-4fe2-9970-343f84e238b8","Type":"ContainerDied","Data":"824436874a175bfa9218eeb601ee4d717c804037c896d515f710b4ef0608c3b7"} Jul 15 12:05:31.006317 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:31.005993 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-hwk8s" event={"ID":"b1c6e24e-3b7a-4fe2-9970-343f84e238b8","Type":"ContainerDied","Data":"1f2c03d50155cbb2618ad8e24d87425b9e8a190af2b78c26037fd19a3976e279"} Jul 15 12:05:31.006317 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:31.006020 2587 scope.go:117] "RemoveContainer" containerID="824436874a175bfa9218eeb601ee4d717c804037c896d515f710b4ef0608c3b7" Jul 15 12:05:31.016881 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:31.016851 2587 scope.go:117] "RemoveContainer" containerID="824436874a175bfa9218eeb601ee4d717c804037c896d515f710b4ef0608c3b7" Jul 15 12:05:31.017233 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:05:31.017210 2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"824436874a175bfa9218eeb601ee4d717c804037c896d515f710b4ef0608c3b7\": container with ID starting with 824436874a175bfa9218eeb601ee4d717c804037c896d515f710b4ef0608c3b7 not found: ID does not exist" containerID="824436874a175bfa9218eeb601ee4d717c804037c896d515f710b4ef0608c3b7" Jul 15 12:05:31.017334 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:31.017246 2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"824436874a175bfa9218eeb601ee4d717c804037c896d515f710b4ef0608c3b7"} err="failed to get container status \"824436874a175bfa9218eeb601ee4d717c804037c896d515f710b4ef0608c3b7\": rpc error: code = NotFound desc = could not find container \"824436874a175bfa9218eeb601ee4d717c804037c896d515f710b4ef0608c3b7\": container with ID starting with 824436874a175bfa9218eeb601ee4d717c804037c896d515f710b4ef0608c3b7 not found: ID does not exist" Jul 15 12:05:31.055036 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:31.055004 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-hwk8s"] Jul 15 12:05:31.071108 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:31.071079 2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-hwk8s"] Jul 15 12:05:31.185653 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:31.185620 2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1c6e24e-3b7a-4fe2-9970-343f84e238b8" path="/var/lib/kubelet/pods/b1c6e24e-3b7a-4fe2-9970-343f84e238b8/volumes" Jul 15 12:05:31.185976 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:31.185959 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-bh8kq"] Jul 15 12:05:31.186268 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:31.186257 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1c6e24e-3b7a-4fe2-9970-343f84e238b8" containerName="discovery" Jul 15 12:05:31.186305 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:31.186271 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1c6e24e-3b7a-4fe2-9970-343f84e238b8" containerName="discovery" Jul 15 12:05:31.186350 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:31.186341 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1c6e24e-3b7a-4fe2-9970-343f84e238b8" containerName="discovery" Jul 15 12:05:31.190388 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:31.190371 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bh8kq" Jul 15 12:05:31.193071 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:31.193043 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 15 12:05:31.193186 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:31.193130 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-69fcl\"" Jul 15 12:05:31.201211 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:31.201184 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-bh8kq"] Jul 15 12:05:31.228231 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:31.228199 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/41bdfb9d-6267-4e66-8925-31ffacf48d39-cert\") pod \"kserve-controller-manager-d6f4b65b8-bh8kq\" (UID: \"41bdfb9d-6267-4e66-8925-31ffacf48d39\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-bh8kq" Jul 15 12:05:31.228377 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:31.228343 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfcvz\" (UniqueName: \"kubernetes.io/projected/41bdfb9d-6267-4e66-8925-31ffacf48d39-kube-api-access-jfcvz\") pod \"kserve-controller-manager-d6f4b65b8-bh8kq\" (UID: \"41bdfb9d-6267-4e66-8925-31ffacf48d39\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-bh8kq" Jul 15 12:05:31.328817 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:31.328787 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfcvz\" (UniqueName: \"kubernetes.io/projected/41bdfb9d-6267-4e66-8925-31ffacf48d39-kube-api-access-jfcvz\") pod \"kserve-controller-manager-d6f4b65b8-bh8kq\" (UID: \"41bdfb9d-6267-4e66-8925-31ffacf48d39\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-bh8kq" Jul 15 12:05:31.328989 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:31.328837 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/41bdfb9d-6267-4e66-8925-31ffacf48d39-cert\") pod \"kserve-controller-manager-d6f4b65b8-bh8kq\" (UID: \"41bdfb9d-6267-4e66-8925-31ffacf48d39\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-bh8kq" Jul 15 12:05:31.328989 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:05:31.328953 2587 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 15 12:05:31.329067 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:05:31.329009 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41bdfb9d-6267-4e66-8925-31ffacf48d39-cert podName:41bdfb9d-6267-4e66-8925-31ffacf48d39 nodeName:}" failed. No retries permitted until 2026-07-15 12:05:31.828990724 +0000 UTC m=+403.286367627 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/41bdfb9d-6267-4e66-8925-31ffacf48d39-cert") pod "kserve-controller-manager-d6f4b65b8-bh8kq" (UID: "41bdfb9d-6267-4e66-8925-31ffacf48d39") : secret "kserve-webhook-server-cert" not found Jul 15 12:05:31.339345 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:31.339315 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfcvz\" (UniqueName: \"kubernetes.io/projected/41bdfb9d-6267-4e66-8925-31ffacf48d39-kube-api-access-jfcvz\") pod \"kserve-controller-manager-d6f4b65b8-bh8kq\" (UID: \"41bdfb9d-6267-4e66-8925-31ffacf48d39\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-bh8kq" Jul 15 12:05:31.832374 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:31.832337 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/41bdfb9d-6267-4e66-8925-31ffacf48d39-cert\") pod \"kserve-controller-manager-d6f4b65b8-bh8kq\" (UID: \"41bdfb9d-6267-4e66-8925-31ffacf48d39\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-bh8kq" Jul 15 12:05:31.834830 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:31.834803 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/41bdfb9d-6267-4e66-8925-31ffacf48d39-cert\") pod \"kserve-controller-manager-d6f4b65b8-bh8kq\" (UID: \"41bdfb9d-6267-4e66-8925-31ffacf48d39\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-bh8kq" Jul 15 12:05:32.101518 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:32.101433 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bh8kq" Jul 15 12:05:32.250671 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:32.250647 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-bh8kq"] Jul 15 12:05:32.252175 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:05:32.252149 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41bdfb9d_6267_4e66_8925_31ffacf48d39.slice/crio-9385f1c9e8e204e6c3d18b4eb6732406f29ebb8c51079092d08703ecbe78696f WatchSource:0}: Error finding container 9385f1c9e8e204e6c3d18b4eb6732406f29ebb8c51079092d08703ecbe78696f: Status 404 returned error can't find the container with id 9385f1c9e8e204e6c3d18b4eb6732406f29ebb8c51079092d08703ecbe78696f Jul 15 12:05:33.017989 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:33.017955 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bh8kq" event={"ID":"41bdfb9d-6267-4e66-8925-31ffacf48d39","Type":"ContainerStarted","Data":"9385f1c9e8e204e6c3d18b4eb6732406f29ebb8c51079092d08703ecbe78696f"} Jul 15 12:05:37.040652 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:37.040616 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bh8kq" event={"ID":"41bdfb9d-6267-4e66-8925-31ffacf48d39","Type":"ContainerStarted","Data":"c3a5ddf71707a79dcbb88d1ba75562c4aadf21df49e83910deba8f1ec5e73b5a"} Jul 15 12:05:37.041056 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:37.040726 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bh8kq" Jul 15 12:05:37.061531 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:37.061484 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bh8kq" podStartSLOduration=1.8106952010000001 podStartE2EDuration="6.061470569s" totalImagesPullingTime="4.250775368s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:05:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:05:37.0589916 +0000 UTC m=+408.516368549" watchObservedRunningTime="2026-07-15 12:05:37.061470569 +0000 UTC m=+408.518847493" Jul 15 12:05:37.990612 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:37.990571 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-p2pr6" Jul 15 12:05:37.993505 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:05:37.993477 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-j4h5d" Jul 15 12:06:08.053110 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:08.053078 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bh8kq" Jul 15 12:06:20.950878 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:20.950796 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jj9cp"] Jul 15 12:06:20.954984 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:20.954965 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jj9cp" Jul 15 12:06:20.966564 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:20.966543 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 15 12:06:20.967121 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:20.967104 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 15 12:06:20.967875 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:20.967860 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-2fkml\"" Jul 15 12:06:20.972896 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:20.972873 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jj9cp"] Jul 15 12:06:21.054176 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:21.054137 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hl42p\" (UniqueName: \"kubernetes.io/projected/09b639ba-5a1e-49c9-83b6-32ecfbe341b3-kube-api-access-hl42p\") pod \"kuadrant-operator-catalog-jj9cp\" (UID: \"09b639ba-5a1e-49c9-83b6-32ecfbe341b3\") " pod="kuadrant-system/kuadrant-operator-catalog-jj9cp" Jul 15 12:06:21.154917 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:21.154874 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hl42p\" (UniqueName: \"kubernetes.io/projected/09b639ba-5a1e-49c9-83b6-32ecfbe341b3-kube-api-access-hl42p\") pod \"kuadrant-operator-catalog-jj9cp\" (UID: \"09b639ba-5a1e-49c9-83b6-32ecfbe341b3\") " pod="kuadrant-system/kuadrant-operator-catalog-jj9cp" Jul 15 12:06:21.164148 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:21.164117 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hl42p\" (UniqueName: \"kubernetes.io/projected/09b639ba-5a1e-49c9-83b6-32ecfbe341b3-kube-api-access-hl42p\") pod \"kuadrant-operator-catalog-jj9cp\" (UID: \"09b639ba-5a1e-49c9-83b6-32ecfbe341b3\") " pod="kuadrant-system/kuadrant-operator-catalog-jj9cp" Jul 15 12:06:21.263330 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:21.263252 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jj9cp" Jul 15 12:06:21.284720 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:21.284687 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jj9cp"] Jul 15 12:06:21.441518 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:21.441493 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jj9cp"] Jul 15 12:06:21.443922 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:06:21.443889 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09b639ba_5a1e_49c9_83b6_32ecfbe341b3.slice/crio-362684c481d3691a435467d5c377b1a131ed66abdc769adfed2b5336336e54e4 WatchSource:0}: Error finding container 362684c481d3691a435467d5c377b1a131ed66abdc769adfed2b5336336e54e4: Status 404 returned error can't find the container with id 362684c481d3691a435467d5c377b1a131ed66abdc769adfed2b5336336e54e4 Jul 15 12:06:22.267190 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:22.267147 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jj9cp" event={"ID":"09b639ba-5a1e-49c9-83b6-32ecfbe341b3","Type":"ContainerStarted","Data":"362684c481d3691a435467d5c377b1a131ed66abdc769adfed2b5336336e54e4"} Jul 15 12:06:25.283086 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:25.283046 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jj9cp" event={"ID":"09b639ba-5a1e-49c9-83b6-32ecfbe341b3","Type":"ContainerStarted","Data":"cf162b9df06e42d334b0f9fd4b5c7228c1c1369fd6b3945111009568485f4b6e"} Jul 15 12:06:25.283086 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:25.283076 2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-jj9cp" podUID="09b639ba-5a1e-49c9-83b6-32ecfbe341b3" containerName="registry-server" containerID="cri-o://cf162b9df06e42d334b0f9fd4b5c7228c1c1369fd6b3945111009568485f4b6e" gracePeriod=2 Jul 15 12:06:25.308382 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:25.308336 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-jj9cp" podStartSLOduration=2.327094121 podStartE2EDuration="5.308319147s" totalImagesPullingTime="2.981225026s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:06:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:06:25.305660725 +0000 UTC m=+456.763037663" watchObservedRunningTime="2026-07-15 12:06:25.308319147 +0000 UTC m=+456.765696072" Jul 15 12:06:25.540037 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:25.539974 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jj9cp" Jul 15 12:06:25.597616 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:25.597587 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hl42p\" (UniqueName: \"kubernetes.io/projected/09b639ba-5a1e-49c9-83b6-32ecfbe341b3-kube-api-access-hl42p\") pod \"09b639ba-5a1e-49c9-83b6-32ecfbe341b3\" (UID: \"09b639ba-5a1e-49c9-83b6-32ecfbe341b3\") " Jul 15 12:06:25.599718 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:25.599692 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09b639ba-5a1e-49c9-83b6-32ecfbe341b3-kube-api-access-hl42p" (OuterVolumeSpecName: "kube-api-access-hl42p") pod "09b639ba-5a1e-49c9-83b6-32ecfbe341b3" (UID: "09b639ba-5a1e-49c9-83b6-32ecfbe341b3"). InnerVolumeSpecName "kube-api-access-hl42p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:06:25.699143 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:25.699107 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hl42p\" (UniqueName: \"kubernetes.io/projected/09b639ba-5a1e-49c9-83b6-32ecfbe341b3-kube-api-access-hl42p\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:06:26.293975 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:26.293943 2587 generic.go:356] "Generic (PLEG): container finished" podID="09b639ba-5a1e-49c9-83b6-32ecfbe341b3" containerID="cf162b9df06e42d334b0f9fd4b5c7228c1c1369fd6b3945111009568485f4b6e" exitCode=0 Jul 15 12:06:26.294365 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:26.293993 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jj9cp" Jul 15 12:06:26.294365 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:26.294026 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jj9cp" event={"ID":"09b639ba-5a1e-49c9-83b6-32ecfbe341b3","Type":"ContainerDied","Data":"cf162b9df06e42d334b0f9fd4b5c7228c1c1369fd6b3945111009568485f4b6e"} Jul 15 12:06:26.294365 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:26.294100 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jj9cp" event={"ID":"09b639ba-5a1e-49c9-83b6-32ecfbe341b3","Type":"ContainerDied","Data":"362684c481d3691a435467d5c377b1a131ed66abdc769adfed2b5336336e54e4"} Jul 15 12:06:26.294365 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:26.294126 2587 scope.go:117] "RemoveContainer" containerID="cf162b9df06e42d334b0f9fd4b5c7228c1c1369fd6b3945111009568485f4b6e" Jul 15 12:06:26.303381 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:26.303366 2587 scope.go:117] "RemoveContainer" containerID="cf162b9df06e42d334b0f9fd4b5c7228c1c1369fd6b3945111009568485f4b6e" Jul 15 12:06:26.303637 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:06:26.303620 2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf162b9df06e42d334b0f9fd4b5c7228c1c1369fd6b3945111009568485f4b6e\": container with ID starting with cf162b9df06e42d334b0f9fd4b5c7228c1c1369fd6b3945111009568485f4b6e not found: ID does not exist" containerID="cf162b9df06e42d334b0f9fd4b5c7228c1c1369fd6b3945111009568485f4b6e" Jul 15 12:06:26.303699 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:26.303643 2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf162b9df06e42d334b0f9fd4b5c7228c1c1369fd6b3945111009568485f4b6e"} err="failed to get container status \"cf162b9df06e42d334b0f9fd4b5c7228c1c1369fd6b3945111009568485f4b6e\": rpc error: code = NotFound desc = could not find container \"cf162b9df06e42d334b0f9fd4b5c7228c1c1369fd6b3945111009568485f4b6e\": container with ID starting with cf162b9df06e42d334b0f9fd4b5c7228c1c1369fd6b3945111009568485f4b6e not found: ID does not exist" Jul 15 12:06:26.317182 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:26.317160 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jj9cp"] Jul 15 12:06:26.322367 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:26.322343 2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jj9cp"] Jul 15 12:06:27.185157 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:27.185125 2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09b639ba-5a1e-49c9-83b6-32ecfbe341b3" path="/var/lib/kubelet/pods/09b639ba-5a1e-49c9-83b6-32ecfbe341b3/volumes" Jul 15 12:06:36.123430 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:36.123391 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tcccd"] Jul 15 12:06:36.123825 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:36.123760 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09b639ba-5a1e-49c9-83b6-32ecfbe341b3" containerName="registry-server" Jul 15 12:06:36.123825 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:36.123772 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="09b639ba-5a1e-49c9-83b6-32ecfbe341b3" containerName="registry-server" Jul 15 12:06:36.123899 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:36.123845 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="09b639ba-5a1e-49c9-83b6-32ecfbe341b3" containerName="registry-server" Jul 15 12:06:36.128181 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:36.128164 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tcccd" Jul 15 12:06:36.131162 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:36.131143 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 15 12:06:36.132640 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:36.132619 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 15 12:06:36.133074 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:36.132666 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-vs7gg\"" Jul 15 12:06:36.135511 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:36.135483 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tcccd"] Jul 15 12:06:36.185413 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:36.185376 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnknr\" (UniqueName: \"kubernetes.io/projected/760fbdcd-8e40-4d90-8d93-7402a4aafee6-kube-api-access-nnknr\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tcccd\" (UID: \"760fbdcd-8e40-4d90-8d93-7402a4aafee6\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tcccd" Jul 15 12:06:36.185603 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:36.185430 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/760fbdcd-8e40-4d90-8d93-7402a4aafee6-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tcccd\" (UID: \"760fbdcd-8e40-4d90-8d93-7402a4aafee6\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tcccd" Jul 15 12:06:36.185603 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:36.185495 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/760fbdcd-8e40-4d90-8d93-7402a4aafee6-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tcccd\" (UID: \"760fbdcd-8e40-4d90-8d93-7402a4aafee6\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tcccd" Jul 15 12:06:36.286782 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:36.286713 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnknr\" (UniqueName: \"kubernetes.io/projected/760fbdcd-8e40-4d90-8d93-7402a4aafee6-kube-api-access-nnknr\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tcccd\" (UID: \"760fbdcd-8e40-4d90-8d93-7402a4aafee6\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tcccd" Jul 15 12:06:36.286958 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:36.286803 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/760fbdcd-8e40-4d90-8d93-7402a4aafee6-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tcccd\" (UID: \"760fbdcd-8e40-4d90-8d93-7402a4aafee6\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tcccd" Jul 15 12:06:36.286958 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:36.286860 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/760fbdcd-8e40-4d90-8d93-7402a4aafee6-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tcccd\" (UID: \"760fbdcd-8e40-4d90-8d93-7402a4aafee6\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tcccd" Jul 15 12:06:36.287248 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:36.287228 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/760fbdcd-8e40-4d90-8d93-7402a4aafee6-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tcccd\" (UID: \"760fbdcd-8e40-4d90-8d93-7402a4aafee6\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tcccd" Jul 15 12:06:36.287289 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:36.287245 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/760fbdcd-8e40-4d90-8d93-7402a4aafee6-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tcccd\" (UID: \"760fbdcd-8e40-4d90-8d93-7402a4aafee6\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tcccd" Jul 15 12:06:36.295881 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:36.295854 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnknr\" (UniqueName: \"kubernetes.io/projected/760fbdcd-8e40-4d90-8d93-7402a4aafee6-kube-api-access-nnknr\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tcccd\" (UID: \"760fbdcd-8e40-4d90-8d93-7402a4aafee6\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tcccd" Jul 15 12:06:36.438664 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:36.438569 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tcccd" Jul 15 12:06:36.576028 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:36.575918 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tcccd"] Jul 15 12:06:36.578721 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:06:36.578694 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod760fbdcd_8e40_4d90_8d93_7402a4aafee6.slice/crio-e80df9b497e377b6309f8e3d174c9c70839a605cc8ce5d46545febab00706528 WatchSource:0}: Error finding container e80df9b497e377b6309f8e3d174c9c70839a605cc8ce5d46545febab00706528: Status 404 returned error can't find the container with id e80df9b497e377b6309f8e3d174c9c70839a605cc8ce5d46545febab00706528 Jul 15 12:06:36.930624 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:36.930587 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8khc"] Jul 15 12:06:36.934173 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:36.934156 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8khc" Jul 15 12:06:36.947231 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:36.947203 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8khc"] Jul 15 12:06:36.993904 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:36.993868 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/70f2106a-f040-4eb3-ae55-4c46d899932b-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8khc\" (UID: \"70f2106a-f040-4eb3-ae55-4c46d899932b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8khc" Jul 15 12:06:36.993904 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:36.993909 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/70f2106a-f040-4eb3-ae55-4c46d899932b-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8khc\" (UID: \"70f2106a-f040-4eb3-ae55-4c46d899932b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8khc" Jul 15 12:06:36.994125 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:36.993936 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkfqj\" (UniqueName: \"kubernetes.io/projected/70f2106a-f040-4eb3-ae55-4c46d899932b-kube-api-access-kkfqj\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8khc\" (UID: \"70f2106a-f040-4eb3-ae55-4c46d899932b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8khc" Jul 15 12:06:37.094392 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:37.094358 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/70f2106a-f040-4eb3-ae55-4c46d899932b-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8khc\" (UID: \"70f2106a-f040-4eb3-ae55-4c46d899932b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8khc" Jul 15 12:06:37.094392 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:37.094397 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/70f2106a-f040-4eb3-ae55-4c46d899932b-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8khc\" (UID: \"70f2106a-f040-4eb3-ae55-4c46d899932b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8khc" Jul 15 12:06:37.094601 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:37.094441 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkfqj\" (UniqueName: \"kubernetes.io/projected/70f2106a-f040-4eb3-ae55-4c46d899932b-kube-api-access-kkfqj\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8khc\" (UID: \"70f2106a-f040-4eb3-ae55-4c46d899932b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8khc" Jul 15 12:06:37.094862 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:37.094842 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/70f2106a-f040-4eb3-ae55-4c46d899932b-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8khc\" (UID: \"70f2106a-f040-4eb3-ae55-4c46d899932b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8khc" Jul 15 12:06:37.094914 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:37.094899 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/70f2106a-f040-4eb3-ae55-4c46d899932b-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8khc\" (UID: \"70f2106a-f040-4eb3-ae55-4c46d899932b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8khc" Jul 15 12:06:37.105116 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:37.105085 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkfqj\" (UniqueName: \"kubernetes.io/projected/70f2106a-f040-4eb3-ae55-4c46d899932b-kube-api-access-kkfqj\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8khc\" (UID: \"70f2106a-f040-4eb3-ae55-4c46d899932b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8khc" Jul 15 12:06:37.243399 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:37.243315 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8khc" Jul 15 12:06:37.342718 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:37.342684 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qspjl"] Jul 15 12:06:37.347476 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:37.347451 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qspjl" Jul 15 12:06:37.351884 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:37.351854 2587 generic.go:356] "Generic (PLEG): container finished" podID="760fbdcd-8e40-4d90-8d93-7402a4aafee6" containerID="0f8e7aed05fe7fa824a5c0da62b42abb0b4ca00e9c9d4cc6bafa308f8d722f79" exitCode=0 Jul 15 12:06:37.352023 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:37.351988 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tcccd" event={"ID":"760fbdcd-8e40-4d90-8d93-7402a4aafee6","Type":"ContainerDied","Data":"0f8e7aed05fe7fa824a5c0da62b42abb0b4ca00e9c9d4cc6bafa308f8d722f79"} Jul 15 12:06:37.352071 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:37.352033 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tcccd" event={"ID":"760fbdcd-8e40-4d90-8d93-7402a4aafee6","Type":"ContainerStarted","Data":"e80df9b497e377b6309f8e3d174c9c70839a605cc8ce5d46545febab00706528"} Jul 15 12:06:37.361915 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:37.361875 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qspjl"] Jul 15 12:06:37.382935 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:37.382898 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8khc"] Jul 15 12:06:37.385284 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:06:37.385244 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70f2106a_f040_4eb3_ae55_4c46d899932b.slice/crio-d519b31a1ebdfde32142157cdb319d42eea88249c121005e6238dfe4186ac182 WatchSource:0}: Error finding container d519b31a1ebdfde32142157cdb319d42eea88249c121005e6238dfe4186ac182: Status 404 returned error can't find the container with id d519b31a1ebdfde32142157cdb319d42eea88249c121005e6238dfe4186ac182 Jul 15 12:06:37.397291 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:37.397264 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkv25\" (UniqueName: \"kubernetes.io/projected/4e4e75e4-4576-4bab-922e-ffe34d7890ff-kube-api-access-gkv25\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qspjl\" (UID: \"4e4e75e4-4576-4bab-922e-ffe34d7890ff\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qspjl" Jul 15 12:06:37.397389 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:37.397375 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4e4e75e4-4576-4bab-922e-ffe34d7890ff-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qspjl\" (UID: \"4e4e75e4-4576-4bab-922e-ffe34d7890ff\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qspjl" Jul 15 12:06:37.397431 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:37.397420 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4e4e75e4-4576-4bab-922e-ffe34d7890ff-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qspjl\" (UID: \"4e4e75e4-4576-4bab-922e-ffe34d7890ff\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qspjl" Jul 15 12:06:37.498536 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:37.498508 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4e4e75e4-4576-4bab-922e-ffe34d7890ff-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qspjl\" (UID: \"4e4e75e4-4576-4bab-922e-ffe34d7890ff\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qspjl" Jul 15 12:06:37.498650 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:37.498565 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4e4e75e4-4576-4bab-922e-ffe34d7890ff-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qspjl\" (UID: \"4e4e75e4-4576-4bab-922e-ffe34d7890ff\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qspjl" Jul 15 12:06:37.498711 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:37.498635 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkv25\" (UniqueName: \"kubernetes.io/projected/4e4e75e4-4576-4bab-922e-ffe34d7890ff-kube-api-access-gkv25\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qspjl\" (UID: \"4e4e75e4-4576-4bab-922e-ffe34d7890ff\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qspjl" Jul 15 12:06:37.498899 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:37.498880 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4e4e75e4-4576-4bab-922e-ffe34d7890ff-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qspjl\" (UID: \"4e4e75e4-4576-4bab-922e-ffe34d7890ff\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qspjl" Jul 15 12:06:37.498967 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:37.498925 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4e4e75e4-4576-4bab-922e-ffe34d7890ff-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qspjl\" (UID: \"4e4e75e4-4576-4bab-922e-ffe34d7890ff\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qspjl" Jul 15 12:06:37.508321 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:37.508299 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkv25\" (UniqueName: \"kubernetes.io/projected/4e4e75e4-4576-4bab-922e-ffe34d7890ff-kube-api-access-gkv25\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qspjl\" (UID: \"4e4e75e4-4576-4bab-922e-ffe34d7890ff\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qspjl" Jul 15 12:06:37.660558 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:37.660525 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qspjl" Jul 15 12:06:37.752874 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:37.752714 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l8c6h"] Jul 15 12:06:37.764508 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:37.764477 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l8c6h" Jul 15 12:06:37.779263 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:37.779236 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l8c6h"] Jul 15 12:06:37.802030 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:37.801977 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qspjl"] Jul 15 12:06:37.804258 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:06:37.804232 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e4e75e4_4576_4bab_922e_ffe34d7890ff.slice/crio-2370683744dc95443571ad2a23d9d078558c702180b868975c9c7c95bbb67a7a WatchSource:0}: Error finding container 2370683744dc95443571ad2a23d9d078558c702180b868975c9c7c95bbb67a7a: Status 404 returned error can't find the container with id 2370683744dc95443571ad2a23d9d078558c702180b868975c9c7c95bbb67a7a Jul 15 12:06:37.902069 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:37.902030 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msd8m\" (UniqueName: \"kubernetes.io/projected/825aeecb-52ad-4bd5-bba0-b4438705430d-kube-api-access-msd8m\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l8c6h\" (UID: \"825aeecb-52ad-4bd5-bba0-b4438705430d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l8c6h" Jul 15 12:06:37.902266 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:37.902095 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/825aeecb-52ad-4bd5-bba0-b4438705430d-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l8c6h\" (UID: \"825aeecb-52ad-4bd5-bba0-b4438705430d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l8c6h" Jul 15 12:06:37.902266 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:37.902167 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/825aeecb-52ad-4bd5-bba0-b4438705430d-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l8c6h\" (UID: \"825aeecb-52ad-4bd5-bba0-b4438705430d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l8c6h" Jul 15 12:06:38.003152 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:38.003078 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/825aeecb-52ad-4bd5-bba0-b4438705430d-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l8c6h\" (UID: \"825aeecb-52ad-4bd5-bba0-b4438705430d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l8c6h" Jul 15 12:06:38.003152 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:38.003125 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/825aeecb-52ad-4bd5-bba0-b4438705430d-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l8c6h\" (UID: \"825aeecb-52ad-4bd5-bba0-b4438705430d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l8c6h" Jul 15 12:06:38.003328 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:38.003176 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msd8m\" (UniqueName: \"kubernetes.io/projected/825aeecb-52ad-4bd5-bba0-b4438705430d-kube-api-access-msd8m\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l8c6h\" (UID: \"825aeecb-52ad-4bd5-bba0-b4438705430d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l8c6h" Jul 15 12:06:38.003431 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:38.003414 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/825aeecb-52ad-4bd5-bba0-b4438705430d-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l8c6h\" (UID: \"825aeecb-52ad-4bd5-bba0-b4438705430d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l8c6h" Jul 15 12:06:38.003535 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:38.003515 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/825aeecb-52ad-4bd5-bba0-b4438705430d-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l8c6h\" (UID: \"825aeecb-52ad-4bd5-bba0-b4438705430d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l8c6h" Jul 15 12:06:38.013603 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:38.013578 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-msd8m\" (UniqueName: \"kubernetes.io/projected/825aeecb-52ad-4bd5-bba0-b4438705430d-kube-api-access-msd8m\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l8c6h\" (UID: \"825aeecb-52ad-4bd5-bba0-b4438705430d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l8c6h" Jul 15 12:06:38.075474 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:38.075444 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l8c6h" Jul 15 12:06:38.322271 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:38.322248 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l8c6h"] Jul 15 12:06:38.324666 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:06:38.324640 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod825aeecb_52ad_4bd5_bba0_b4438705430d.slice/crio-6d9e62cb367f89bd6907c46173ea2111605732b418f41b2e57aba4398cc52770 WatchSource:0}: Error finding container 6d9e62cb367f89bd6907c46173ea2111605732b418f41b2e57aba4398cc52770: Status 404 returned error can't find the container with id 6d9e62cb367f89bd6907c46173ea2111605732b418f41b2e57aba4398cc52770 Jul 15 12:06:38.359001 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:38.358975 2587 generic.go:356] "Generic (PLEG): container finished" podID="4e4e75e4-4576-4bab-922e-ffe34d7890ff" containerID="d5c45bd89da0b3c51aff86cd270ab9f07eed9db0d8e854d548d6439daf1746b1" exitCode=0 Jul 15 12:06:38.359116 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:38.359050 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qspjl" event={"ID":"4e4e75e4-4576-4bab-922e-ffe34d7890ff","Type":"ContainerDied","Data":"d5c45bd89da0b3c51aff86cd270ab9f07eed9db0d8e854d548d6439daf1746b1"} Jul 15 12:06:38.359116 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:38.359103 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qspjl" event={"ID":"4e4e75e4-4576-4bab-922e-ffe34d7890ff","Type":"ContainerStarted","Data":"2370683744dc95443571ad2a23d9d078558c702180b868975c9c7c95bbb67a7a"} Jul 15 12:06:38.360669 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:38.360649 2587 generic.go:356] "Generic (PLEG): container finished" podID="70f2106a-f040-4eb3-ae55-4c46d899932b" containerID="bf1607d26cd10e87fc92490e61fcf15151f5ae7ccbcbc67d2d4aef817150086d" exitCode=0 Jul 15 12:06:38.360768 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:38.360747 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8khc" event={"ID":"70f2106a-f040-4eb3-ae55-4c46d899932b","Type":"ContainerDied","Data":"bf1607d26cd10e87fc92490e61fcf15151f5ae7ccbcbc67d2d4aef817150086d"} Jul 15 12:06:38.360831 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:38.360794 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8khc" event={"ID":"70f2106a-f040-4eb3-ae55-4c46d899932b","Type":"ContainerStarted","Data":"d519b31a1ebdfde32142157cdb319d42eea88249c121005e6238dfe4186ac182"} Jul 15 12:06:38.362563 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:38.362539 2587 generic.go:356] "Generic (PLEG): container finished" podID="760fbdcd-8e40-4d90-8d93-7402a4aafee6" containerID="95e177f9e0645e99da3edce71c697566e88b50c0ea6d669593ccb24a2ea8d89a" exitCode=0 Jul 15 12:06:38.362670 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:38.362609 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tcccd" event={"ID":"760fbdcd-8e40-4d90-8d93-7402a4aafee6","Type":"ContainerDied","Data":"95e177f9e0645e99da3edce71c697566e88b50c0ea6d669593ccb24a2ea8d89a"} Jul 15 12:06:38.363840 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:38.363817 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l8c6h" event={"ID":"825aeecb-52ad-4bd5-bba0-b4438705430d","Type":"ContainerStarted","Data":"6d9e62cb367f89bd6907c46173ea2111605732b418f41b2e57aba4398cc52770"} Jul 15 12:06:39.371718 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:39.371679 2587 generic.go:356] "Generic (PLEG): container finished" podID="760fbdcd-8e40-4d90-8d93-7402a4aafee6" containerID="be3ec2f68a3b015299065a0f3c670440672f658890c381fcab52c9ab6cd371eb" exitCode=0 Jul 15 12:06:39.372154 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:39.371770 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tcccd" event={"ID":"760fbdcd-8e40-4d90-8d93-7402a4aafee6","Type":"ContainerDied","Data":"be3ec2f68a3b015299065a0f3c670440672f658890c381fcab52c9ab6cd371eb"} Jul 15 12:06:39.373259 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:39.373239 2587 generic.go:356] "Generic (PLEG): container finished" podID="825aeecb-52ad-4bd5-bba0-b4438705430d" containerID="d3c09f189096f86c6e7281cacd629a912403843c655d4db7afbab4cfcf8545b9" exitCode=0 Jul 15 12:06:39.373348 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:39.373326 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l8c6h" event={"ID":"825aeecb-52ad-4bd5-bba0-b4438705430d","Type":"ContainerDied","Data":"d3c09f189096f86c6e7281cacd629a912403843c655d4db7afbab4cfcf8545b9"} Jul 15 12:06:39.375277 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:39.375257 2587 generic.go:356] "Generic (PLEG): container finished" podID="4e4e75e4-4576-4bab-922e-ffe34d7890ff" containerID="eeda6b14b0014634204354b9433fc7b7fe0b5dca0874d5c6df655bf7b7de3bfa" exitCode=0 Jul 15 12:06:39.375433 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:39.375325 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qspjl" event={"ID":"4e4e75e4-4576-4bab-922e-ffe34d7890ff","Type":"ContainerDied","Data":"eeda6b14b0014634204354b9433fc7b7fe0b5dca0874d5c6df655bf7b7de3bfa"} Jul 15 12:06:39.377427 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:39.377405 2587 generic.go:356] "Generic (PLEG): container finished" podID="70f2106a-f040-4eb3-ae55-4c46d899932b" containerID="61560be0a442e77afebc2eea01ad5740ed92c4c549c9f2535ca015e77b53272f" exitCode=0 Jul 15 12:06:39.377521 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:39.377482 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8khc" event={"ID":"70f2106a-f040-4eb3-ae55-4c46d899932b","Type":"ContainerDied","Data":"61560be0a442e77afebc2eea01ad5740ed92c4c549c9f2535ca015e77b53272f"} Jul 15 12:06:40.183158 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:06:40.183124 2587 helpers.go:245] readString: Failed to read "/sys/fs/cgroup/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod825aeecb_52ad_4bd5_bba0_b4438705430d.slice/crio-9417024ee1e43bf3684193f42b67bcf0067202ae09fb419bf9d23750c521fbac.scope/pids.max": read /sys/fs/cgroup/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod825aeecb_52ad_4bd5_bba0_b4438705430d.slice/crio-9417024ee1e43bf3684193f42b67bcf0067202ae09fb419bf9d23750c521fbac.scope/pids.max: no such device Jul 15 12:06:40.385013 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:40.384917 2587 generic.go:356] "Generic (PLEG): container finished" podID="70f2106a-f040-4eb3-ae55-4c46d899932b" containerID="670eca1b67b86aa7bdcdf5d6653f5fd2941732a052cce1a95c54eac4d892e8d4" exitCode=0 Jul 15 12:06:40.385013 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:40.384998 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8khc" event={"ID":"70f2106a-f040-4eb3-ae55-4c46d899932b","Type":"ContainerDied","Data":"670eca1b67b86aa7bdcdf5d6653f5fd2941732a052cce1a95c54eac4d892e8d4"} Jul 15 12:06:40.386801 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:40.386775 2587 generic.go:356] "Generic (PLEG): container finished" podID="825aeecb-52ad-4bd5-bba0-b4438705430d" containerID="9417024ee1e43bf3684193f42b67bcf0067202ae09fb419bf9d23750c521fbac" exitCode=0 Jul 15 12:06:40.386930 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:40.386857 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l8c6h" event={"ID":"825aeecb-52ad-4bd5-bba0-b4438705430d","Type":"ContainerDied","Data":"9417024ee1e43bf3684193f42b67bcf0067202ae09fb419bf9d23750c521fbac"} Jul 15 12:06:40.388987 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:40.388963 2587 generic.go:356] "Generic (PLEG): container finished" podID="4e4e75e4-4576-4bab-922e-ffe34d7890ff" containerID="5378da9c563bc3e11f083bca21ff5b74d409a9b62a03177e9a629c537f996e4d" exitCode=0 Jul 15 12:06:40.389085 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:40.389017 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qspjl" event={"ID":"4e4e75e4-4576-4bab-922e-ffe34d7890ff","Type":"ContainerDied","Data":"5378da9c563bc3e11f083bca21ff5b74d409a9b62a03177e9a629c537f996e4d"} Jul 15 12:06:40.578702 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:40.578674 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tcccd" Jul 15 12:06:40.627429 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:40.627394 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/760fbdcd-8e40-4d90-8d93-7402a4aafee6-bundle\") pod \"760fbdcd-8e40-4d90-8d93-7402a4aafee6\" (UID: \"760fbdcd-8e40-4d90-8d93-7402a4aafee6\") " Jul 15 12:06:40.627612 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:40.627463 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/760fbdcd-8e40-4d90-8d93-7402a4aafee6-util\") pod \"760fbdcd-8e40-4d90-8d93-7402a4aafee6\" (UID: \"760fbdcd-8e40-4d90-8d93-7402a4aafee6\") " Jul 15 12:06:40.627612 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:40.627512 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nnknr\" (UniqueName: \"kubernetes.io/projected/760fbdcd-8e40-4d90-8d93-7402a4aafee6-kube-api-access-nnknr\") pod \"760fbdcd-8e40-4d90-8d93-7402a4aafee6\" (UID: \"760fbdcd-8e40-4d90-8d93-7402a4aafee6\") " Jul 15 12:06:40.627869 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:40.627844 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/760fbdcd-8e40-4d90-8d93-7402a4aafee6-bundle" (OuterVolumeSpecName: "bundle") pod "760fbdcd-8e40-4d90-8d93-7402a4aafee6" (UID: "760fbdcd-8e40-4d90-8d93-7402a4aafee6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:06:40.629546 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:40.629523 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/760fbdcd-8e40-4d90-8d93-7402a4aafee6-kube-api-access-nnknr" (OuterVolumeSpecName: "kube-api-access-nnknr") pod "760fbdcd-8e40-4d90-8d93-7402a4aafee6" (UID: "760fbdcd-8e40-4d90-8d93-7402a4aafee6"). InnerVolumeSpecName "kube-api-access-nnknr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:06:40.632481 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:40.632457 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/760fbdcd-8e40-4d90-8d93-7402a4aafee6-util" (OuterVolumeSpecName: "util") pod "760fbdcd-8e40-4d90-8d93-7402a4aafee6" (UID: "760fbdcd-8e40-4d90-8d93-7402a4aafee6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:06:40.728835 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:40.728744 2587 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/760fbdcd-8e40-4d90-8d93-7402a4aafee6-util\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:06:40.728835 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:40.728776 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nnknr\" (UniqueName: \"kubernetes.io/projected/760fbdcd-8e40-4d90-8d93-7402a4aafee6-kube-api-access-nnknr\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:06:40.728835 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:40.728791 2587 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/760fbdcd-8e40-4d90-8d93-7402a4aafee6-bundle\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:06:41.396517 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:41.396485 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tcccd" Jul 15 12:06:41.396968 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:41.396480 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tcccd" event={"ID":"760fbdcd-8e40-4d90-8d93-7402a4aafee6","Type":"ContainerDied","Data":"e80df9b497e377b6309f8e3d174c9c70839a605cc8ce5d46545febab00706528"} Jul 15 12:06:41.396968 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:41.396613 2587 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e80df9b497e377b6309f8e3d174c9c70839a605cc8ce5d46545febab00706528" Jul 15 12:06:41.399019 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:41.398991 2587 generic.go:356] "Generic (PLEG): container finished" podID="825aeecb-52ad-4bd5-bba0-b4438705430d" containerID="8978c8c8a98394c411a355fda9815ea64acf5d9e163bcb9c3e213a8cb74b2049" exitCode=0 Jul 15 12:06:41.399135 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:41.399058 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l8c6h" event={"ID":"825aeecb-52ad-4bd5-bba0-b4438705430d","Type":"ContainerDied","Data":"8978c8c8a98394c411a355fda9815ea64acf5d9e163bcb9c3e213a8cb74b2049"} Jul 15 12:06:41.558699 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:41.558677 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8khc" Jul 15 12:06:41.561659 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:41.561636 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qspjl" Jul 15 12:06:41.635647 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:41.635618 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/70f2106a-f040-4eb3-ae55-4c46d899932b-bundle\") pod \"70f2106a-f040-4eb3-ae55-4c46d899932b\" (UID: \"70f2106a-f040-4eb3-ae55-4c46d899932b\") " Jul 15 12:06:41.635838 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:41.635686 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gkv25\" (UniqueName: \"kubernetes.io/projected/4e4e75e4-4576-4bab-922e-ffe34d7890ff-kube-api-access-gkv25\") pod \"4e4e75e4-4576-4bab-922e-ffe34d7890ff\" (UID: \"4e4e75e4-4576-4bab-922e-ffe34d7890ff\") " Jul 15 12:06:41.635838 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:41.635721 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kkfqj\" (UniqueName: \"kubernetes.io/projected/70f2106a-f040-4eb3-ae55-4c46d899932b-kube-api-access-kkfqj\") pod \"70f2106a-f040-4eb3-ae55-4c46d899932b\" (UID: \"70f2106a-f040-4eb3-ae55-4c46d899932b\") " Jul 15 12:06:41.635838 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:41.635815 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4e4e75e4-4576-4bab-922e-ffe34d7890ff-bundle\") pod \"4e4e75e4-4576-4bab-922e-ffe34d7890ff\" (UID: \"4e4e75e4-4576-4bab-922e-ffe34d7890ff\") " Jul 15 12:06:41.635981 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:41.635847 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/70f2106a-f040-4eb3-ae55-4c46d899932b-util\") pod \"70f2106a-f040-4eb3-ae55-4c46d899932b\" (UID: \"70f2106a-f040-4eb3-ae55-4c46d899932b\") " Jul 15 12:06:41.635981 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:41.635871 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4e4e75e4-4576-4bab-922e-ffe34d7890ff-util\") pod \"4e4e75e4-4576-4bab-922e-ffe34d7890ff\" (UID: \"4e4e75e4-4576-4bab-922e-ffe34d7890ff\") " Jul 15 12:06:41.636500 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:41.636465 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e4e75e4-4576-4bab-922e-ffe34d7890ff-bundle" (OuterVolumeSpecName: "bundle") pod "4e4e75e4-4576-4bab-922e-ffe34d7890ff" (UID: "4e4e75e4-4576-4bab-922e-ffe34d7890ff"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:06:41.636713 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:41.636687 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70f2106a-f040-4eb3-ae55-4c46d899932b-bundle" (OuterVolumeSpecName: "bundle") pod "70f2106a-f040-4eb3-ae55-4c46d899932b" (UID: "70f2106a-f040-4eb3-ae55-4c46d899932b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:06:41.638421 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:41.638390 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e4e75e4-4576-4bab-922e-ffe34d7890ff-kube-api-access-gkv25" (OuterVolumeSpecName: "kube-api-access-gkv25") pod "4e4e75e4-4576-4bab-922e-ffe34d7890ff" (UID: "4e4e75e4-4576-4bab-922e-ffe34d7890ff"). InnerVolumeSpecName "kube-api-access-gkv25". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:06:41.638522 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:41.638470 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70f2106a-f040-4eb3-ae55-4c46d899932b-kube-api-access-kkfqj" (OuterVolumeSpecName: "kube-api-access-kkfqj") pod "70f2106a-f040-4eb3-ae55-4c46d899932b" (UID: "70f2106a-f040-4eb3-ae55-4c46d899932b"). InnerVolumeSpecName "kube-api-access-kkfqj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:06:41.641813 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:41.641792 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70f2106a-f040-4eb3-ae55-4c46d899932b-util" (OuterVolumeSpecName: "util") pod "70f2106a-f040-4eb3-ae55-4c46d899932b" (UID: "70f2106a-f040-4eb3-ae55-4c46d899932b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:06:41.641884 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:41.641860 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e4e75e4-4576-4bab-922e-ffe34d7890ff-util" (OuterVolumeSpecName: "util") pod "4e4e75e4-4576-4bab-922e-ffe34d7890ff" (UID: "4e4e75e4-4576-4bab-922e-ffe34d7890ff"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:06:41.736553 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:41.736463 2587 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/70f2106a-f040-4eb3-ae55-4c46d899932b-bundle\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:06:41.736553 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:41.736497 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gkv25\" (UniqueName: \"kubernetes.io/projected/4e4e75e4-4576-4bab-922e-ffe34d7890ff-kube-api-access-gkv25\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:06:41.736553 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:41.736507 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kkfqj\" (UniqueName: \"kubernetes.io/projected/70f2106a-f040-4eb3-ae55-4c46d899932b-kube-api-access-kkfqj\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:06:41.736553 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:41.736519 2587 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4e4e75e4-4576-4bab-922e-ffe34d7890ff-bundle\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:06:41.736553 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:41.736528 2587 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/70f2106a-f040-4eb3-ae55-4c46d899932b-util\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:06:41.736553 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:41.736536 2587 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4e4e75e4-4576-4bab-922e-ffe34d7890ff-util\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:06:42.411436 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:42.411403 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8khc" Jul 15 12:06:42.411912 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:42.411441 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8khc" event={"ID":"70f2106a-f040-4eb3-ae55-4c46d899932b","Type":"ContainerDied","Data":"d519b31a1ebdfde32142157cdb319d42eea88249c121005e6238dfe4186ac182"} Jul 15 12:06:42.411912 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:42.411494 2587 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d519b31a1ebdfde32142157cdb319d42eea88249c121005e6238dfe4186ac182" Jul 15 12:06:42.413368 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:42.413348 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qspjl" Jul 15 12:06:42.413473 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:42.413381 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qspjl" event={"ID":"4e4e75e4-4576-4bab-922e-ffe34d7890ff","Type":"ContainerDied","Data":"2370683744dc95443571ad2a23d9d078558c702180b868975c9c7c95bbb67a7a"} Jul 15 12:06:42.413473 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:42.413417 2587 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2370683744dc95443571ad2a23d9d078558c702180b868975c9c7c95bbb67a7a" Jul 15 12:06:42.544285 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:42.544259 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l8c6h" Jul 15 12:06:42.642501 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:42.642462 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-msd8m\" (UniqueName: \"kubernetes.io/projected/825aeecb-52ad-4bd5-bba0-b4438705430d-kube-api-access-msd8m\") pod \"825aeecb-52ad-4bd5-bba0-b4438705430d\" (UID: \"825aeecb-52ad-4bd5-bba0-b4438705430d\") " Jul 15 12:06:42.642678 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:42.642550 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/825aeecb-52ad-4bd5-bba0-b4438705430d-bundle\") pod \"825aeecb-52ad-4bd5-bba0-b4438705430d\" (UID: \"825aeecb-52ad-4bd5-bba0-b4438705430d\") " Jul 15 12:06:42.642678 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:42.642622 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/825aeecb-52ad-4bd5-bba0-b4438705430d-util\") pod \"825aeecb-52ad-4bd5-bba0-b4438705430d\" (UID: \"825aeecb-52ad-4bd5-bba0-b4438705430d\") " Jul 15 12:06:42.643119 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:42.643092 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/825aeecb-52ad-4bd5-bba0-b4438705430d-bundle" (OuterVolumeSpecName: "bundle") pod "825aeecb-52ad-4bd5-bba0-b4438705430d" (UID: "825aeecb-52ad-4bd5-bba0-b4438705430d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:06:42.644636 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:42.644612 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/825aeecb-52ad-4bd5-bba0-b4438705430d-kube-api-access-msd8m" (OuterVolumeSpecName: "kube-api-access-msd8m") pod "825aeecb-52ad-4bd5-bba0-b4438705430d" (UID: "825aeecb-52ad-4bd5-bba0-b4438705430d"). InnerVolumeSpecName "kube-api-access-msd8m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:06:42.647898 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:42.647866 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/825aeecb-52ad-4bd5-bba0-b4438705430d-util" (OuterVolumeSpecName: "util") pod "825aeecb-52ad-4bd5-bba0-b4438705430d" (UID: "825aeecb-52ad-4bd5-bba0-b4438705430d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:06:42.743820 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:42.743714 2587 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/825aeecb-52ad-4bd5-bba0-b4438705430d-util\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:06:42.743820 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:42.743765 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-msd8m\" (UniqueName: \"kubernetes.io/projected/825aeecb-52ad-4bd5-bba0-b4438705430d-kube-api-access-msd8m\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:06:42.743820 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:42.743776 2587 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/825aeecb-52ad-4bd5-bba0-b4438705430d-bundle\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:06:43.426930 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:43.426872 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l8c6h" event={"ID":"825aeecb-52ad-4bd5-bba0-b4438705430d","Type":"ContainerDied","Data":"6d9e62cb367f89bd6907c46173ea2111605732b418f41b2e57aba4398cc52770"} Jul 15 12:06:43.427383 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:43.426963 2587 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6d9e62cb367f89bd6907c46173ea2111605732b418f41b2e57aba4398cc52770" Jul 15 12:06:43.427383 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:43.426873 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l8c6h" Jul 15 12:06:53.364795 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:53.364757 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-jscrc"] Jul 15 12:06:53.365377 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:53.365356 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e4e75e4-4576-4bab-922e-ffe34d7890ff" containerName="util" Jul 15 12:06:53.365426 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:53.365384 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e4e75e4-4576-4bab-922e-ffe34d7890ff" containerName="util" Jul 15 12:06:53.365426 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:53.365405 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="760fbdcd-8e40-4d90-8d93-7402a4aafee6" containerName="util" Jul 15 12:06:53.365426 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:53.365414 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="760fbdcd-8e40-4d90-8d93-7402a4aafee6" containerName="util" Jul 15 12:06:53.365517 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:53.365427 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e4e75e4-4576-4bab-922e-ffe34d7890ff" containerName="pull" Jul 15 12:06:53.365517 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:53.365436 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e4e75e4-4576-4bab-922e-ffe34d7890ff" containerName="pull" Jul 15 12:06:53.365517 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:53.365445 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e4e75e4-4576-4bab-922e-ffe34d7890ff" containerName="extract" Jul 15 12:06:53.365517 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:53.365454 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e4e75e4-4576-4bab-922e-ffe34d7890ff" containerName="extract" Jul 15 12:06:53.365517 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:53.365464 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="825aeecb-52ad-4bd5-bba0-b4438705430d" containerName="util" Jul 15 12:06:53.365517 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:53.365472 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="825aeecb-52ad-4bd5-bba0-b4438705430d" containerName="util" Jul 15 12:06:53.365517 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:53.365480 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="825aeecb-52ad-4bd5-bba0-b4438705430d" containerName="extract" Jul 15 12:06:53.365517 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:53.365487 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="825aeecb-52ad-4bd5-bba0-b4438705430d" containerName="extract" Jul 15 12:06:53.365517 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:53.365506 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70f2106a-f040-4eb3-ae55-4c46d899932b" containerName="pull" Jul 15 12:06:53.365517 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:53.365514 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="70f2106a-f040-4eb3-ae55-4c46d899932b" containerName="pull" Jul 15 12:06:53.365825 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:53.365531 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70f2106a-f040-4eb3-ae55-4c46d899932b" containerName="util" Jul 15 12:06:53.365825 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:53.365539 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="70f2106a-f040-4eb3-ae55-4c46d899932b" containerName="util" Jul 15 12:06:53.365825 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:53.365550 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70f2106a-f040-4eb3-ae55-4c46d899932b" containerName="extract" Jul 15 12:06:53.365825 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:53.365558 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="70f2106a-f040-4eb3-ae55-4c46d899932b" containerName="extract" Jul 15 12:06:53.365825 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:53.365570 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="825aeecb-52ad-4bd5-bba0-b4438705430d" containerName="pull" Jul 15 12:06:53.365825 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:53.365578 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="825aeecb-52ad-4bd5-bba0-b4438705430d" containerName="pull" Jul 15 12:06:53.365825 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:53.365598 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="760fbdcd-8e40-4d90-8d93-7402a4aafee6" containerName="pull" Jul 15 12:06:53.365825 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:53.365606 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="760fbdcd-8e40-4d90-8d93-7402a4aafee6" containerName="pull" Jul 15 12:06:53.365825 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:53.365615 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="760fbdcd-8e40-4d90-8d93-7402a4aafee6" containerName="extract" Jul 15 12:06:53.365825 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:53.365623 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="760fbdcd-8e40-4d90-8d93-7402a4aafee6" containerName="extract" Jul 15 12:06:53.365825 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:53.365715 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="70f2106a-f040-4eb3-ae55-4c46d899932b" containerName="extract" Jul 15 12:06:53.365825 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:53.365751 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="760fbdcd-8e40-4d90-8d93-7402a4aafee6" containerName="extract" Jul 15 12:06:53.365825 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:53.365763 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e4e75e4-4576-4bab-922e-ffe34d7890ff" containerName="extract" Jul 15 12:06:53.365825 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:53.365771 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="825aeecb-52ad-4bd5-bba0-b4438705430d" containerName="extract" Jul 15 12:06:53.372941 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:53.372913 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-jscrc" Jul 15 12:06:53.376014 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:53.375992 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 15 12:06:53.376152 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:53.376029 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 15 12:06:53.377234 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:53.377215 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-jkm4v\"" Jul 15 12:06:53.383236 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:53.383214 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-jscrc"] Jul 15 12:06:53.543390 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:53.543344 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58zw6\" (UniqueName: \"kubernetes.io/projected/51f92a56-4f2d-4608-8c65-6958e300c98b-kube-api-access-58zw6\") pod \"authorino-operator-75cc8d5cb7-jscrc\" (UID: \"51f92a56-4f2d-4608-8c65-6958e300c98b\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-jscrc" Jul 15 12:06:53.644045 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:53.643952 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58zw6\" (UniqueName: \"kubernetes.io/projected/51f92a56-4f2d-4608-8c65-6958e300c98b-kube-api-access-58zw6\") pod \"authorino-operator-75cc8d5cb7-jscrc\" (UID: \"51f92a56-4f2d-4608-8c65-6958e300c98b\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-jscrc" Jul 15 12:06:53.656879 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:53.656855 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-58zw6\" (UniqueName: \"kubernetes.io/projected/51f92a56-4f2d-4608-8c65-6958e300c98b-kube-api-access-58zw6\") pod \"authorino-operator-75cc8d5cb7-jscrc\" (UID: \"51f92a56-4f2d-4608-8c65-6958e300c98b\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-jscrc" Jul 15 12:06:53.685833 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:53.685799 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-jscrc" Jul 15 12:06:53.828853 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:53.828828 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-jscrc"] Jul 15 12:06:53.831056 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:06:53.831031 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51f92a56_4f2d_4608_8c65_6958e300c98b.slice/crio-07d6c19d1bb97174f0b3700aa21205796a6710bdb630493cba477208d39e33a6 WatchSource:0}: Error finding container 07d6c19d1bb97174f0b3700aa21205796a6710bdb630493cba477208d39e33a6: Status 404 returned error can't find the container with id 07d6c19d1bb97174f0b3700aa21205796a6710bdb630493cba477208d39e33a6 Jul 15 12:06:54.486930 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:54.486891 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-jscrc" event={"ID":"51f92a56-4f2d-4608-8c65-6958e300c98b","Type":"ContainerStarted","Data":"07d6c19d1bb97174f0b3700aa21205796a6710bdb630493cba477208d39e33a6"} Jul 15 12:06:57.505852 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:57.505808 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-jscrc" event={"ID":"51f92a56-4f2d-4608-8c65-6958e300c98b","Type":"ContainerStarted","Data":"fac607bbd95b2e0ee1ce998bcc89ae0cc04c8d2974cfdec2a82f212ee1293cdf"} Jul 15 12:06:57.506262 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:57.505894 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-jscrc" Jul 15 12:06:57.529009 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:06:57.528961 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-jscrc" podStartSLOduration=1.418921493 podStartE2EDuration="4.528947979s" totalImagesPullingTime="3.110026486s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:06:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:06:57.526008111 +0000 UTC m=+488.983385041" watchObservedRunningTime="2026-07-15 12:06:57.528947979 +0000 UTC m=+488.986324904" Jul 15 12:07:03.322523 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:03.322486 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l6pgl"] Jul 15 12:07:03.328273 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:03.328253 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l6pgl" Jul 15 12:07:03.331030 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:03.331000 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-8f2ct\"" Jul 15 12:07:03.339248 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:03.339223 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l6pgl"] Jul 15 12:07:03.428590 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:03.428557 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/52c7db17-8ce9-4416-85dc-28dc7f5eb50e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-l6pgl\" (UID: \"52c7db17-8ce9-4416-85dc-28dc7f5eb50e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l6pgl" Jul 15 12:07:03.428590 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:03.428602 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k75v7\" (UniqueName: \"kubernetes.io/projected/52c7db17-8ce9-4416-85dc-28dc7f5eb50e-kube-api-access-k75v7\") pod \"kuadrant-operator-controller-manager-8684c9ff49-l6pgl\" (UID: \"52c7db17-8ce9-4416-85dc-28dc7f5eb50e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l6pgl" Jul 15 12:07:03.529985 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:03.529951 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/52c7db17-8ce9-4416-85dc-28dc7f5eb50e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-l6pgl\" (UID: \"52c7db17-8ce9-4416-85dc-28dc7f5eb50e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l6pgl" Jul 15 12:07:03.530139 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:03.529996 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k75v7\" (UniqueName: \"kubernetes.io/projected/52c7db17-8ce9-4416-85dc-28dc7f5eb50e-kube-api-access-k75v7\") pod \"kuadrant-operator-controller-manager-8684c9ff49-l6pgl\" (UID: \"52c7db17-8ce9-4416-85dc-28dc7f5eb50e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l6pgl" Jul 15 12:07:03.530325 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:03.530305 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/52c7db17-8ce9-4416-85dc-28dc7f5eb50e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-l6pgl\" (UID: \"52c7db17-8ce9-4416-85dc-28dc7f5eb50e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l6pgl" Jul 15 12:07:03.544753 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:03.544706 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k75v7\" (UniqueName: \"kubernetes.io/projected/52c7db17-8ce9-4416-85dc-28dc7f5eb50e-kube-api-access-k75v7\") pod \"kuadrant-operator-controller-manager-8684c9ff49-l6pgl\" (UID: \"52c7db17-8ce9-4416-85dc-28dc7f5eb50e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l6pgl" Jul 15 12:07:03.639355 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:03.639268 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l6pgl" Jul 15 12:07:03.776668 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:03.776644 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l6pgl"] Jul 15 12:07:03.778839 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:07:03.778806 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52c7db17_8ce9_4416_85dc_28dc7f5eb50e.slice/crio-5e80abd5cb14ad662eb31b6a5316da4716db09919fd9e1aa270810e30ae5a370 WatchSource:0}: Error finding container 5e80abd5cb14ad662eb31b6a5316da4716db09919fd9e1aa270810e30ae5a370: Status 404 returned error can't find the container with id 5e80abd5cb14ad662eb31b6a5316da4716db09919fd9e1aa270810e30ae5a370 Jul 15 12:07:04.544040 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:04.544003 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l6pgl" event={"ID":"52c7db17-8ce9-4416-85dc-28dc7f5eb50e","Type":"ContainerStarted","Data":"5e80abd5cb14ad662eb31b6a5316da4716db09919fd9e1aa270810e30ae5a370"} Jul 15 12:07:08.514277 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:08.514241 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-jscrc" Jul 15 12:07:11.585448 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:11.585415 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l6pgl" event={"ID":"52c7db17-8ce9-4416-85dc-28dc7f5eb50e","Type":"ContainerStarted","Data":"04b056fec65fbeed080008f2a60f316ea6077491d08eac265f3a5b68680401af"} Jul 15 12:07:11.585861 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:11.585502 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l6pgl" Jul 15 12:07:11.607929 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:11.607879 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l6pgl" podStartSLOduration=1.701356041 podStartE2EDuration="8.607864058s" totalImagesPullingTime="6.906508017s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:07:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:07:11.606929871 +0000 UTC m=+503.064306815" watchObservedRunningTime="2026-07-15 12:07:11.607864058 +0000 UTC m=+503.065240982" Jul 15 12:07:22.593824 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:22.593794 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l6pgl" Jul 15 12:07:23.519815 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:23.519777 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-zmq8t"] Jul 15 12:07:23.525035 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:23.525016 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-zmq8t" Jul 15 12:07:23.537375 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:23.537344 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-zmq8t"] Jul 15 12:07:23.603324 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:23.603283 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/43ec77dc-4c47-4d08-b959-f9060435d3eb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-zmq8t\" (UID: \"43ec77dc-4c47-4d08-b959-f9060435d3eb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-zmq8t" Jul 15 12:07:23.603717 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:23.603337 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwg5z\" (UniqueName: \"kubernetes.io/projected/43ec77dc-4c47-4d08-b959-f9060435d3eb-kube-api-access-zwg5z\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-zmq8t\" (UID: \"43ec77dc-4c47-4d08-b959-f9060435d3eb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-zmq8t" Jul 15 12:07:23.704399 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:23.704362 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwg5z\" (UniqueName: \"kubernetes.io/projected/43ec77dc-4c47-4d08-b959-f9060435d3eb-kube-api-access-zwg5z\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-zmq8t\" (UID: \"43ec77dc-4c47-4d08-b959-f9060435d3eb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-zmq8t" Jul 15 12:07:23.704579 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:23.704512 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/43ec77dc-4c47-4d08-b959-f9060435d3eb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-zmq8t\" (UID: \"43ec77dc-4c47-4d08-b959-f9060435d3eb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-zmq8t" Jul 15 12:07:23.704908 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:23.704888 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/43ec77dc-4c47-4d08-b959-f9060435d3eb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-zmq8t\" (UID: \"43ec77dc-4c47-4d08-b959-f9060435d3eb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-zmq8t" Jul 15 12:07:23.715547 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:23.715517 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwg5z\" (UniqueName: \"kubernetes.io/projected/43ec77dc-4c47-4d08-b959-f9060435d3eb-kube-api-access-zwg5z\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-zmq8t\" (UID: \"43ec77dc-4c47-4d08-b959-f9060435d3eb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-zmq8t" Jul 15 12:07:23.835717 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:23.835676 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-zmq8t" Jul 15 12:07:23.981823 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:23.981796 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-zmq8t"] Jul 15 12:07:23.984902 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:07:23.984872 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43ec77dc_4c47_4d08_b959_f9060435d3eb.slice/crio-e0f322f6006732a188c64411469f9bfc38751d995087f7f1d1ab0a6c02a85111 WatchSource:0}: Error finding container e0f322f6006732a188c64411469f9bfc38751d995087f7f1d1ab0a6c02a85111: Status 404 returned error can't find the container with id e0f322f6006732a188c64411469f9bfc38751d995087f7f1d1ab0a6c02a85111 Jul 15 12:07:24.275204 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:24.275117 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-zmq8t"] Jul 15 12:07:24.280882 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:24.280851 2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-zmq8t"] Jul 15 12:07:24.290238 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:24.290208 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l6pgl"] Jul 15 12:07:24.290443 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:24.290413 2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l6pgl" podUID="52c7db17-8ce9-4416-85dc-28dc7f5eb50e" containerName="manager" containerID="cri-o://04b056fec65fbeed080008f2a60f316ea6077491d08eac265f3a5b68680401af" gracePeriod=2 Jul 15 12:07:24.305293 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:24.305260 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-75qsl"] Jul 15 12:07:24.305846 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:24.305820 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43ec77dc-4c47-4d08-b959-f9060435d3eb" containerName="manager" Jul 15 12:07:24.305846 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:24.305842 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="43ec77dc-4c47-4d08-b959-f9060435d3eb" containerName="manager" Jul 15 12:07:24.306012 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:24.305954 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="43ec77dc-4c47-4d08-b959-f9060435d3eb" containerName="manager" Jul 15 12:07:24.309149 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:24.309127 2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l6pgl"] Jul 15 12:07:24.309246 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:24.309234 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-75qsl" Jul 15 12:07:24.311197 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:07:24.311114 2587 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"52c7db17-8ce9-4416-85dc-28dc7f5eb50e\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-l6pgl\": pods \"kuadrant-operator-controller-manager-8684c9ff49-l6pgl\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l6pgl" Jul 15 12:07:24.313542 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:07:24.313502 2587 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-l6pgl\" is forbidden: User \"system:node:ip-10-0-135-243.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-243.ec2.internal' and this object" podUID="52c7db17-8ce9-4416-85dc-28dc7f5eb50e" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l6pgl" Jul 15 12:07:24.325901 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:24.325858 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9cvv9"] Jul 15 12:07:24.326839 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:24.326811 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52c7db17-8ce9-4416-85dc-28dc7f5eb50e" containerName="manager" Jul 15 12:07:24.326839 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:24.326840 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="52c7db17-8ce9-4416-85dc-28dc7f5eb50e" containerName="manager" Jul 15 12:07:24.327360 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:24.326998 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="52c7db17-8ce9-4416-85dc-28dc7f5eb50e" containerName="manager" Jul 15 12:07:24.331386 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:24.331363 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-75qsl"] Jul 15 12:07:24.331487 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:24.331455 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9cvv9" Jul 15 12:07:24.340907 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:07:24.340866 2587 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-l6pgl\" is forbidden: User \"system:node:ip-10-0-135-243.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-243.ec2.internal' and this object" podUID="52c7db17-8ce9-4416-85dc-28dc7f5eb50e" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l6pgl" Jul 15 12:07:24.343270 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:24.343244 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9cvv9"] Jul 15 12:07:24.410630 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:24.410602 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ae03dcff-e1fb-4b1d-a483-a190d0ca9ee4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9cvv9\" (UID: \"ae03dcff-e1fb-4b1d-a483-a190d0ca9ee4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9cvv9" Jul 15 12:07:24.410801 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:24.410666 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcl4b\" (UniqueName: \"kubernetes.io/projected/ae03dcff-e1fb-4b1d-a483-a190d0ca9ee4-kube-api-access-wcl4b\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9cvv9\" (UID: \"ae03dcff-e1fb-4b1d-a483-a190d0ca9ee4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9cvv9" Jul 15 12:07:24.410801 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:24.410694 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krhfm\" (UniqueName: \"kubernetes.io/projected/740c4fe1-9154-4288-82eb-985723c24d7b-kube-api-access-krhfm\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-75qsl\" (UID: \"740c4fe1-9154-4288-82eb-985723c24d7b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-75qsl" Jul 15 12:07:24.410801 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:24.410721 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/740c4fe1-9154-4288-82eb-985723c24d7b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-75qsl\" (UID: \"740c4fe1-9154-4288-82eb-985723c24d7b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-75qsl" Jul 15 12:07:24.512025 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:24.511989 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ae03dcff-e1fb-4b1d-a483-a190d0ca9ee4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9cvv9\" (UID: \"ae03dcff-e1fb-4b1d-a483-a190d0ca9ee4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9cvv9" Jul 15 12:07:24.512198 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:24.512051 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcl4b\" (UniqueName: \"kubernetes.io/projected/ae03dcff-e1fb-4b1d-a483-a190d0ca9ee4-kube-api-access-wcl4b\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9cvv9\" (UID: \"ae03dcff-e1fb-4b1d-a483-a190d0ca9ee4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9cvv9" Jul 15 12:07:24.512198 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:24.512083 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krhfm\" (UniqueName: \"kubernetes.io/projected/740c4fe1-9154-4288-82eb-985723c24d7b-kube-api-access-krhfm\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-75qsl\" (UID: \"740c4fe1-9154-4288-82eb-985723c24d7b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-75qsl" Jul 15 12:07:24.512198 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:24.512122 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/740c4fe1-9154-4288-82eb-985723c24d7b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-75qsl\" (UID: \"740c4fe1-9154-4288-82eb-985723c24d7b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-75qsl" Jul 15 12:07:24.512468 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:24.512444 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ae03dcff-e1fb-4b1d-a483-a190d0ca9ee4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9cvv9\" (UID: \"ae03dcff-e1fb-4b1d-a483-a190d0ca9ee4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9cvv9" Jul 15 12:07:24.512513 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:24.512492 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/740c4fe1-9154-4288-82eb-985723c24d7b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-75qsl\" (UID: \"740c4fe1-9154-4288-82eb-985723c24d7b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-75qsl" Jul 15 12:07:24.521544 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:24.521520 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krhfm\" (UniqueName: \"kubernetes.io/projected/740c4fe1-9154-4288-82eb-985723c24d7b-kube-api-access-krhfm\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-75qsl\" (UID: \"740c4fe1-9154-4288-82eb-985723c24d7b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-75qsl" Jul 15 12:07:24.521665 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:24.521589 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcl4b\" (UniqueName: \"kubernetes.io/projected/ae03dcff-e1fb-4b1d-a483-a190d0ca9ee4-kube-api-access-wcl4b\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9cvv9\" (UID: \"ae03dcff-e1fb-4b1d-a483-a190d0ca9ee4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9cvv9" Jul 15 12:07:24.527976 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:24.527915 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l6pgl" Jul 15 12:07:24.530404 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:07:24.530379 2587 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-l6pgl\" is forbidden: User \"system:node:ip-10-0-135-243.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-243.ec2.internal' and this object" podUID="52c7db17-8ce9-4416-85dc-28dc7f5eb50e" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l6pgl" Jul 15 12:07:24.613240 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:24.613207 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/52c7db17-8ce9-4416-85dc-28dc7f5eb50e-extensions-socket-volume\") pod \"52c7db17-8ce9-4416-85dc-28dc7f5eb50e\" (UID: \"52c7db17-8ce9-4416-85dc-28dc7f5eb50e\") " Jul 15 12:07:24.613726 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:24.613309 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k75v7\" (UniqueName: \"kubernetes.io/projected/52c7db17-8ce9-4416-85dc-28dc7f5eb50e-kube-api-access-k75v7\") pod \"52c7db17-8ce9-4416-85dc-28dc7f5eb50e\" (UID: \"52c7db17-8ce9-4416-85dc-28dc7f5eb50e\") " Jul 15 12:07:24.613888 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:24.613862 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52c7db17-8ce9-4416-85dc-28dc7f5eb50e-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "52c7db17-8ce9-4416-85dc-28dc7f5eb50e" (UID: "52c7db17-8ce9-4416-85dc-28dc7f5eb50e"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:07:24.615365 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:24.615345 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52c7db17-8ce9-4416-85dc-28dc7f5eb50e-kube-api-access-k75v7" (OuterVolumeSpecName: "kube-api-access-k75v7") pod "52c7db17-8ce9-4416-85dc-28dc7f5eb50e" (UID: "52c7db17-8ce9-4416-85dc-28dc7f5eb50e"). InnerVolumeSpecName "kube-api-access-k75v7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:07:24.655970 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:24.655936 2587 generic.go:356] "Generic (PLEG): container finished" podID="52c7db17-8ce9-4416-85dc-28dc7f5eb50e" containerID="04b056fec65fbeed080008f2a60f316ea6077491d08eac265f3a5b68680401af" exitCode=0 Jul 15 12:07:24.656140 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:24.655984 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l6pgl" Jul 15 12:07:24.656140 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:24.656022 2587 scope.go:117] "RemoveContainer" containerID="04b056fec65fbeed080008f2a60f316ea6077491d08eac265f3a5b68680401af" Jul 15 12:07:24.658000 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:07:24.657978 2587 kuberuntime_manager.go:649] "Missing actuated resource record" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-zmq8t" container="manager" Jul 15 12:07:24.658669 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:07:24.658635 2587 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-l6pgl\" is forbidden: User \"system:node:ip-10-0-135-243.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-243.ec2.internal' and this object" podUID="52c7db17-8ce9-4416-85dc-28dc7f5eb50e" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l6pgl" Jul 15 12:07:24.660785 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:24.660760 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-75qsl" Jul 15 12:07:24.661095 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:07:24.661064 2587 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-l6pgl\" is forbidden: User \"system:node:ip-10-0-135-243.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-243.ec2.internal' and this object" podUID="52c7db17-8ce9-4416-85dc-28dc7f5eb50e" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l6pgl" Jul 15 12:07:24.663451 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:07:24.663425 2587 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-zmq8t\" is forbidden: User \"system:node:ip-10-0-135-243.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-243.ec2.internal' and this object" podUID="43ec77dc-4c47-4d08-b959-f9060435d3eb" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-zmq8t" Jul 15 12:07:24.666100 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:24.666079 2587 scope.go:117] "RemoveContainer" containerID="04b056fec65fbeed080008f2a60f316ea6077491d08eac265f3a5b68680401af" Jul 15 12:07:24.666449 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:07:24.666420 2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04b056fec65fbeed080008f2a60f316ea6077491d08eac265f3a5b68680401af\": container with ID starting with 04b056fec65fbeed080008f2a60f316ea6077491d08eac265f3a5b68680401af not found: ID does not exist" containerID="04b056fec65fbeed080008f2a60f316ea6077491d08eac265f3a5b68680401af" Jul 15 12:07:24.666537 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:24.666461 2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04b056fec65fbeed080008f2a60f316ea6077491d08eac265f3a5b68680401af"} err="failed to get container status \"04b056fec65fbeed080008f2a60f316ea6077491d08eac265f3a5b68680401af\": rpc error: code = NotFound desc = could not find container \"04b056fec65fbeed080008f2a60f316ea6077491d08eac265f3a5b68680401af\": container with ID starting with 04b056fec65fbeed080008f2a60f316ea6077491d08eac265f3a5b68680401af not found: ID does not exist" Jul 15 12:07:24.668105 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:24.668085 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9cvv9" Jul 15 12:07:24.669636 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:07:24.669613 2587 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-l6pgl\" is forbidden: User \"system:node:ip-10-0-135-243.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-243.ec2.internal' and this object" podUID="52c7db17-8ce9-4416-85dc-28dc7f5eb50e" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l6pgl" Jul 15 12:07:24.671975 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:07:24.671948 2587 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-zmq8t\" is forbidden: User \"system:node:ip-10-0-135-243.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-243.ec2.internal' and this object" podUID="43ec77dc-4c47-4d08-b959-f9060435d3eb" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-zmq8t" Jul 15 12:07:24.714893 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:24.714859 2587 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/52c7db17-8ce9-4416-85dc-28dc7f5eb50e-extensions-socket-volume\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:07:24.714893 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:24.714889 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k75v7\" (UniqueName: \"kubernetes.io/projected/52c7db17-8ce9-4416-85dc-28dc7f5eb50e-kube-api-access-k75v7\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:07:24.829948 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:07:24.829919 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod740c4fe1_9154_4288_82eb_985723c24d7b.slice/crio-52bd13b298e04d972af73d6f042761daf9def1425949d86bba10bb952692189d WatchSource:0}: Error finding container 52bd13b298e04d972af73d6f042761daf9def1425949d86bba10bb952692189d: Status 404 returned error can't find the container with id 52bd13b298e04d972af73d6f042761daf9def1425949d86bba10bb952692189d Jul 15 12:07:24.832444 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:24.832421 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-75qsl"] Jul 15 12:07:24.855655 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:24.855625 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9cvv9"] Jul 15 12:07:24.859520 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:07:24.859496 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae03dcff_e1fb_4b1d_a483_a190d0ca9ee4.slice/crio-8046037ebee3b44202df676097e3b371bd0cc71e33dc5ea2973f9c3910c89bd4 WatchSource:0}: Error finding container 8046037ebee3b44202df676097e3b371bd0cc71e33dc5ea2973f9c3910c89bd4: Status 404 returned error can't find the container with id 8046037ebee3b44202df676097e3b371bd0cc71e33dc5ea2973f9c3910c89bd4 Jul 15 12:07:25.185937 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:25.185853 2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52c7db17-8ce9-4416-85dc-28dc7f5eb50e" path="/var/lib/kubelet/pods/52c7db17-8ce9-4416-85dc-28dc7f5eb50e/volumes" Jul 15 12:07:25.256106 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:25.256073 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-75cdfcc548-8bjw6"] Jul 15 12:07:25.666959 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:25.666912 2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-zmq8t" podUID="43ec77dc-4c47-4d08-b959-f9060435d3eb" containerName="manager" containerID="cri-o://b534c001f22d7fcda902a98a496fc3a9d79da9bf6fe95fbeb1ec91b95c6a4e41" gracePeriod=2 Jul 15 12:07:25.668557 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:25.668525 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9cvv9" event={"ID":"ae03dcff-e1fb-4b1d-a483-a190d0ca9ee4","Type":"ContainerStarted","Data":"c3ac176039d27217aba7a54b195a00eff3105dc531983b022abfd9b2f40962e3"} Jul 15 12:07:25.668725 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:25.668704 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9cvv9" Jul 15 12:07:25.669028 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:25.668770 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9cvv9" event={"ID":"ae03dcff-e1fb-4b1d-a483-a190d0ca9ee4","Type":"ContainerStarted","Data":"8046037ebee3b44202df676097e3b371bd0cc71e33dc5ea2973f9c3910c89bd4"} Jul 15 12:07:25.670110 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:25.670087 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-75qsl" event={"ID":"740c4fe1-9154-4288-82eb-985723c24d7b","Type":"ContainerStarted","Data":"cb9015ed374972a796f087aa82cc24c481565b1a2f0d2eaa5ce44789c37dff29"} Jul 15 12:07:25.670211 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:25.670128 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-75qsl" event={"ID":"740c4fe1-9154-4288-82eb-985723c24d7b","Type":"ContainerStarted","Data":"52bd13b298e04d972af73d6f042761daf9def1425949d86bba10bb952692189d"} Jul 15 12:07:25.670211 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:25.670166 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-75qsl" Jul 15 12:07:25.710029 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:25.709977 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-75qsl" podStartSLOduration=1.709959933 podStartE2EDuration="1.709959933s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:07:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:07:25.709339276 +0000 UTC m=+517.166716201" watchObservedRunningTime="2026-07-15 12:07:25.709959933 +0000 UTC m=+517.167336859" Jul 15 12:07:25.710489 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:25.710466 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9cvv9" podStartSLOduration=1.710458446 podStartE2EDuration="1.710458446s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:07:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:07:25.689407551 +0000 UTC m=+517.146784475" watchObservedRunningTime="2026-07-15 12:07:25.710458446 +0000 UTC m=+517.167835371" Jul 15 12:07:25.919282 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:25.919226 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-zmq8t" Jul 15 12:07:25.922275 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:07:25.922250 2587 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-zmq8t\" is forbidden: User \"system:node:ip-10-0-135-243.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-243.ec2.internal' and this object" podUID="43ec77dc-4c47-4d08-b959-f9060435d3eb" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-zmq8t" Jul 15 12:07:26.028562 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:26.028520 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zwg5z\" (UniqueName: \"kubernetes.io/projected/43ec77dc-4c47-4d08-b959-f9060435d3eb-kube-api-access-zwg5z\") pod \"43ec77dc-4c47-4d08-b959-f9060435d3eb\" (UID: \"43ec77dc-4c47-4d08-b959-f9060435d3eb\") " Jul 15 12:07:26.028760 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:26.028613 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/43ec77dc-4c47-4d08-b959-f9060435d3eb-extensions-socket-volume\") pod \"43ec77dc-4c47-4d08-b959-f9060435d3eb\" (UID: \"43ec77dc-4c47-4d08-b959-f9060435d3eb\") " Jul 15 12:07:26.028922 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:26.028894 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43ec77dc-4c47-4d08-b959-f9060435d3eb-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "43ec77dc-4c47-4d08-b959-f9060435d3eb" (UID: "43ec77dc-4c47-4d08-b959-f9060435d3eb"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:07:26.030609 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:26.030585 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43ec77dc-4c47-4d08-b959-f9060435d3eb-kube-api-access-zwg5z" (OuterVolumeSpecName: "kube-api-access-zwg5z") pod "43ec77dc-4c47-4d08-b959-f9060435d3eb" (UID: "43ec77dc-4c47-4d08-b959-f9060435d3eb"). InnerVolumeSpecName "kube-api-access-zwg5z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:07:26.129856 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:26.129819 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zwg5z\" (UniqueName: \"kubernetes.io/projected/43ec77dc-4c47-4d08-b959-f9060435d3eb-kube-api-access-zwg5z\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:07:26.129856 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:26.129849 2587 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/43ec77dc-4c47-4d08-b959-f9060435d3eb-extensions-socket-volume\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:07:26.676983 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:26.676942 2587 generic.go:356] "Generic (PLEG): container finished" podID="43ec77dc-4c47-4d08-b959-f9060435d3eb" containerID="b534c001f22d7fcda902a98a496fc3a9d79da9bf6fe95fbeb1ec91b95c6a4e41" exitCode=2 Jul 15 12:07:26.677384 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:26.676993 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-zmq8t" Jul 15 12:07:26.677384 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:26.677041 2587 scope.go:117] "RemoveContainer" containerID="b534c001f22d7fcda902a98a496fc3a9d79da9bf6fe95fbeb1ec91b95c6a4e41" Jul 15 12:07:26.679756 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:07:26.679684 2587 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-zmq8t\" is forbidden: User \"system:node:ip-10-0-135-243.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-243.ec2.internal' and this object" podUID="43ec77dc-4c47-4d08-b959-f9060435d3eb" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-zmq8t" Jul 15 12:07:26.686572 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:26.686553 2587 scope.go:117] "RemoveContainer" containerID="b534c001f22d7fcda902a98a496fc3a9d79da9bf6fe95fbeb1ec91b95c6a4e41" Jul 15 12:07:26.686854 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:07:26.686834 2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b534c001f22d7fcda902a98a496fc3a9d79da9bf6fe95fbeb1ec91b95c6a4e41\": container with ID starting with b534c001f22d7fcda902a98a496fc3a9d79da9bf6fe95fbeb1ec91b95c6a4e41 not found: ID does not exist" containerID="b534c001f22d7fcda902a98a496fc3a9d79da9bf6fe95fbeb1ec91b95c6a4e41" Jul 15 12:07:26.686921 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:26.686861 2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b534c001f22d7fcda902a98a496fc3a9d79da9bf6fe95fbeb1ec91b95c6a4e41"} err="failed to get container status \"b534c001f22d7fcda902a98a496fc3a9d79da9bf6fe95fbeb1ec91b95c6a4e41\": rpc error: code = NotFound desc = could not find container \"b534c001f22d7fcda902a98a496fc3a9d79da9bf6fe95fbeb1ec91b95c6a4e41\": container with ID starting with b534c001f22d7fcda902a98a496fc3a9d79da9bf6fe95fbeb1ec91b95c6a4e41 not found: ID does not exist" Jul 15 12:07:26.687469 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:07:26.687445 2587 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-zmq8t\" is forbidden: User \"system:node:ip-10-0-135-243.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-243.ec2.internal' and this object" podUID="43ec77dc-4c47-4d08-b959-f9060435d3eb" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-zmq8t" Jul 15 12:07:27.185782 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:27.185725 2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43ec77dc-4c47-4d08-b959-f9060435d3eb" path="/var/lib/kubelet/pods/43ec77dc-4c47-4d08-b959-f9060435d3eb/volumes" Jul 15 12:07:36.679661 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:36.679624 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-75qsl" Jul 15 12:07:36.680107 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:36.679695 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9cvv9" Jul 15 12:07:36.760635 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:36.760605 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9cvv9"] Jul 15 12:07:36.760858 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:36.760832 2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9cvv9" podUID="ae03dcff-e1fb-4b1d-a483-a190d0ca9ee4" containerName="manager" containerID="cri-o://c3ac176039d27217aba7a54b195a00eff3105dc531983b022abfd9b2f40962e3" gracePeriod=10 Jul 15 12:07:36.963865 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:36.963830 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pqllv"] Jul 15 12:07:36.967972 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:36.967949 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pqllv" Jul 15 12:07:36.981091 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:36.981066 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pqllv"] Jul 15 12:07:37.010997 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:37.010967 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9cvv9" Jul 15 12:07:37.124052 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:37.124024 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wcl4b\" (UniqueName: \"kubernetes.io/projected/ae03dcff-e1fb-4b1d-a483-a190d0ca9ee4-kube-api-access-wcl4b\") pod \"ae03dcff-e1fb-4b1d-a483-a190d0ca9ee4\" (UID: \"ae03dcff-e1fb-4b1d-a483-a190d0ca9ee4\") " Jul 15 12:07:37.124239 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:37.124098 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ae03dcff-e1fb-4b1d-a483-a190d0ca9ee4-extensions-socket-volume\") pod \"ae03dcff-e1fb-4b1d-a483-a190d0ca9ee4\" (UID: \"ae03dcff-e1fb-4b1d-a483-a190d0ca9ee4\") " Jul 15 12:07:37.124239 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:37.124223 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f415e718-89a6-4841-bc38-2cea520390a0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-pqllv\" (UID: \"f415e718-89a6-4841-bc38-2cea520390a0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pqllv" Jul 15 12:07:37.124332 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:37.124296 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4grmz\" (UniqueName: \"kubernetes.io/projected/f415e718-89a6-4841-bc38-2cea520390a0-kube-api-access-4grmz\") pod \"kuadrant-operator-controller-manager-86878c769f-pqllv\" (UID: \"f415e718-89a6-4841-bc38-2cea520390a0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pqllv" Jul 15 12:07:37.124517 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:37.124496 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae03dcff-e1fb-4b1d-a483-a190d0ca9ee4-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ae03dcff-e1fb-4b1d-a483-a190d0ca9ee4" (UID: "ae03dcff-e1fb-4b1d-a483-a190d0ca9ee4"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:07:37.126292 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:37.126268 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae03dcff-e1fb-4b1d-a483-a190d0ca9ee4-kube-api-access-wcl4b" (OuterVolumeSpecName: "kube-api-access-wcl4b") pod "ae03dcff-e1fb-4b1d-a483-a190d0ca9ee4" (UID: "ae03dcff-e1fb-4b1d-a483-a190d0ca9ee4"). InnerVolumeSpecName "kube-api-access-wcl4b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:07:37.225780 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:37.225687 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f415e718-89a6-4841-bc38-2cea520390a0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-pqllv\" (UID: \"f415e718-89a6-4841-bc38-2cea520390a0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pqllv" Jul 15 12:07:37.225919 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:37.225801 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4grmz\" (UniqueName: \"kubernetes.io/projected/f415e718-89a6-4841-bc38-2cea520390a0-kube-api-access-4grmz\") pod \"kuadrant-operator-controller-manager-86878c769f-pqllv\" (UID: \"f415e718-89a6-4841-bc38-2cea520390a0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pqllv" Jul 15 12:07:37.225919 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:37.225839 2587 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ae03dcff-e1fb-4b1d-a483-a190d0ca9ee4-extensions-socket-volume\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:07:37.225919 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:37.225852 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wcl4b\" (UniqueName: \"kubernetes.io/projected/ae03dcff-e1fb-4b1d-a483-a190d0ca9ee4-kube-api-access-wcl4b\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:07:37.226105 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:37.226085 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f415e718-89a6-4841-bc38-2cea520390a0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-pqllv\" (UID: \"f415e718-89a6-4841-bc38-2cea520390a0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pqllv" Jul 15 12:07:37.254040 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:37.254004 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4grmz\" (UniqueName: \"kubernetes.io/projected/f415e718-89a6-4841-bc38-2cea520390a0-kube-api-access-4grmz\") pod \"kuadrant-operator-controller-manager-86878c769f-pqllv\" (UID: \"f415e718-89a6-4841-bc38-2cea520390a0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pqllv" Jul 15 12:07:37.280974 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:37.280943 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pqllv" Jul 15 12:07:37.429891 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:37.429862 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pqllv"] Jul 15 12:07:37.431205 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:07:37.431180 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf415e718_89a6_4841_bc38_2cea520390a0.slice/crio-49dd24435c86338f507d21116bff1dbd7c0b1b15867ab43123f373f4ed72cc7a WatchSource:0}: Error finding container 49dd24435c86338f507d21116bff1dbd7c0b1b15867ab43123f373f4ed72cc7a: Status 404 returned error can't find the container with id 49dd24435c86338f507d21116bff1dbd7c0b1b15867ab43123f373f4ed72cc7a Jul 15 12:07:37.744273 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:37.744188 2587 generic.go:356] "Generic (PLEG): container finished" podID="ae03dcff-e1fb-4b1d-a483-a190d0ca9ee4" containerID="c3ac176039d27217aba7a54b195a00eff3105dc531983b022abfd9b2f40962e3" exitCode=0 Jul 15 12:07:37.744273 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:37.744234 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9cvv9" Jul 15 12:07:37.744757 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:37.744273 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9cvv9" event={"ID":"ae03dcff-e1fb-4b1d-a483-a190d0ca9ee4","Type":"ContainerDied","Data":"c3ac176039d27217aba7a54b195a00eff3105dc531983b022abfd9b2f40962e3"} Jul 15 12:07:37.744757 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:37.744328 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9cvv9" event={"ID":"ae03dcff-e1fb-4b1d-a483-a190d0ca9ee4","Type":"ContainerDied","Data":"8046037ebee3b44202df676097e3b371bd0cc71e33dc5ea2973f9c3910c89bd4"} Jul 15 12:07:37.744757 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:37.744358 2587 scope.go:117] "RemoveContainer" containerID="c3ac176039d27217aba7a54b195a00eff3105dc531983b022abfd9b2f40962e3" Jul 15 12:07:37.746133 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:37.746106 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pqllv" event={"ID":"f415e718-89a6-4841-bc38-2cea520390a0","Type":"ContainerStarted","Data":"efdeb2f200c6e0cbe70b06bff3a1e9cebbd98749f87d3c6a69e31aed3ceb30ef"} Jul 15 12:07:37.746251 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:37.746155 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pqllv" event={"ID":"f415e718-89a6-4841-bc38-2cea520390a0","Type":"ContainerStarted","Data":"49dd24435c86338f507d21116bff1dbd7c0b1b15867ab43123f373f4ed72cc7a"} Jul 15 12:07:37.746328 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:37.746247 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pqllv" Jul 15 12:07:37.753409 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:37.753390 2587 scope.go:117] "RemoveContainer" containerID="c3ac176039d27217aba7a54b195a00eff3105dc531983b022abfd9b2f40962e3" Jul 15 12:07:37.753670 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:07:37.753651 2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3ac176039d27217aba7a54b195a00eff3105dc531983b022abfd9b2f40962e3\": container with ID starting with c3ac176039d27217aba7a54b195a00eff3105dc531983b022abfd9b2f40962e3 not found: ID does not exist" containerID="c3ac176039d27217aba7a54b195a00eff3105dc531983b022abfd9b2f40962e3" Jul 15 12:07:37.753716 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:37.753678 2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3ac176039d27217aba7a54b195a00eff3105dc531983b022abfd9b2f40962e3"} err="failed to get container status \"c3ac176039d27217aba7a54b195a00eff3105dc531983b022abfd9b2f40962e3\": rpc error: code = NotFound desc = could not find container \"c3ac176039d27217aba7a54b195a00eff3105dc531983b022abfd9b2f40962e3\": container with ID starting with c3ac176039d27217aba7a54b195a00eff3105dc531983b022abfd9b2f40962e3 not found: ID does not exist" Jul 15 12:07:37.771926 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:37.771895 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9cvv9"] Jul 15 12:07:37.780220 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:37.780190 2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9cvv9"] Jul 15 12:07:37.798001 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:37.797950 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pqllv" podStartSLOduration=1.7979363579999998 podStartE2EDuration="1.797936358s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:07:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:07:37.796293617 +0000 UTC m=+529.253670544" watchObservedRunningTime="2026-07-15 12:07:37.797936358 +0000 UTC m=+529.255313284" Jul 15 12:07:39.184472 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:39.184438 2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae03dcff-e1fb-4b1d-a483-a190d0ca9ee4" path="/var/lib/kubelet/pods/ae03dcff-e1fb-4b1d-a483-a190d0ca9ee4/volumes" Jul 15 12:07:48.754487 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:48.754411 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pqllv" Jul 15 12:07:48.810305 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:48.810273 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-75qsl"] Jul 15 12:07:48.810552 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:48.810511 2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-75qsl" podUID="740c4fe1-9154-4288-82eb-985723c24d7b" containerName="manager" containerID="cri-o://cb9015ed374972a796f087aa82cc24c481565b1a2f0d2eaa5ce44789c37dff29" gracePeriod=10 Jul 15 12:07:49.059625 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:49.059596 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-75qsl" Jul 15 12:07:49.236691 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:49.236663 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-krhfm\" (UniqueName: \"kubernetes.io/projected/740c4fe1-9154-4288-82eb-985723c24d7b-kube-api-access-krhfm\") pod \"740c4fe1-9154-4288-82eb-985723c24d7b\" (UID: \"740c4fe1-9154-4288-82eb-985723c24d7b\") " Jul 15 12:07:49.236876 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:49.236760 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/740c4fe1-9154-4288-82eb-985723c24d7b-extensions-socket-volume\") pod \"740c4fe1-9154-4288-82eb-985723c24d7b\" (UID: \"740c4fe1-9154-4288-82eb-985723c24d7b\") " Jul 15 12:07:49.237128 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:49.237101 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/740c4fe1-9154-4288-82eb-985723c24d7b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "740c4fe1-9154-4288-82eb-985723c24d7b" (UID: "740c4fe1-9154-4288-82eb-985723c24d7b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:07:49.238755 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:49.238723 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/740c4fe1-9154-4288-82eb-985723c24d7b-kube-api-access-krhfm" (OuterVolumeSpecName: "kube-api-access-krhfm") pod "740c4fe1-9154-4288-82eb-985723c24d7b" (UID: "740c4fe1-9154-4288-82eb-985723c24d7b"). InnerVolumeSpecName "kube-api-access-krhfm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:07:49.337935 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:49.337902 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-krhfm\" (UniqueName: \"kubernetes.io/projected/740c4fe1-9154-4288-82eb-985723c24d7b-kube-api-access-krhfm\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:07:49.337935 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:49.337930 2587 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/740c4fe1-9154-4288-82eb-985723c24d7b-extensions-socket-volume\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:07:49.819499 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:49.819465 2587 generic.go:356] "Generic (PLEG): container finished" podID="740c4fe1-9154-4288-82eb-985723c24d7b" containerID="cb9015ed374972a796f087aa82cc24c481565b1a2f0d2eaa5ce44789c37dff29" exitCode=0 Jul 15 12:07:49.819976 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:49.819513 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-75qsl" Jul 15 12:07:49.819976 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:49.819548 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-75qsl" event={"ID":"740c4fe1-9154-4288-82eb-985723c24d7b","Type":"ContainerDied","Data":"cb9015ed374972a796f087aa82cc24c481565b1a2f0d2eaa5ce44789c37dff29"} Jul 15 12:07:49.819976 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:49.819600 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-75qsl" event={"ID":"740c4fe1-9154-4288-82eb-985723c24d7b","Type":"ContainerDied","Data":"52bd13b298e04d972af73d6f042761daf9def1425949d86bba10bb952692189d"} Jul 15 12:07:49.819976 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:49.819624 2587 scope.go:117] "RemoveContainer" containerID="cb9015ed374972a796f087aa82cc24c481565b1a2f0d2eaa5ce44789c37dff29" Jul 15 12:07:49.828971 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:49.828948 2587 scope.go:117] "RemoveContainer" containerID="cb9015ed374972a796f087aa82cc24c481565b1a2f0d2eaa5ce44789c37dff29" Jul 15 12:07:49.829224 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:07:49.829206 2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb9015ed374972a796f087aa82cc24c481565b1a2f0d2eaa5ce44789c37dff29\": container with ID starting with cb9015ed374972a796f087aa82cc24c481565b1a2f0d2eaa5ce44789c37dff29 not found: ID does not exist" containerID="cb9015ed374972a796f087aa82cc24c481565b1a2f0d2eaa5ce44789c37dff29" Jul 15 12:07:49.829331 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:49.829231 2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb9015ed374972a796f087aa82cc24c481565b1a2f0d2eaa5ce44789c37dff29"} err="failed to get container status \"cb9015ed374972a796f087aa82cc24c481565b1a2f0d2eaa5ce44789c37dff29\": rpc error: code = NotFound desc = could not find container \"cb9015ed374972a796f087aa82cc24c481565b1a2f0d2eaa5ce44789c37dff29\": container with ID starting with cb9015ed374972a796f087aa82cc24c481565b1a2f0d2eaa5ce44789c37dff29 not found: ID does not exist" Jul 15 12:07:49.857029 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:49.856997 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-75qsl"] Jul 15 12:07:49.863173 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:49.863149 2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-75qsl"] Jul 15 12:07:50.276924 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:50.276827 2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-75cdfcc548-8bjw6" podUID="d958cc3b-aa17-48e1-84b6-9bfa9bfd4307" containerName="console" containerID="cri-o://2f974203c065d3ba0bb2a8110b8b9e2c5a43b5354eb0961a62cf3a533804bd2b" gracePeriod=15 Jul 15 12:07:50.530112 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:50.530047 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75cdfcc548-8bjw6" Jul 15 12:07:50.649144 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:50.649110 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hfqst\" (UniqueName: \"kubernetes.io/projected/d958cc3b-aa17-48e1-84b6-9bfa9bfd4307-kube-api-access-hfqst\") pod \"d958cc3b-aa17-48e1-84b6-9bfa9bfd4307\" (UID: \"d958cc3b-aa17-48e1-84b6-9bfa9bfd4307\") " Jul 15 12:07:50.649334 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:50.649156 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d958cc3b-aa17-48e1-84b6-9bfa9bfd4307-console-config\") pod \"d958cc3b-aa17-48e1-84b6-9bfa9bfd4307\" (UID: \"d958cc3b-aa17-48e1-84b6-9bfa9bfd4307\") " Jul 15 12:07:50.649334 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:50.649193 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d958cc3b-aa17-48e1-84b6-9bfa9bfd4307-console-serving-cert\") pod \"d958cc3b-aa17-48e1-84b6-9bfa9bfd4307\" (UID: \"d958cc3b-aa17-48e1-84b6-9bfa9bfd4307\") " Jul 15 12:07:50.649334 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:50.649211 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d958cc3b-aa17-48e1-84b6-9bfa9bfd4307-console-oauth-config\") pod \"d958cc3b-aa17-48e1-84b6-9bfa9bfd4307\" (UID: \"d958cc3b-aa17-48e1-84b6-9bfa9bfd4307\") " Jul 15 12:07:50.649334 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:50.649230 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d958cc3b-aa17-48e1-84b6-9bfa9bfd4307-oauth-serving-cert\") pod \"d958cc3b-aa17-48e1-84b6-9bfa9bfd4307\" (UID: \"d958cc3b-aa17-48e1-84b6-9bfa9bfd4307\") " Jul 15 12:07:50.649334 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:50.649272 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d958cc3b-aa17-48e1-84b6-9bfa9bfd4307-service-ca\") pod \"d958cc3b-aa17-48e1-84b6-9bfa9bfd4307\" (UID: \"d958cc3b-aa17-48e1-84b6-9bfa9bfd4307\") " Jul 15 12:07:50.649334 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:50.649294 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d958cc3b-aa17-48e1-84b6-9bfa9bfd4307-trusted-ca-bundle\") pod \"d958cc3b-aa17-48e1-84b6-9bfa9bfd4307\" (UID: \"d958cc3b-aa17-48e1-84b6-9bfa9bfd4307\") " Jul 15 12:07:50.649748 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:50.649702 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d958cc3b-aa17-48e1-84b6-9bfa9bfd4307-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d958cc3b-aa17-48e1-84b6-9bfa9bfd4307" (UID: "d958cc3b-aa17-48e1-84b6-9bfa9bfd4307"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:07:50.649838 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:50.649715 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d958cc3b-aa17-48e1-84b6-9bfa9bfd4307-service-ca" (OuterVolumeSpecName: "service-ca") pod "d958cc3b-aa17-48e1-84b6-9bfa9bfd4307" (UID: "d958cc3b-aa17-48e1-84b6-9bfa9bfd4307"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:07:50.649838 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:50.649708 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d958cc3b-aa17-48e1-84b6-9bfa9bfd4307-console-config" (OuterVolumeSpecName: "console-config") pod "d958cc3b-aa17-48e1-84b6-9bfa9bfd4307" (UID: "d958cc3b-aa17-48e1-84b6-9bfa9bfd4307"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:07:50.649838 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:50.649772 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d958cc3b-aa17-48e1-84b6-9bfa9bfd4307-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d958cc3b-aa17-48e1-84b6-9bfa9bfd4307" (UID: "d958cc3b-aa17-48e1-84b6-9bfa9bfd4307"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:07:50.651376 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:50.651350 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d958cc3b-aa17-48e1-84b6-9bfa9bfd4307-kube-api-access-hfqst" (OuterVolumeSpecName: "kube-api-access-hfqst") pod "d958cc3b-aa17-48e1-84b6-9bfa9bfd4307" (UID: "d958cc3b-aa17-48e1-84b6-9bfa9bfd4307"). InnerVolumeSpecName "kube-api-access-hfqst". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:07:50.651455 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:50.651437 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d958cc3b-aa17-48e1-84b6-9bfa9bfd4307-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d958cc3b-aa17-48e1-84b6-9bfa9bfd4307" (UID: "d958cc3b-aa17-48e1-84b6-9bfa9bfd4307"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 12:07:50.651514 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:50.651501 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d958cc3b-aa17-48e1-84b6-9bfa9bfd4307-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d958cc3b-aa17-48e1-84b6-9bfa9bfd4307" (UID: "d958cc3b-aa17-48e1-84b6-9bfa9bfd4307"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 12:07:50.750293 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:50.750255 2587 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d958cc3b-aa17-48e1-84b6-9bfa9bfd4307-console-serving-cert\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:07:50.750293 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:50.750285 2587 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d958cc3b-aa17-48e1-84b6-9bfa9bfd4307-console-oauth-config\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:07:50.750293 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:50.750301 2587 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d958cc3b-aa17-48e1-84b6-9bfa9bfd4307-oauth-serving-cert\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:07:50.750538 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:50.750309 2587 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d958cc3b-aa17-48e1-84b6-9bfa9bfd4307-service-ca\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:07:50.750538 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:50.750318 2587 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d958cc3b-aa17-48e1-84b6-9bfa9bfd4307-trusted-ca-bundle\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:07:50.750538 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:50.750326 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hfqst\" (UniqueName: \"kubernetes.io/projected/d958cc3b-aa17-48e1-84b6-9bfa9bfd4307-kube-api-access-hfqst\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:07:50.750538 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:50.750335 2587 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d958cc3b-aa17-48e1-84b6-9bfa9bfd4307-console-config\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:07:50.827350 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:50.827315 2587 generic.go:356] "Generic (PLEG): container finished" podID="d958cc3b-aa17-48e1-84b6-9bfa9bfd4307" containerID="2f974203c065d3ba0bb2a8110b8b9e2c5a43b5354eb0961a62cf3a533804bd2b" exitCode=0 Jul 15 12:07:50.827780 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:50.827362 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75cdfcc548-8bjw6" Jul 15 12:07:50.827780 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:50.827399 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75cdfcc548-8bjw6" event={"ID":"d958cc3b-aa17-48e1-84b6-9bfa9bfd4307","Type":"ContainerDied","Data":"2f974203c065d3ba0bb2a8110b8b9e2c5a43b5354eb0961a62cf3a533804bd2b"} Jul 15 12:07:50.827780 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:50.827446 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75cdfcc548-8bjw6" event={"ID":"d958cc3b-aa17-48e1-84b6-9bfa9bfd4307","Type":"ContainerDied","Data":"4ef0ccbfc1b8d92b6451a6072e564cf4bede2432ad2861ba7fa8675ca7e468ac"} Jul 15 12:07:50.827780 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:50.827467 2587 scope.go:117] "RemoveContainer" containerID="2f974203c065d3ba0bb2a8110b8b9e2c5a43b5354eb0961a62cf3a533804bd2b" Jul 15 12:07:50.837128 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:50.837105 2587 scope.go:117] "RemoveContainer" containerID="2f974203c065d3ba0bb2a8110b8b9e2c5a43b5354eb0961a62cf3a533804bd2b" Jul 15 12:07:50.837381 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:07:50.837362 2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f974203c065d3ba0bb2a8110b8b9e2c5a43b5354eb0961a62cf3a533804bd2b\": container with ID starting with 2f974203c065d3ba0bb2a8110b8b9e2c5a43b5354eb0961a62cf3a533804bd2b not found: ID does not exist" containerID="2f974203c065d3ba0bb2a8110b8b9e2c5a43b5354eb0961a62cf3a533804bd2b" Jul 15 12:07:50.837438 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:50.837394 2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f974203c065d3ba0bb2a8110b8b9e2c5a43b5354eb0961a62cf3a533804bd2b"} err="failed to get container status \"2f974203c065d3ba0bb2a8110b8b9e2c5a43b5354eb0961a62cf3a533804bd2b\": rpc error: code = NotFound desc = could not find container \"2f974203c065d3ba0bb2a8110b8b9e2c5a43b5354eb0961a62cf3a533804bd2b\": container with ID starting with 2f974203c065d3ba0bb2a8110b8b9e2c5a43b5354eb0961a62cf3a533804bd2b not found: ID does not exist" Jul 15 12:07:50.850087 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:50.850060 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-75cdfcc548-8bjw6"] Jul 15 12:07:50.856153 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:50.856128 2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-75cdfcc548-8bjw6"] Jul 15 12:07:51.184680 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:51.184602 2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="740c4fe1-9154-4288-82eb-985723c24d7b" path="/var/lib/kubelet/pods/740c4fe1-9154-4288-82eb-985723c24d7b/volumes" Jul 15 12:07:51.185030 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:07:51.185016 2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d958cc3b-aa17-48e1-84b6-9bfa9bfd4307" path="/var/lib/kubelet/pods/d958cc3b-aa17-48e1-84b6-9bfa9bfd4307/volumes" Jul 15 12:08:07.460981 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:08:07.460946 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:08:07.461535 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:08:07.461470 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae03dcff-e1fb-4b1d-a483-a190d0ca9ee4" containerName="manager" Jul 15 12:08:07.461535 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:08:07.461491 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae03dcff-e1fb-4b1d-a483-a190d0ca9ee4" containerName="manager" Jul 15 12:08:07.461535 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:08:07.461514 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="740c4fe1-9154-4288-82eb-985723c24d7b" containerName="manager" Jul 15 12:08:07.461535 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:08:07.461522 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="740c4fe1-9154-4288-82eb-985723c24d7b" containerName="manager" Jul 15 12:08:07.461815 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:08:07.461552 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d958cc3b-aa17-48e1-84b6-9bfa9bfd4307" containerName="console" Jul 15 12:08:07.461815 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:08:07.461560 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="d958cc3b-aa17-48e1-84b6-9bfa9bfd4307" containerName="console" Jul 15 12:08:07.461815 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:08:07.461647 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="740c4fe1-9154-4288-82eb-985723c24d7b" containerName="manager" Jul 15 12:08:07.461815 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:08:07.461661 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae03dcff-e1fb-4b1d-a483-a190d0ca9ee4" containerName="manager" Jul 15 12:08:07.461815 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:08:07.461672 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="d958cc3b-aa17-48e1-84b6-9bfa9bfd4307" containerName="console" Jul 15 12:08:07.466214 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:08:07.466193 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-7bz5l" Jul 15 12:08:07.469009 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:08:07.468906 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 15 12:08:07.469441 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:08:07.469411 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-vs7gg\"" Jul 15 12:08:07.472903 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:08:07.472876 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:08:07.485486 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:08:07.485453 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a7a6f20e-072d-4ede-9cd9-9e5c82860905-config-file\") pod \"limitador-limitador-54d6d988d5-7bz5l\" (UID: \"a7a6f20e-072d-4ede-9cd9-9e5c82860905\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-7bz5l" Jul 15 12:08:07.485626 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:08:07.485503 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhg65\" (UniqueName: \"kubernetes.io/projected/a7a6f20e-072d-4ede-9cd9-9e5c82860905-kube-api-access-bhg65\") pod \"limitador-limitador-54d6d988d5-7bz5l\" (UID: \"a7a6f20e-072d-4ede-9cd9-9e5c82860905\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-7bz5l" Jul 15 12:08:07.487175 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:08:07.487153 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:08:07.586449 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:08:07.586415 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhg65\" (UniqueName: \"kubernetes.io/projected/a7a6f20e-072d-4ede-9cd9-9e5c82860905-kube-api-access-bhg65\") pod \"limitador-limitador-54d6d988d5-7bz5l\" (UID: \"a7a6f20e-072d-4ede-9cd9-9e5c82860905\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-7bz5l" Jul 15 12:08:07.586616 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:08:07.586513 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a7a6f20e-072d-4ede-9cd9-9e5c82860905-config-file\") pod \"limitador-limitador-54d6d988d5-7bz5l\" (UID: \"a7a6f20e-072d-4ede-9cd9-9e5c82860905\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-7bz5l" Jul 15 12:08:07.587148 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:08:07.587122 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a7a6f20e-072d-4ede-9cd9-9e5c82860905-config-file\") pod \"limitador-limitador-54d6d988d5-7bz5l\" (UID: \"a7a6f20e-072d-4ede-9cd9-9e5c82860905\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-7bz5l" Jul 15 12:08:07.595264 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:08:07.595233 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhg65\" (UniqueName: \"kubernetes.io/projected/a7a6f20e-072d-4ede-9cd9-9e5c82860905-kube-api-access-bhg65\") pod \"limitador-limitador-54d6d988d5-7bz5l\" (UID: \"a7a6f20e-072d-4ede-9cd9-9e5c82860905\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-7bz5l" Jul 15 12:08:07.787674 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:08:07.787594 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-7bz5l" Jul 15 12:08:07.924683 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:08:07.924656 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:08:07.926443 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:08:07.926417 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7a6f20e_072d_4ede_9cd9_9e5c82860905.slice/crio-0f21f70936c7fbe31cf96251ad47abde2b236c158f1cac222e2c135ad3c30b6b WatchSource:0}: Error finding container 0f21f70936c7fbe31cf96251ad47abde2b236c158f1cac222e2c135ad3c30b6b: Status 404 returned error can't find the container with id 0f21f70936c7fbe31cf96251ad47abde2b236c158f1cac222e2c135ad3c30b6b Jul 15 12:08:08.928658 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:08:08.928592 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-7bz5l" event={"ID":"a7a6f20e-072d-4ede-9cd9-9e5c82860905","Type":"ContainerStarted","Data":"0f21f70936c7fbe31cf96251ad47abde2b236c158f1cac222e2c135ad3c30b6b"} Jul 15 12:08:10.941351 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:08:10.941254 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-7bz5l" event={"ID":"a7a6f20e-072d-4ede-9cd9-9e5c82860905","Type":"ContainerStarted","Data":"a972f6c3a0800b093dfde3a24cf8102cec522a2121576ac3c38e9d32ee698ef1"} Jul 15 12:08:10.941714 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:08:10.941349 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-7bz5l" Jul 15 12:08:10.958887 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:08:10.958832 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-7bz5l" podStartSLOduration=1.285879564 podStartE2EDuration="3.958816008s" totalImagesPullingTime="2.672936444s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:08:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:08:10.958585923 +0000 UTC m=+562.415962849" watchObservedRunningTime="2026-07-15 12:08:10.958816008 +0000 UTC m=+562.416192932" Jul 15 12:08:21.948426 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:08:21.948391 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-7bz5l" Jul 15 12:08:49.051699 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:08:49.051669 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-4xrn6_85ce5adf-d361-4258-b4ca-7f13d35fe13d/console-operator/1.log" Jul 15 12:08:49.052293 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:08:49.051805 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-4xrn6_85ce5adf-d361-4258-b4ca-7f13d35fe13d/console-operator/1.log" Jul 15 12:08:49.058307 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:08:49.058284 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mcrkx_ea415947-4742-4cf7-8b71-e22b418a131f/ovn-acl-logging/0.log" Jul 15 12:08:49.058437 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:08:49.058284 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mcrkx_ea415947-4742-4cf7-8b71-e22b418a131f/ovn-acl-logging/0.log" Jul 15 12:09:24.533114 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:09:24.533028 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:09:46.298056 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:09:46.298019 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:10:20.696641 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:10:20.696597 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:10:28.145506 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:10:28.145461 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-9gcsv"] Jul 15 12:10:28.149277 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:10:28.149254 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9gcsv" Jul 15 12:10:28.152413 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:10:28.152391 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 15 12:10:28.153990 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:10:28.153958 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 15 12:10:28.153990 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:10:28.153963 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-hz8rk\"" Jul 15 12:10:28.154152 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:10:28.153965 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 15 12:10:28.160747 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:10:28.160714 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-9gcsv"] Jul 15 12:10:28.290856 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:10:28.290818 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e667ff0b-0f12-4fa1-bb39-1e7bb96b9c93-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9gcsv\" (UID: \"e667ff0b-0f12-4fa1-bb39-1e7bb96b9c93\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9gcsv" Jul 15 12:10:28.291070 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:10:28.290869 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e667ff0b-0f12-4fa1-bb39-1e7bb96b9c93-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9gcsv\" (UID: \"e667ff0b-0f12-4fa1-bb39-1e7bb96b9c93\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9gcsv" Jul 15 12:10:28.291070 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:10:28.290946 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e667ff0b-0f12-4fa1-bb39-1e7bb96b9c93-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9gcsv\" (UID: \"e667ff0b-0f12-4fa1-bb39-1e7bb96b9c93\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9gcsv" Jul 15 12:10:28.291070 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:10:28.291017 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e667ff0b-0f12-4fa1-bb39-1e7bb96b9c93-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9gcsv\" (UID: \"e667ff0b-0f12-4fa1-bb39-1e7bb96b9c93\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9gcsv" Jul 15 12:10:28.291070 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:10:28.291061 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmcg5\" (UniqueName: \"kubernetes.io/projected/e667ff0b-0f12-4fa1-bb39-1e7bb96b9c93-kube-api-access-lmcg5\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9gcsv\" (UID: \"e667ff0b-0f12-4fa1-bb39-1e7bb96b9c93\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9gcsv" Jul 15 12:10:28.291232 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:10:28.291196 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e667ff0b-0f12-4fa1-bb39-1e7bb96b9c93-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9gcsv\" (UID: \"e667ff0b-0f12-4fa1-bb39-1e7bb96b9c93\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9gcsv" Jul 15 12:10:28.392568 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:10:28.392528 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e667ff0b-0f12-4fa1-bb39-1e7bb96b9c93-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9gcsv\" (UID: \"e667ff0b-0f12-4fa1-bb39-1e7bb96b9c93\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9gcsv" Jul 15 12:10:28.392771 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:10:28.392580 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmcg5\" (UniqueName: \"kubernetes.io/projected/e667ff0b-0f12-4fa1-bb39-1e7bb96b9c93-kube-api-access-lmcg5\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9gcsv\" (UID: \"e667ff0b-0f12-4fa1-bb39-1e7bb96b9c93\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9gcsv" Jul 15 12:10:28.392771 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:10:28.392608 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e667ff0b-0f12-4fa1-bb39-1e7bb96b9c93-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9gcsv\" (UID: \"e667ff0b-0f12-4fa1-bb39-1e7bb96b9c93\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9gcsv" Jul 15 12:10:28.392771 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:10:28.392647 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e667ff0b-0f12-4fa1-bb39-1e7bb96b9c93-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9gcsv\" (UID: \"e667ff0b-0f12-4fa1-bb39-1e7bb96b9c93\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9gcsv" Jul 15 12:10:28.392771 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:10:28.392665 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e667ff0b-0f12-4fa1-bb39-1e7bb96b9c93-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9gcsv\" (UID: \"e667ff0b-0f12-4fa1-bb39-1e7bb96b9c93\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9gcsv" Jul 15 12:10:28.392771 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:10:28.392686 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e667ff0b-0f12-4fa1-bb39-1e7bb96b9c93-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9gcsv\" (UID: \"e667ff0b-0f12-4fa1-bb39-1e7bb96b9c93\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9gcsv" Jul 15 12:10:28.393076 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:10:28.393044 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e667ff0b-0f12-4fa1-bb39-1e7bb96b9c93-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9gcsv\" (UID: \"e667ff0b-0f12-4fa1-bb39-1e7bb96b9c93\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9gcsv" Jul 15 12:10:28.393114 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:10:28.393062 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e667ff0b-0f12-4fa1-bb39-1e7bb96b9c93-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9gcsv\" (UID: \"e667ff0b-0f12-4fa1-bb39-1e7bb96b9c93\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9gcsv" Jul 15 12:10:28.393148 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:10:28.393105 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e667ff0b-0f12-4fa1-bb39-1e7bb96b9c93-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9gcsv\" (UID: \"e667ff0b-0f12-4fa1-bb39-1e7bb96b9c93\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9gcsv" Jul 15 12:10:28.394982 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:10:28.394959 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e667ff0b-0f12-4fa1-bb39-1e7bb96b9c93-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9gcsv\" (UID: \"e667ff0b-0f12-4fa1-bb39-1e7bb96b9c93\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9gcsv" Jul 15 12:10:28.395167 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:10:28.395148 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e667ff0b-0f12-4fa1-bb39-1e7bb96b9c93-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9gcsv\" (UID: \"e667ff0b-0f12-4fa1-bb39-1e7bb96b9c93\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9gcsv" Jul 15 12:10:28.400357 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:10:28.400298 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmcg5\" (UniqueName: \"kubernetes.io/projected/e667ff0b-0f12-4fa1-bb39-1e7bb96b9c93-kube-api-access-lmcg5\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9gcsv\" (UID: \"e667ff0b-0f12-4fa1-bb39-1e7bb96b9c93\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9gcsv" Jul 15 12:10:28.460412 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:10:28.460377 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9gcsv" Jul 15 12:10:28.599906 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:10:28.599869 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-9gcsv"] Jul 15 12:10:28.601190 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:10:28.601161 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode667ff0b_0f12_4fa1_bb39_1e7bb96b9c93.slice/crio-4013df9f7f39c0fe57698fac7286b38cfa96a08761c3d47d33d553ceae6c18bf WatchSource:0}: Error finding container 4013df9f7f39c0fe57698fac7286b38cfa96a08761c3d47d33d553ceae6c18bf: Status 404 returned error can't find the container with id 4013df9f7f39c0fe57698fac7286b38cfa96a08761c3d47d33d553ceae6c18bf Jul 15 12:10:28.602530 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:10:28.602512 2587 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 12:10:28.683163 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:10:28.683067 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9gcsv" event={"ID":"e667ff0b-0f12-4fa1-bb39-1e7bb96b9c93","Type":"ContainerStarted","Data":"4013df9f7f39c0fe57698fac7286b38cfa96a08761c3d47d33d553ceae6c18bf"} Jul 15 12:10:29.690366 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:10:29.690331 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:10:30.694989 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:10:30.694950 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9gcsv" event={"ID":"e667ff0b-0f12-4fa1-bb39-1e7bb96b9c93","Type":"ContainerStarted","Data":"d5c15f2471f083682a47de7b2da62734eb0f615e69487d94de7906f530db7e0e"} Jul 15 12:10:30.715895 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:10:30.715843 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9gcsv" podStartSLOduration=1.663636313 podStartE2EDuration="2.715827677s" totalImagesPullingTime="1.052191364s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:10:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:10:30.714990841 +0000 UTC m=+702.172367767" watchObservedRunningTime="2026-07-15 12:10:30.715827677 +0000 UTC m=+702.173204605" Jul 15 12:10:38.460667 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:10:38.460631 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9gcsv" Jul 15 12:10:38.461144 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:10:38.460695 2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9gcsv" Jul 15 12:10:38.472945 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:10:38.472917 2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9gcsv" Jul 15 12:10:38.747490 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:10:38.747396 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9gcsv" Jul 15 12:10:53.085568 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:10:53.085474 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:10:58.500750 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:10:58.500701 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:11:09.881692 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:11:09.881656 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:12:42.389856 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:12:42.389815 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:12:53.175716 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:12:53.175681 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:13:03.489355 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:13:03.489312 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:13:12.896302 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:13:12.896267 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:13:23.286325 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:13:23.286287 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:13:49.090455 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:13:49.090375 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-4xrn6_85ce5adf-d361-4258-b4ca-7f13d35fe13d/console-operator/1.log" Jul 15 12:13:49.090455 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:13:49.090434 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-4xrn6_85ce5adf-d361-4258-b4ca-7f13d35fe13d/console-operator/1.log" Jul 15 12:13:49.104202 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:13:49.104132 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mcrkx_ea415947-4742-4cf7-8b71-e22b418a131f/ovn-acl-logging/0.log" Jul 15 12:13:49.104202 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:13:49.104172 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mcrkx_ea415947-4742-4cf7-8b71-e22b418a131f/ovn-acl-logging/0.log" Jul 15 12:13:50.986579 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:13:50.986542 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:14:03.382976 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:14:03.382939 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:14:38.687411 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:14:38.687370 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:14:49.586443 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:14:49.586408 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:14:58.884685 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:14:58.884648 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:15:05.085575 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:15:05.085542 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:15:16.282000 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:15:16.281916 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:15:27.789700 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:15:27.789668 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:15:38.398987 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:15:38.398951 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:15:48.297182 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:15:48.297143 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:16:14.011241 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:16:14.011206 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:16:24.890458 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:16:24.890417 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:16:47.489870 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:16:47.489792 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:16:57.290683 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:16:57.290648 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:17:13.684587 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:17:13.684554 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:17:22.589714 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:17:22.589678 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:17:35.292466 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:17:35.292425 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:17:47.091317 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:17:47.091278 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:18:18.667982 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:18:18.667901 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pqllv"] Jul 15 12:18:18.668460 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:18:18.668118 2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pqllv" podUID="f415e718-89a6-4841-bc38-2cea520390a0" containerName="manager" containerID="cri-o://efdeb2f200c6e0cbe70b06bff3a1e9cebbd98749f87d3c6a69e31aed3ceb30ef" gracePeriod=10 Jul 15 12:18:21.429568 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:18:21.429539 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pqllv" Jul 15 12:18:21.510166 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:18:21.510078 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4grmz\" (UniqueName: \"kubernetes.io/projected/f415e718-89a6-4841-bc38-2cea520390a0-kube-api-access-4grmz\") pod \"f415e718-89a6-4841-bc38-2cea520390a0\" (UID: \"f415e718-89a6-4841-bc38-2cea520390a0\") " Jul 15 12:18:21.510340 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:18:21.510181 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f415e718-89a6-4841-bc38-2cea520390a0-extensions-socket-volume\") pod \"f415e718-89a6-4841-bc38-2cea520390a0\" (UID: \"f415e718-89a6-4841-bc38-2cea520390a0\") " Jul 15 12:18:21.510637 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:18:21.510614 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f415e718-89a6-4841-bc38-2cea520390a0-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f415e718-89a6-4841-bc38-2cea520390a0" (UID: "f415e718-89a6-4841-bc38-2cea520390a0"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 12:18:21.512296 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:18:21.512271 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f415e718-89a6-4841-bc38-2cea520390a0-kube-api-access-4grmz" (OuterVolumeSpecName: "kube-api-access-4grmz") pod "f415e718-89a6-4841-bc38-2cea520390a0" (UID: "f415e718-89a6-4841-bc38-2cea520390a0"). InnerVolumeSpecName "kube-api-access-4grmz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:18:21.611422 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:18:21.611383 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4grmz\" (UniqueName: \"kubernetes.io/projected/f415e718-89a6-4841-bc38-2cea520390a0-kube-api-access-4grmz\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:18:21.611422 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:18:21.611415 2587 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f415e718-89a6-4841-bc38-2cea520390a0-extensions-socket-volume\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:18:22.242637 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:18:22.242592 2587 generic.go:356] "Generic (PLEG): container finished" podID="f415e718-89a6-4841-bc38-2cea520390a0" containerID="efdeb2f200c6e0cbe70b06bff3a1e9cebbd98749f87d3c6a69e31aed3ceb30ef" exitCode=0 Jul 15 12:18:22.242637 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:18:22.242640 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pqllv" Jul 15 12:18:22.242934 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:18:22.242672 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pqllv" event={"ID":"f415e718-89a6-4841-bc38-2cea520390a0","Type":"ContainerDied","Data":"efdeb2f200c6e0cbe70b06bff3a1e9cebbd98749f87d3c6a69e31aed3ceb30ef"} Jul 15 12:18:22.242934 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:18:22.242744 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pqllv" event={"ID":"f415e718-89a6-4841-bc38-2cea520390a0","Type":"ContainerDied","Data":"49dd24435c86338f507d21116bff1dbd7c0b1b15867ab43123f373f4ed72cc7a"} Jul 15 12:18:22.242934 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:18:22.242766 2587 scope.go:117] "RemoveContainer" containerID="efdeb2f200c6e0cbe70b06bff3a1e9cebbd98749f87d3c6a69e31aed3ceb30ef" Jul 15 12:18:22.253046 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:18:22.253023 2587 scope.go:117] "RemoveContainer" containerID="efdeb2f200c6e0cbe70b06bff3a1e9cebbd98749f87d3c6a69e31aed3ceb30ef" Jul 15 12:18:22.253339 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:18:22.253315 2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"efdeb2f200c6e0cbe70b06bff3a1e9cebbd98749f87d3c6a69e31aed3ceb30ef\": container with ID starting with efdeb2f200c6e0cbe70b06bff3a1e9cebbd98749f87d3c6a69e31aed3ceb30ef not found: ID does not exist" containerID="efdeb2f200c6e0cbe70b06bff3a1e9cebbd98749f87d3c6a69e31aed3ceb30ef" Jul 15 12:18:22.253410 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:18:22.253352 2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efdeb2f200c6e0cbe70b06bff3a1e9cebbd98749f87d3c6a69e31aed3ceb30ef"} err="failed to get container status \"efdeb2f200c6e0cbe70b06bff3a1e9cebbd98749f87d3c6a69e31aed3ceb30ef\": rpc error: code = NotFound desc = could not find container \"efdeb2f200c6e0cbe70b06bff3a1e9cebbd98749f87d3c6a69e31aed3ceb30ef\": container with ID starting with efdeb2f200c6e0cbe70b06bff3a1e9cebbd98749f87d3c6a69e31aed3ceb30ef not found: ID does not exist" Jul 15 12:18:22.266614 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:18:22.266574 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pqllv"] Jul 15 12:18:22.270302 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:18:22.270276 2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pqllv"] Jul 15 12:18:23.185566 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:18:23.185539 2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f415e718-89a6-4841-bc38-2cea520390a0" path="/var/lib/kubelet/pods/f415e718-89a6-4841-bc38-2cea520390a0/volumes" Jul 15 12:18:49.132516 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:18:49.132489 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-4xrn6_85ce5adf-d361-4258-b4ca-7f13d35fe13d/console-operator/1.log" Jul 15 12:18:49.133755 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:18:49.133718 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-4xrn6_85ce5adf-d361-4258-b4ca-7f13d35fe13d/console-operator/1.log" Jul 15 12:18:49.138889 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:18:49.138865 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mcrkx_ea415947-4742-4cf7-8b71-e22b418a131f/ovn-acl-logging/0.log" Jul 15 12:18:49.140164 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:18:49.140146 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mcrkx_ea415947-4742-4cf7-8b71-e22b418a131f/ovn-acl-logging/0.log" Jul 15 12:19:19.842489 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:19:19.842454 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-96jwd"] Jul 15 12:19:19.842998 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:19:19.842905 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f415e718-89a6-4841-bc38-2cea520390a0" containerName="manager" Jul 15 12:19:19.842998 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:19:19.842919 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="f415e718-89a6-4841-bc38-2cea520390a0" containerName="manager" Jul 15 12:19:19.842998 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:19:19.842986 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="f415e718-89a6-4841-bc38-2cea520390a0" containerName="manager" Jul 15 12:19:19.846099 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:19:19.846083 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-96jwd" Jul 15 12:19:19.849020 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:19:19.848992 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-8f2ct\"" Jul 15 12:19:19.859397 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:19:19.859373 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-96jwd"] Jul 15 12:19:20.011477 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:19:20.011442 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4rc5\" (UniqueName: \"kubernetes.io/projected/3ca440f5-57ad-4103-a99c-964e0bd478ca-kube-api-access-m4rc5\") pod \"kuadrant-operator-controller-manager-86878c769f-96jwd\" (UID: \"3ca440f5-57ad-4103-a99c-964e0bd478ca\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-96jwd" Jul 15 12:19:20.011647 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:19:20.011526 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3ca440f5-57ad-4103-a99c-964e0bd478ca-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-96jwd\" (UID: \"3ca440f5-57ad-4103-a99c-964e0bd478ca\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-96jwd" Jul 15 12:19:20.112796 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:19:20.112708 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3ca440f5-57ad-4103-a99c-964e0bd478ca-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-96jwd\" (UID: \"3ca440f5-57ad-4103-a99c-964e0bd478ca\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-96jwd" Jul 15 12:19:20.112796 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:19:20.112773 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4rc5\" (UniqueName: \"kubernetes.io/projected/3ca440f5-57ad-4103-a99c-964e0bd478ca-kube-api-access-m4rc5\") pod \"kuadrant-operator-controller-manager-86878c769f-96jwd\" (UID: \"3ca440f5-57ad-4103-a99c-964e0bd478ca\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-96jwd" Jul 15 12:19:20.113145 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:19:20.113120 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3ca440f5-57ad-4103-a99c-964e0bd478ca-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-96jwd\" (UID: \"3ca440f5-57ad-4103-a99c-964e0bd478ca\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-96jwd" Jul 15 12:19:20.121035 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:19:20.121016 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4rc5\" (UniqueName: \"kubernetes.io/projected/3ca440f5-57ad-4103-a99c-964e0bd478ca-kube-api-access-m4rc5\") pod \"kuadrant-operator-controller-manager-86878c769f-96jwd\" (UID: \"3ca440f5-57ad-4103-a99c-964e0bd478ca\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-96jwd" Jul 15 12:19:20.155877 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:19:20.155846 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-96jwd" Jul 15 12:19:20.300438 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:19:20.300411 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-96jwd"] Jul 15 12:19:20.301939 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:19:20.301908 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ca440f5_57ad_4103_a99c_964e0bd478ca.slice/crio-e157be1048cbd88415e38d52e93b61037e506f39f6b34f233d44ec047744adbf WatchSource:0}: Error finding container e157be1048cbd88415e38d52e93b61037e506f39f6b34f233d44ec047744adbf: Status 404 returned error can't find the container with id e157be1048cbd88415e38d52e93b61037e506f39f6b34f233d44ec047744adbf Jul 15 12:19:20.545658 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:19:20.545622 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-96jwd" event={"ID":"3ca440f5-57ad-4103-a99c-964e0bd478ca","Type":"ContainerStarted","Data":"2bc2ebf210dbddbeacd65142621dd578194d80f1f04dfe08577cdb515621d5c1"} Jul 15 12:19:20.545868 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:19:20.545674 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-96jwd" event={"ID":"3ca440f5-57ad-4103-a99c-964e0bd478ca","Type":"ContainerStarted","Data":"e157be1048cbd88415e38d52e93b61037e506f39f6b34f233d44ec047744adbf"} Jul 15 12:19:20.545868 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:19:20.545705 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-96jwd" Jul 15 12:19:20.565578 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:19:20.565525 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-96jwd" podStartSLOduration=1.565511477 podStartE2EDuration="1.565511477s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:19:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:19:20.564036693 +0000 UTC m=+1232.021413630" watchObservedRunningTime="2026-07-15 12:19:20.565511477 +0000 UTC m=+1232.022888401" Jul 15 12:19:31.553046 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:19:31.553014 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-96jwd" Jul 15 12:19:40.187184 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:19:40.187149 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:19:53.195742 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:19:53.195660 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:20:01.617829 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:20:01.617794 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:20:12.092411 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:20:12.092376 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:20:22.693794 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:20:22.693753 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:20:33.287942 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:20:33.287900 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:20:43.587473 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:20:43.587439 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:20:52.786187 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:20:52.786146 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:21:04.581922 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:21:04.581886 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:21:13.387839 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:21:13.387758 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:21:25.584551 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:21:25.584513 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:21:46.189534 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:21:46.189495 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:21:55.190481 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:21:55.190450 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:22:04.884535 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:22:04.884489 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:22:16.379862 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:22:16.379820 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:22:34.391554 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:22:34.391506 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:22:43.681291 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:22:43.681213 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:22:53.284288 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:22:53.284254 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:23:00.691779 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:23:00.691725 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:23:10.080782 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:23:10.080719 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:23:19.087533 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:23:19.087501 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:23:33.493043 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:23:33.493006 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:23:42.486004 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:23:42.485966 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:23:49.169687 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:23:49.169657 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-4xrn6_85ce5adf-d361-4258-b4ca-7f13d35fe13d/console-operator/1.log" Jul 15 12:23:49.171433 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:23:49.171411 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-4xrn6_85ce5adf-d361-4258-b4ca-7f13d35fe13d/console-operator/1.log" Jul 15 12:23:49.176304 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:23:49.176281 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mcrkx_ea415947-4742-4cf7-8b71-e22b418a131f/ovn-acl-logging/0.log" Jul 15 12:23:49.177968 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:23:49.177949 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mcrkx_ea415947-4742-4cf7-8b71-e22b418a131f/ovn-acl-logging/0.log" Jul 15 12:23:51.982062 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:23:51.982027 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:24:02.096434 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:24:02.096395 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:24:11.483217 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:24:11.483140 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:24:20.382669 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:24:20.382636 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:24:31.481120 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:24:31.481089 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:24:40.696523 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:24:40.696485 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:24:50.283935 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:24:50.283899 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:24:57.002290 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:24:57.002254 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:25:12.786977 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:25:12.786936 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:26:28.385887 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:26:28.385854 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:26:46.747840 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:26:46.747802 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bbac7fd2-wvmfj"] Jul 15 12:26:46.751378 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:26:46.751362 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bbac7fd2-wvmfj" Jul 15 12:26:46.754168 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:26:46.754147 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tf4tp\"" Jul 15 12:26:46.754268 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:26:46.754195 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 12:26:46.760247 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:26:46.760217 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bbac7fd2-wvmfj"] Jul 15 12:26:46.800847 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:26:46.800813 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a280591-d33c-4e8c-8c94-fc64f83d528f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bbac7fd2-wvmfj\" (UID: \"9a280591-d33c-4e8c-8c94-fc64f83d528f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bbac7fd2-wvmfj" Jul 15 12:26:46.901909 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:26:46.901874 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a280591-d33c-4e8c-8c94-fc64f83d528f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bbac7fd2-wvmfj\" (UID: \"9a280591-d33c-4e8c-8c94-fc64f83d528f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bbac7fd2-wvmfj" Jul 15 12:26:46.902443 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:26:46.902424 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a280591-d33c-4e8c-8c94-fc64f83d528f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bbac7fd2-wvmfj\" (UID: \"9a280591-d33c-4e8c-8c94-fc64f83d528f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bbac7fd2-wvmfj" Jul 15 12:26:47.062963 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:26:47.062941 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bbac7fd2-wvmfj" Jul 15 12:26:47.202556 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:26:47.202533 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bbac7fd2-wvmfj"] Jul 15 12:26:47.203970 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:26:47.203943 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a280591_d33c_4e8c_8c94_fc64f83d528f.slice/crio-801cef3d14c8eb73fa71b4b507733141ad2f2c1acd61defce013a78ba463efa1 WatchSource:0}: Error finding container 801cef3d14c8eb73fa71b4b507733141ad2f2c1acd61defce013a78ba463efa1: Status 404 returned error can't find the container with id 801cef3d14c8eb73fa71b4b507733141ad2f2c1acd61defce013a78ba463efa1 Jul 15 12:26:47.205767 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:26:47.205754 2587 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 12:26:48.005385 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:26:48.005332 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bbac7fd2-wvmfj" event={"ID":"9a280591-d33c-4e8c-8c94-fc64f83d528f","Type":"ContainerStarted","Data":"801cef3d14c8eb73fa71b4b507733141ad2f2c1acd61defce013a78ba463efa1"} Jul 15 12:26:49.782896 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:26:49.782862 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:26:50.018280 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:26:50.018246 2587 generic.go:356] "Generic (PLEG): container finished" podID="9a280591-d33c-4e8c-8c94-fc64f83d528f" containerID="c4e27375194a1733231dbdc26a365dabc9d6809be202819bd6f538e906232066" exitCode=7 Jul 15 12:26:50.018451 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:26:50.018332 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bbac7fd2-wvmfj" event={"ID":"9a280591-d33c-4e8c-8c94-fc64f83d528f","Type":"ContainerDied","Data":"c4e27375194a1733231dbdc26a365dabc9d6809be202819bd6f538e906232066"} Jul 15 12:26:50.018533 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:26:50.018521 2587 scope.go:117] "RemoveContainer" containerID="c4e27375194a1733231dbdc26a365dabc9d6809be202819bd6f538e906232066" Jul 15 12:26:51.025963 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:26:51.025874 2587 generic.go:356] "Generic (PLEG): container finished" podID="9a280591-d33c-4e8c-8c94-fc64f83d528f" containerID="ce037bfc13840914b57cfd2a26249c75076f4191f3d33f938903ccf1cd3cb1a2" exitCode=7 Jul 15 12:26:51.025963 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:26:51.025935 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bbac7fd2-wvmfj" event={"ID":"9a280591-d33c-4e8c-8c94-fc64f83d528f","Type":"ContainerDied","Data":"ce037bfc13840914b57cfd2a26249c75076f4191f3d33f938903ccf1cd3cb1a2"} Jul 15 12:26:51.026425 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:26:51.025989 2587 scope.go:117] "RemoveContainer" containerID="c4e27375194a1733231dbdc26a365dabc9d6809be202819bd6f538e906232066" Jul 15 12:26:51.026425 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:26:51.026125 2587 scope.go:117] "RemoveContainer" containerID="ce037bfc13840914b57cfd2a26249c75076f4191f3d33f938903ccf1cd3cb1a2" Jul 15 12:26:51.026425 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:26:51.026395 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bbac7fd2-wvmfj_odh-ai-gateway-infra(9a280591-d33c-4e8c-8c94-fc64f83d528f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bbac7fd2-wvmfj" podUID="9a280591-d33c-4e8c-8c94-fc64f83d528f" Jul 15 12:26:52.034265 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:26:52.034237 2587 scope.go:117] "RemoveContainer" containerID="ce037bfc13840914b57cfd2a26249c75076f4191f3d33f938903ccf1cd3cb1a2" Jul 15 12:26:52.034650 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:26:52.034442 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bbac7fd2-wvmfj_odh-ai-gateway-infra(9a280591-d33c-4e8c-8c94-fc64f83d528f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bbac7fd2-wvmfj" podUID="9a280591-d33c-4e8c-8c94-fc64f83d528f" Jul 15 12:27:05.179646 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:27:05.179609 2587 scope.go:117] "RemoveContainer" containerID="ce037bfc13840914b57cfd2a26249c75076f4191f3d33f938903ccf1cd3cb1a2" Jul 15 12:27:06.109047 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:27:06.108966 2587 generic.go:356] "Generic (PLEG): container finished" podID="9a280591-d33c-4e8c-8c94-fc64f83d528f" containerID="a0aed627b9a76fd4518643a4e8e3bf2332559979d79e3cbb2fbcf521af9ab262" exitCode=0 Jul 15 12:27:06.109047 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:27:06.109007 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bbac7fd2-wvmfj" event={"ID":"9a280591-d33c-4e8c-8c94-fc64f83d528f","Type":"ContainerDied","Data":"a0aed627b9a76fd4518643a4e8e3bf2332559979d79e3cbb2fbcf521af9ab262"} Jul 15 12:27:06.109219 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:27:06.109063 2587 scope.go:117] "RemoveContainer" containerID="ce037bfc13840914b57cfd2a26249c75076f4191f3d33f938903ccf1cd3cb1a2" Jul 15 12:27:07.143145 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:27:07.143107 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bbac7fd2-wvmfj"] Jul 15 12:27:07.256333 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:27:07.256303 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bbac7fd2-wvmfj" Jul 15 12:27:07.383245 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:27:07.383168 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a280591-d33c-4e8c-8c94-fc64f83d528f-maas-api-service-ca\") pod \"9a280591-d33c-4e8c-8c94-fc64f83d528f\" (UID: \"9a280591-d33c-4e8c-8c94-fc64f83d528f\") " Jul 15 12:27:07.383492 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:27:07.383472 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a280591-d33c-4e8c-8c94-fc64f83d528f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9a280591-d33c-4e8c-8c94-fc64f83d528f" (UID: "9a280591-d33c-4e8c-8c94-fc64f83d528f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:27:07.484239 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:27:07.484210 2587 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a280591-d33c-4e8c-8c94-fc64f83d528f-maas-api-service-ca\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:27:08.121548 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:27:08.121518 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bbac7fd2-wvmfj" Jul 15 12:27:08.121548 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:27:08.121539 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bbac7fd2-wvmfj" event={"ID":"9a280591-d33c-4e8c-8c94-fc64f83d528f","Type":"ContainerDied","Data":"801cef3d14c8eb73fa71b4b507733141ad2f2c1acd61defce013a78ba463efa1"} Jul 15 12:27:08.121844 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:27:08.121581 2587 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="801cef3d14c8eb73fa71b4b507733141ad2f2c1acd61defce013a78ba463efa1" Jul 15 12:27:08.155726 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:27:08.155698 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bbac7fd2-wvmfj"] Jul 15 12:27:08.158314 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:27:08.158289 2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bbac7fd2-wvmfj"] Jul 15 12:27:09.186383 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:27:09.186349 2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a280591-d33c-4e8c-8c94-fc64f83d528f" path="/var/lib/kubelet/pods/9a280591-d33c-4e8c-8c94-fc64f83d528f/volumes" Jul 15 12:27:49.487817 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:27:49.487780 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:28:03.419120 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:28:03.419084 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:28:49.207191 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:28:49.207166 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-4xrn6_85ce5adf-d361-4258-b4ca-7f13d35fe13d/console-operator/1.log" Jul 15 12:28:49.209681 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:28:49.209659 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-4xrn6_85ce5adf-d361-4258-b4ca-7f13d35fe13d/console-operator/1.log" Jul 15 12:28:49.213303 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:28:49.213283 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mcrkx_ea415947-4742-4cf7-8b71-e22b418a131f/ovn-acl-logging/0.log" Jul 15 12:28:49.215986 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:28:49.215969 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mcrkx_ea415947-4742-4cf7-8b71-e22b418a131f/ovn-acl-logging/0.log" Jul 15 12:29:19.190086 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:29:19.190053 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:29:35.695600 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:29:35.695560 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:29:37.824132 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:29:37.824098 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-99db84ea-2b2j4"] Jul 15 12:29:37.824514 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:29:37.824466 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a280591-d33c-4e8c-8c94-fc64f83d528f" containerName="revoke-keys" Jul 15 12:29:37.824514 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:29:37.824482 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a280591-d33c-4e8c-8c94-fc64f83d528f" containerName="revoke-keys" Jul 15 12:29:37.824514 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:29:37.824512 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a280591-d33c-4e8c-8c94-fc64f83d528f" containerName="revoke-keys" Jul 15 12:29:37.824636 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:29:37.824518 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a280591-d33c-4e8c-8c94-fc64f83d528f" containerName="revoke-keys" Jul 15 12:29:37.824636 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:29:37.824569 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a280591-d33c-4e8c-8c94-fc64f83d528f" containerName="revoke-keys" Jul 15 12:29:37.824636 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:29:37.824580 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a280591-d33c-4e8c-8c94-fc64f83d528f" containerName="revoke-keys" Jul 15 12:29:37.827670 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:29:37.827652 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-99db84ea-2b2j4" Jul 15 12:29:37.830573 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:29:37.830547 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tf4tp\"" Jul 15 12:29:37.830769 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:29:37.830709 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 12:29:37.836104 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:29:37.836034 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-99db84ea-2b2j4"] Jul 15 12:29:37.940876 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:29:37.940843 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b38e667-b8bb-47ff-a431-3dbe6435f20c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-99db84ea-2b2j4\" (UID: \"9b38e667-b8bb-47ff-a431-3dbe6435f20c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-99db84ea-2b2j4" Jul 15 12:29:38.041600 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:29:38.041567 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b38e667-b8bb-47ff-a431-3dbe6435f20c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-99db84ea-2b2j4\" (UID: \"9b38e667-b8bb-47ff-a431-3dbe6435f20c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-99db84ea-2b2j4" Jul 15 12:29:38.042175 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:29:38.042154 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b38e667-b8bb-47ff-a431-3dbe6435f20c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-99db84ea-2b2j4\" (UID: \"9b38e667-b8bb-47ff-a431-3dbe6435f20c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-99db84ea-2b2j4" Jul 15 12:29:38.139310 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:29:38.139236 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-99db84ea-2b2j4" Jul 15 12:29:38.271960 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:29:38.271937 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-99db84ea-2b2j4"] Jul 15 12:29:38.274166 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:29:38.274137 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b38e667_b8bb_47ff_a431_3dbe6435f20c.slice/crio-468085b4374241d057f1c404ffc3d34144e95a528cc1e448d49508d768666f12 WatchSource:0}: Error finding container 468085b4374241d057f1c404ffc3d34144e95a528cc1e448d49508d768666f12: Status 404 returned error can't find the container with id 468085b4374241d057f1c404ffc3d34144e95a528cc1e448d49508d768666f12 Jul 15 12:29:38.952412 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:29:38.952379 2587 generic.go:356] "Generic (PLEG): container finished" podID="9b38e667-b8bb-47ff-a431-3dbe6435f20c" containerID="ba8b331ffbe4bf2fe75b6d8370fab89fe7c9c4227f2721f033f5d334bc57b6cc" exitCode=7 Jul 15 12:29:38.952976 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:29:38.952466 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-99db84ea-2b2j4" event={"ID":"9b38e667-b8bb-47ff-a431-3dbe6435f20c","Type":"ContainerDied","Data":"ba8b331ffbe4bf2fe75b6d8370fab89fe7c9c4227f2721f033f5d334bc57b6cc"} Jul 15 12:29:38.952976 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:29:38.952524 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-99db84ea-2b2j4" event={"ID":"9b38e667-b8bb-47ff-a431-3dbe6435f20c","Type":"ContainerStarted","Data":"468085b4374241d057f1c404ffc3d34144e95a528cc1e448d49508d768666f12"} Jul 15 12:29:38.952976 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:29:38.952678 2587 scope.go:117] "RemoveContainer" containerID="ba8b331ffbe4bf2fe75b6d8370fab89fe7c9c4227f2721f033f5d334bc57b6cc" Jul 15 12:29:39.961025 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:29:39.960988 2587 generic.go:356] "Generic (PLEG): container finished" podID="9b38e667-b8bb-47ff-a431-3dbe6435f20c" containerID="701a25cbba56ebf93e97834e23a134269398d473ab8f2bf8ba3e0f5b7567a0e3" exitCode=7 Jul 15 12:29:39.961423 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:29:39.961078 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-99db84ea-2b2j4" event={"ID":"9b38e667-b8bb-47ff-a431-3dbe6435f20c","Type":"ContainerDied","Data":"701a25cbba56ebf93e97834e23a134269398d473ab8f2bf8ba3e0f5b7567a0e3"} Jul 15 12:29:39.961423 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:29:39.961137 2587 scope.go:117] "RemoveContainer" containerID="ba8b331ffbe4bf2fe75b6d8370fab89fe7c9c4227f2721f033f5d334bc57b6cc" Jul 15 12:29:39.961423 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:29:39.961234 2587 scope.go:117] "RemoveContainer" containerID="701a25cbba56ebf93e97834e23a134269398d473ab8f2bf8ba3e0f5b7567a0e3" Jul 15 12:29:39.961546 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:29:39.961444 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-99db84ea-2b2j4_odh-ai-gateway-infra(9b38e667-b8bb-47ff-a431-3dbe6435f20c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-99db84ea-2b2j4" podUID="9b38e667-b8bb-47ff-a431-3dbe6435f20c" Jul 15 12:29:40.968064 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:29:40.968034 2587 scope.go:117] "RemoveContainer" containerID="701a25cbba56ebf93e97834e23a134269398d473ab8f2bf8ba3e0f5b7567a0e3" Jul 15 12:29:40.968528 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:29:40.968272 2587 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-99db84ea-2b2j4_odh-ai-gateway-infra(9b38e667-b8bb-47ff-a431-3dbe6435f20c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-99db84ea-2b2j4" podUID="9b38e667-b8bb-47ff-a431-3dbe6435f20c" Jul 15 12:29:52.179307 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:29:52.179269 2587 scope.go:117] "RemoveContainer" containerID="701a25cbba56ebf93e97834e23a134269398d473ab8f2bf8ba3e0f5b7567a0e3" Jul 15 12:29:53.039659 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:29:53.039571 2587 generic.go:356] "Generic (PLEG): container finished" podID="9b38e667-b8bb-47ff-a431-3dbe6435f20c" containerID="38757e2e03f3cd99b8fbe82deae094449237f09582760d22e32f7dc71789374d" exitCode=0 Jul 15 12:29:53.039659 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:29:53.039641 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-99db84ea-2b2j4" event={"ID":"9b38e667-b8bb-47ff-a431-3dbe6435f20c","Type":"ContainerDied","Data":"38757e2e03f3cd99b8fbe82deae094449237f09582760d22e32f7dc71789374d"} Jul 15 12:29:53.039894 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:29:53.039699 2587 scope.go:117] "RemoveContainer" containerID="701a25cbba56ebf93e97834e23a134269398d473ab8f2bf8ba3e0f5b7567a0e3" Jul 15 12:29:54.074980 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:29:54.074942 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-99db84ea-2b2j4"] Jul 15 12:29:54.178432 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:29:54.178407 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-99db84ea-2b2j4" Jul 15 12:29:54.279440 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:29:54.279410 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b38e667-b8bb-47ff-a431-3dbe6435f20c-maas-api-service-ca\") pod \"9b38e667-b8bb-47ff-a431-3dbe6435f20c\" (UID: \"9b38e667-b8bb-47ff-a431-3dbe6435f20c\") " Jul 15 12:29:54.279774 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:29:54.279754 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b38e667-b8bb-47ff-a431-3dbe6435f20c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9b38e667-b8bb-47ff-a431-3dbe6435f20c" (UID: "9b38e667-b8bb-47ff-a431-3dbe6435f20c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:29:54.380790 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:29:54.380691 2587 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b38e667-b8bb-47ff-a431-3dbe6435f20c-maas-api-service-ca\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:29:55.054012 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:29:55.053980 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-99db84ea-2b2j4" event={"ID":"9b38e667-b8bb-47ff-a431-3dbe6435f20c","Type":"ContainerDied","Data":"468085b4374241d057f1c404ffc3d34144e95a528cc1e448d49508d768666f12"} Jul 15 12:29:55.054012 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:29:55.054016 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-99db84ea-2b2j4" Jul 15 12:29:55.054218 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:29:55.054025 2587 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="468085b4374241d057f1c404ffc3d34144e95a528cc1e448d49508d768666f12" Jul 15 12:29:55.082442 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:29:55.082415 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-99db84ea-2b2j4"] Jul 15 12:29:55.091760 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:29:55.091721 2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-99db84ea-2b2j4"] Jul 15 12:29:55.184643 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:29:55.184614 2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b38e667-b8bb-47ff-a431-3dbe6435f20c" path="/var/lib/kubelet/pods/9b38e667-b8bb-47ff-a431-3dbe6435f20c/volumes" Jul 15 12:30:41.599452 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:30:41.599420 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:32:18.184210 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:32:18.184119 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:32:26.799173 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:32:26.799134 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:32:39.187148 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:32:39.187116 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:33:00.885246 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:33:00.885212 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:33:43.985226 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:33:43.985192 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:33:49.243397 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:33:49.243368 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-4xrn6_85ce5adf-d361-4258-b4ca-7f13d35fe13d/console-operator/1.log" Jul 15 12:33:49.248527 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:33:49.248502 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-4xrn6_85ce5adf-d361-4258-b4ca-7f13d35fe13d/console-operator/1.log" Jul 15 12:33:49.249694 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:33:49.249676 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mcrkx_ea415947-4742-4cf7-8b71-e22b418a131f/ovn-acl-logging/0.log" Jul 15 12:33:49.254555 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:33:49.254538 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mcrkx_ea415947-4742-4cf7-8b71-e22b418a131f/ovn-acl-logging/0.log" Jul 15 12:33:49.314078 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:33:49.314056 2587 scope.go:117] "RemoveContainer" containerID="a0aed627b9a76fd4518643a4e8e3bf2332559979d79e3cbb2fbcf521af9ab262" Jul 15 12:34:01.890081 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:34:01.890039 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:34:16.695132 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:34:16.695101 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:34:19.791936 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:34:19.791901 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-145feecc-jz8rk"] Jul 15 12:34:19.792307 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:34:19.792282 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a280591-d33c-4e8c-8c94-fc64f83d528f" containerName="revoke-keys" Jul 15 12:34:19.792307 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:34:19.792294 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a280591-d33c-4e8c-8c94-fc64f83d528f" containerName="revoke-keys" Jul 15 12:34:19.792377 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:34:19.792314 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b38e667-b8bb-47ff-a431-3dbe6435f20c" containerName="revoke-keys" Jul 15 12:34:19.792377 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:34:19.792320 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b38e667-b8bb-47ff-a431-3dbe6435f20c" containerName="revoke-keys" Jul 15 12:34:19.792377 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:34:19.792329 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b38e667-b8bb-47ff-a431-3dbe6435f20c" containerName="revoke-keys" Jul 15 12:34:19.792377 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:34:19.792335 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b38e667-b8bb-47ff-a431-3dbe6435f20c" containerName="revoke-keys" Jul 15 12:34:19.792377 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:34:19.792342 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b38e667-b8bb-47ff-a431-3dbe6435f20c" containerName="revoke-keys" Jul 15 12:34:19.792377 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:34:19.792347 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b38e667-b8bb-47ff-a431-3dbe6435f20c" containerName="revoke-keys" Jul 15 12:34:19.792555 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:34:19.792417 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b38e667-b8bb-47ff-a431-3dbe6435f20c" containerName="revoke-keys" Jul 15 12:34:19.792555 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:34:19.792426 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a280591-d33c-4e8c-8c94-fc64f83d528f" containerName="revoke-keys" Jul 15 12:34:19.792555 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:34:19.792433 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b38e667-b8bb-47ff-a431-3dbe6435f20c" containerName="revoke-keys" Jul 15 12:34:19.792555 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:34:19.792439 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b38e667-b8bb-47ff-a431-3dbe6435f20c" containerName="revoke-keys" Jul 15 12:34:19.795613 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:34:19.795595 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-145feecc-jz8rk" Jul 15 12:34:19.798259 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:34:19.798237 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 12:34:19.798385 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:34:19.798275 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tf4tp\"" Jul 15 12:34:19.803471 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:34:19.803449 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-145feecc-jz8rk"] Jul 15 12:34:19.902432 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:34:19.902398 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/faa078fb-5623-4357-9149-3c02bc392c4b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-145feecc-jz8rk\" (UID: \"faa078fb-5623-4357-9149-3c02bc392c4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-145feecc-jz8rk" Jul 15 12:34:20.003102 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:34:20.003051 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/faa078fb-5623-4357-9149-3c02bc392c4b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-145feecc-jz8rk\" (UID: \"faa078fb-5623-4357-9149-3c02bc392c4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-145feecc-jz8rk" Jul 15 12:34:20.003659 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:34:20.003638 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/faa078fb-5623-4357-9149-3c02bc392c4b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-145feecc-jz8rk\" (UID: \"faa078fb-5623-4357-9149-3c02bc392c4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-145feecc-jz8rk" Jul 15 12:34:20.107216 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:34:20.107137 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-145feecc-jz8rk" Jul 15 12:34:20.240558 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:34:20.240535 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-145feecc-jz8rk"] Jul 15 12:34:20.242657 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:34:20.242625 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfaa078fb_5623_4357_9149_3c02bc392c4b.slice/crio-87da10846f73c9e45b24d7f0c0a4c021d0829d0f3d6d03dd34aaaebfa9725608 WatchSource:0}: Error finding container 87da10846f73c9e45b24d7f0c0a4c021d0829d0f3d6d03dd34aaaebfa9725608: Status 404 returned error can't find the container with id 87da10846f73c9e45b24d7f0c0a4c021d0829d0f3d6d03dd34aaaebfa9725608 Jul 15 12:34:20.518188 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:34:20.518151 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-145feecc-jz8rk" event={"ID":"faa078fb-5623-4357-9149-3c02bc392c4b","Type":"ContainerStarted","Data":"87da10846f73c9e45b24d7f0c0a4c021d0829d0f3d6d03dd34aaaebfa9725608"} Jul 15 12:34:21.524605 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:34:21.524571 2587 generic.go:356] "Generic (PLEG): container finished" podID="faa078fb-5623-4357-9149-3c02bc392c4b" containerID="0a28b5f1ec29102ca049c9941dfa217e986c81fdc10768af0cf6a9aa6806faf1" exitCode=0 Jul 15 12:34:21.525093 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:34:21.524650 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-145feecc-jz8rk" event={"ID":"faa078fb-5623-4357-9149-3c02bc392c4b","Type":"ContainerDied","Data":"0a28b5f1ec29102ca049c9941dfa217e986c81fdc10768af0cf6a9aa6806faf1"} Jul 15 12:34:22.665582 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:34:22.665556 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-145feecc-jz8rk" Jul 15 12:34:22.830113 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:34:22.830086 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/faa078fb-5623-4357-9149-3c02bc392c4b-maas-api-service-ca\") pod \"faa078fb-5623-4357-9149-3c02bc392c4b\" (UID: \"faa078fb-5623-4357-9149-3c02bc392c4b\") " Jul 15 12:34:22.830429 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:34:22.830407 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/faa078fb-5623-4357-9149-3c02bc392c4b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "faa078fb-5623-4357-9149-3c02bc392c4b" (UID: "faa078fb-5623-4357-9149-3c02bc392c4b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:34:22.931411 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:34:22.931384 2587 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/faa078fb-5623-4357-9149-3c02bc392c4b-maas-api-service-ca\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:34:23.537689 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:34:23.537613 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-145feecc-jz8rk" Jul 15 12:34:23.537841 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:34:23.537612 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-145feecc-jz8rk" event={"ID":"faa078fb-5623-4357-9149-3c02bc392c4b","Type":"ContainerDied","Data":"87da10846f73c9e45b24d7f0c0a4c021d0829d0f3d6d03dd34aaaebfa9725608"} Jul 15 12:34:23.537841 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:34:23.537754 2587 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="87da10846f73c9e45b24d7f0c0a4c021d0829d0f3d6d03dd34aaaebfa9725608" Jul 15 12:34:33.981294 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:34:33.981251 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:35:33.382907 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:35:33.382872 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:35:42.179750 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:35:42.179697 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:36:32.086753 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:36:32.086686 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:36:32.832231 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:36:32.832199 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-6dba6e-5b745dd8dd-nr58h"] Jul 15 12:36:32.832713 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:36:32.832694 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="faa078fb-5623-4357-9149-3c02bc392c4b" containerName="revoke-keys" Jul 15 12:36:32.832840 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:36:32.832718 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="faa078fb-5623-4357-9149-3c02bc392c4b" containerName="revoke-keys" Jul 15 12:36:32.832902 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:36:32.832844 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="faa078fb-5623-4357-9149-3c02bc392c4b" containerName="revoke-keys" Jul 15 12:36:32.836497 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:36:32.836472 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6dba6e-5b745dd8dd-nr58h" Jul 15 12:36:32.839351 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:36:32.839324 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-6dba6e-serving-cert\"" Jul 15 12:36:32.839451 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:36:32.839358 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 15 12:36:32.839451 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:36:32.839394 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 12:36:32.840505 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:36:32.840484 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-hgh4d\"" Jul 15 12:36:32.844942 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:36:32.844844 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-6dba6e-5b745dd8dd-nr58h"] Jul 15 12:36:32.926689 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:36:32.926657 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gn5fd\" (UniqueName: \"kubernetes.io/projected/2874298c-fa1e-4cce-8c03-b643e6d33537-kube-api-access-gn5fd\") pod \"maas-api-e2e-inf-b-6dba6e-5b745dd8dd-nr58h\" (UID: \"2874298c-fa1e-4cce-8c03-b643e6d33537\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6dba6e-5b745dd8dd-nr58h" Jul 15 12:36:32.926961 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:36:32.926798 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2874298c-fa1e-4cce-8c03-b643e6d33537-openshift-service-ca\") pod \"maas-api-e2e-inf-b-6dba6e-5b745dd8dd-nr58h\" (UID: \"2874298c-fa1e-4cce-8c03-b643e6d33537\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6dba6e-5b745dd8dd-nr58h" Jul 15 12:36:32.926961 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:36:32.926835 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2874298c-fa1e-4cce-8c03-b643e6d33537-maas-api-tls\") pod \"maas-api-e2e-inf-b-6dba6e-5b745dd8dd-nr58h\" (UID: \"2874298c-fa1e-4cce-8c03-b643e6d33537\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6dba6e-5b745dd8dd-nr58h" Jul 15 12:36:33.027892 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:36:33.027852 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gn5fd\" (UniqueName: \"kubernetes.io/projected/2874298c-fa1e-4cce-8c03-b643e6d33537-kube-api-access-gn5fd\") pod \"maas-api-e2e-inf-b-6dba6e-5b745dd8dd-nr58h\" (UID: \"2874298c-fa1e-4cce-8c03-b643e6d33537\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6dba6e-5b745dd8dd-nr58h" Jul 15 12:36:33.028056 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:36:33.027950 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2874298c-fa1e-4cce-8c03-b643e6d33537-openshift-service-ca\") pod \"maas-api-e2e-inf-b-6dba6e-5b745dd8dd-nr58h\" (UID: \"2874298c-fa1e-4cce-8c03-b643e6d33537\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6dba6e-5b745dd8dd-nr58h" Jul 15 12:36:33.028056 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:36:33.027987 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2874298c-fa1e-4cce-8c03-b643e6d33537-maas-api-tls\") pod \"maas-api-e2e-inf-b-6dba6e-5b745dd8dd-nr58h\" (UID: \"2874298c-fa1e-4cce-8c03-b643e6d33537\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6dba6e-5b745dd8dd-nr58h" Jul 15 12:36:33.028131 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:36:33.028111 2587 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-6dba6e-serving-cert: secret "maas-api-e2e-inf-b-6dba6e-serving-cert" not found Jul 15 12:36:33.028188 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:36:33.028177 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2874298c-fa1e-4cce-8c03-b643e6d33537-maas-api-tls podName:2874298c-fa1e-4cce-8c03-b643e6d33537 nodeName:}" failed. No retries permitted until 2026-07-15 12:36:33.528154424 +0000 UTC m=+2264.985531346 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2874298c-fa1e-4cce-8c03-b643e6d33537-maas-api-tls") pod "maas-api-e2e-inf-b-6dba6e-5b745dd8dd-nr58h" (UID: "2874298c-fa1e-4cce-8c03-b643e6d33537") : secret "maas-api-e2e-inf-b-6dba6e-serving-cert" not found Jul 15 12:36:33.028627 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:36:33.028607 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2874298c-fa1e-4cce-8c03-b643e6d33537-openshift-service-ca\") pod \"maas-api-e2e-inf-b-6dba6e-5b745dd8dd-nr58h\" (UID: \"2874298c-fa1e-4cce-8c03-b643e6d33537\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6dba6e-5b745dd8dd-nr58h" Jul 15 12:36:33.039607 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:36:33.039585 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gn5fd\" (UniqueName: \"kubernetes.io/projected/2874298c-fa1e-4cce-8c03-b643e6d33537-kube-api-access-gn5fd\") pod \"maas-api-e2e-inf-b-6dba6e-5b745dd8dd-nr58h\" (UID: \"2874298c-fa1e-4cce-8c03-b643e6d33537\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6dba6e-5b745dd8dd-nr58h" Jul 15 12:36:33.532907 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:36:33.532872 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2874298c-fa1e-4cce-8c03-b643e6d33537-maas-api-tls\") pod \"maas-api-e2e-inf-b-6dba6e-5b745dd8dd-nr58h\" (UID: \"2874298c-fa1e-4cce-8c03-b643e6d33537\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6dba6e-5b745dd8dd-nr58h" Jul 15 12:36:33.535409 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:36:33.535381 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2874298c-fa1e-4cce-8c03-b643e6d33537-maas-api-tls\") pod \"maas-api-e2e-inf-b-6dba6e-5b745dd8dd-nr58h\" (UID: \"2874298c-fa1e-4cce-8c03-b643e6d33537\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6dba6e-5b745dd8dd-nr58h" Jul 15 12:36:33.748495 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:36:33.748457 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6dba6e-5b745dd8dd-nr58h" Jul 15 12:36:33.900128 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:36:33.900099 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-6dba6e-5b745dd8dd-nr58h"] Jul 15 12:36:33.902039 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:36:33.902008 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2874298c_fa1e_4cce_8c03_b643e6d33537.slice/crio-35bb44cce236b4415c50b28ea4f26f2b282de129d945ddb88732c82e579a9a5b WatchSource:0}: Error finding container 35bb44cce236b4415c50b28ea4f26f2b282de129d945ddb88732c82e579a9a5b: Status 404 returned error can't find the container with id 35bb44cce236b4415c50b28ea4f26f2b282de129d945ddb88732c82e579a9a5b Jul 15 12:36:33.903295 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:36:33.903277 2587 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 12:36:34.268345 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:36:34.268261 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6dba6e-5b745dd8dd-nr58h" event={"ID":"2874298c-fa1e-4cce-8c03-b643e6d33537","Type":"ContainerStarted","Data":"35bb44cce236b4415c50b28ea4f26f2b282de129d945ddb88732c82e579a9a5b"} Jul 15 12:36:37.288585 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:36:37.288530 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6dba6e-5b745dd8dd-nr58h" event={"ID":"2874298c-fa1e-4cce-8c03-b643e6d33537","Type":"ContainerStarted","Data":"060b49a64ab6696417d1c2c28afa637aa583d089c7568a9f40e1cc728abc5f8c"} Jul 15 12:36:37.289121 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:36:37.288833 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6dba6e-5b745dd8dd-nr58h" Jul 15 12:36:37.308025 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:36:37.307976 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6dba6e-5b745dd8dd-nr58h" podStartSLOduration=2.971675973 podStartE2EDuration="5.30796211s" totalImagesPullingTime="2.336286137s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:36:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:36:37.307634046 +0000 UTC m=+2268.765010972" watchObservedRunningTime="2026-07-15 12:36:37.30796211 +0000 UTC m=+2268.765339034" Jul 15 12:36:43.087329 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:36:43.087290 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:36:43.299908 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:36:43.299880 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6dba6e-5b745dd8dd-nr58h" Jul 15 12:36:49.353352 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:36:49.353317 2587 scope.go:117] "RemoveContainer" containerID="38757e2e03f3cd99b8fbe82deae094449237f09582760d22e32f7dc71789374d" Jul 15 12:36:53.988933 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:36:53.988898 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:37:04.978179 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:37:04.978141 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:37:15.385678 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:37:15.385640 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:37:28.277388 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:37:28.277354 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:37:31.926671 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:37:31.926630 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dbcba5-jbnvz"] Jul 15 12:37:31.930944 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:37:31.930922 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dbcba5-jbnvz" Jul 15 12:37:31.934319 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:37:31.934297 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tf4tp\"" Jul 15 12:37:31.936122 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:37:31.936096 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dbcba5-jbnvz"] Jul 15 12:37:32.031717 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:37:32.031687 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b69d907-3a2f-4bcb-94b3-a9ff0c62cd9a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-dbcba5-jbnvz\" (UID: \"9b69d907-3a2f-4bcb-94b3-a9ff0c62cd9a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dbcba5-jbnvz" Jul 15 12:37:32.132898 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:37:32.132863 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b69d907-3a2f-4bcb-94b3-a9ff0c62cd9a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-dbcba5-jbnvz\" (UID: \"9b69d907-3a2f-4bcb-94b3-a9ff0c62cd9a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dbcba5-jbnvz" Jul 15 12:37:32.133426 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:37:32.133408 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b69d907-3a2f-4bcb-94b3-a9ff0c62cd9a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-dbcba5-jbnvz\" (UID: \"9b69d907-3a2f-4bcb-94b3-a9ff0c62cd9a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dbcba5-jbnvz" Jul 15 12:37:32.244086 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:37:32.244016 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dbcba5-jbnvz" Jul 15 12:37:32.383069 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:37:32.383040 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dbcba5-jbnvz"] Jul 15 12:37:32.384883 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:37:32.384859 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b69d907_3a2f_4bcb_94b3_a9ff0c62cd9a.slice/crio-0102289b931f41b0a01fed031270c3d0477d8c0190770f38e18b1290390868f5 WatchSource:0}: Error finding container 0102289b931f41b0a01fed031270c3d0477d8c0190770f38e18b1290390868f5: Status 404 returned error can't find the container with id 0102289b931f41b0a01fed031270c3d0477d8c0190770f38e18b1290390868f5 Jul 15 12:37:32.590786 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:37:32.590754 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dbcba5-jbnvz" event={"ID":"9b69d907-3a2f-4bcb-94b3-a9ff0c62cd9a","Type":"ContainerStarted","Data":"0102289b931f41b0a01fed031270c3d0477d8c0190770f38e18b1290390868f5"} Jul 15 12:37:33.597809 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:37:33.597774 2587 generic.go:356] "Generic (PLEG): container finished" podID="9b69d907-3a2f-4bcb-94b3-a9ff0c62cd9a" containerID="a94c7057b9ce71e05a75bdaef9a12d95a78e2731d761c7def0de5a88088032fd" exitCode=0 Jul 15 12:37:33.598201 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:37:33.597845 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dbcba5-jbnvz" event={"ID":"9b69d907-3a2f-4bcb-94b3-a9ff0c62cd9a","Type":"ContainerDied","Data":"a94c7057b9ce71e05a75bdaef9a12d95a78e2731d761c7def0de5a88088032fd"} Jul 15 12:37:34.735470 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:37:34.735445 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dbcba5-jbnvz" Jul 15 12:37:34.754007 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:37:34.753977 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b69d907-3a2f-4bcb-94b3-a9ff0c62cd9a-maas-api-service-ca\") pod \"9b69d907-3a2f-4bcb-94b3-a9ff0c62cd9a\" (UID: \"9b69d907-3a2f-4bcb-94b3-a9ff0c62cd9a\") " Jul 15 12:37:34.754365 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:37:34.754338 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b69d907-3a2f-4bcb-94b3-a9ff0c62cd9a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9b69d907-3a2f-4bcb-94b3-a9ff0c62cd9a" (UID: "9b69d907-3a2f-4bcb-94b3-a9ff0c62cd9a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:37:34.855069 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:37:34.854992 2587 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b69d907-3a2f-4bcb-94b3-a9ff0c62cd9a-maas-api-service-ca\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:37:35.609889 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:37:35.609859 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dbcba5-jbnvz" Jul 15 12:37:35.609889 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:37:35.609870 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dbcba5-jbnvz" event={"ID":"9b69d907-3a2f-4bcb-94b3-a9ff0c62cd9a","Type":"ContainerDied","Data":"0102289b931f41b0a01fed031270c3d0477d8c0190770f38e18b1290390868f5"} Jul 15 12:37:35.610088 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:37:35.609921 2587 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0102289b931f41b0a01fed031270c3d0477d8c0190770f38e18b1290390868f5" Jul 15 12:37:45.282629 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:37:45.282594 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:38:08.468083 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:08.468052 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-6dba6e-5b745dd8dd-nr58h"] Jul 15 12:38:08.468542 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:08.468255 2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6dba6e-5b745dd8dd-nr58h" podUID="2874298c-fa1e-4cce-8c03-b643e6d33537" containerName="maas-api" containerID="cri-o://060b49a64ab6696417d1c2c28afa637aa583d089c7568a9f40e1cc728abc5f8c" gracePeriod=30 Jul 15 12:38:08.729667 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:08.729603 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6dba6e-5b745dd8dd-nr58h" Jul 15 12:38:08.789939 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:08.789909 2587 generic.go:356] "Generic (PLEG): container finished" podID="2874298c-fa1e-4cce-8c03-b643e6d33537" containerID="060b49a64ab6696417d1c2c28afa637aa583d089c7568a9f40e1cc728abc5f8c" exitCode=0 Jul 15 12:38:08.790098 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:08.789948 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6dba6e-5b745dd8dd-nr58h" Jul 15 12:38:08.790098 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:08.789985 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6dba6e-5b745dd8dd-nr58h" event={"ID":"2874298c-fa1e-4cce-8c03-b643e6d33537","Type":"ContainerDied","Data":"060b49a64ab6696417d1c2c28afa637aa583d089c7568a9f40e1cc728abc5f8c"} Jul 15 12:38:08.790098 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:08.790058 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6dba6e-5b745dd8dd-nr58h" event={"ID":"2874298c-fa1e-4cce-8c03-b643e6d33537","Type":"ContainerDied","Data":"35bb44cce236b4415c50b28ea4f26f2b282de129d945ddb88732c82e579a9a5b"} Jul 15 12:38:08.790098 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:08.790085 2587 scope.go:117] "RemoveContainer" containerID="060b49a64ab6696417d1c2c28afa637aa583d089c7568a9f40e1cc728abc5f8c" Jul 15 12:38:08.800630 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:08.800609 2587 scope.go:117] "RemoveContainer" containerID="060b49a64ab6696417d1c2c28afa637aa583d089c7568a9f40e1cc728abc5f8c" Jul 15 12:38:08.800967 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:38:08.800939 2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"060b49a64ab6696417d1c2c28afa637aa583d089c7568a9f40e1cc728abc5f8c\": container with ID starting with 060b49a64ab6696417d1c2c28afa637aa583d089c7568a9f40e1cc728abc5f8c not found: ID does not exist" containerID="060b49a64ab6696417d1c2c28afa637aa583d089c7568a9f40e1cc728abc5f8c" Jul 15 12:38:08.801080 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:08.800975 2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"060b49a64ab6696417d1c2c28afa637aa583d089c7568a9f40e1cc728abc5f8c"} err="failed to get container status \"060b49a64ab6696417d1c2c28afa637aa583d089c7568a9f40e1cc728abc5f8c\": rpc error: code = NotFound desc = could not find container \"060b49a64ab6696417d1c2c28afa637aa583d089c7568a9f40e1cc728abc5f8c\": container with ID starting with 060b49a64ab6696417d1c2c28afa637aa583d089c7568a9f40e1cc728abc5f8c not found: ID does not exist" Jul 15 12:38:08.838436 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:08.838388 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gn5fd\" (UniqueName: \"kubernetes.io/projected/2874298c-fa1e-4cce-8c03-b643e6d33537-kube-api-access-gn5fd\") pod \"2874298c-fa1e-4cce-8c03-b643e6d33537\" (UID: \"2874298c-fa1e-4cce-8c03-b643e6d33537\") " Jul 15 12:38:08.838631 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:08.838497 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2874298c-fa1e-4cce-8c03-b643e6d33537-openshift-service-ca\") pod \"2874298c-fa1e-4cce-8c03-b643e6d33537\" (UID: \"2874298c-fa1e-4cce-8c03-b643e6d33537\") " Jul 15 12:38:08.838631 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:08.838568 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2874298c-fa1e-4cce-8c03-b643e6d33537-maas-api-tls\") pod \"2874298c-fa1e-4cce-8c03-b643e6d33537\" (UID: \"2874298c-fa1e-4cce-8c03-b643e6d33537\") " Jul 15 12:38:08.838953 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:08.838927 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2874298c-fa1e-4cce-8c03-b643e6d33537-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2874298c-fa1e-4cce-8c03-b643e6d33537" (UID: "2874298c-fa1e-4cce-8c03-b643e6d33537"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:38:08.841055 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:08.841028 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2874298c-fa1e-4cce-8c03-b643e6d33537-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2874298c-fa1e-4cce-8c03-b643e6d33537" (UID: "2874298c-fa1e-4cce-8c03-b643e6d33537"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 12:38:08.841205 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:08.841186 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2874298c-fa1e-4cce-8c03-b643e6d33537-kube-api-access-gn5fd" (OuterVolumeSpecName: "kube-api-access-gn5fd") pod "2874298c-fa1e-4cce-8c03-b643e6d33537" (UID: "2874298c-fa1e-4cce-8c03-b643e6d33537"). InnerVolumeSpecName "kube-api-access-gn5fd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:38:08.940059 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:08.940032 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gn5fd\" (UniqueName: \"kubernetes.io/projected/2874298c-fa1e-4cce-8c03-b643e6d33537-kube-api-access-gn5fd\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:38:08.940059 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:08.940056 2587 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2874298c-fa1e-4cce-8c03-b643e6d33537-openshift-service-ca\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:38:08.940059 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:08.940066 2587 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2874298c-fa1e-4cce-8c03-b643e6d33537-maas-api-tls\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:38:09.112913 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:09.112884 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-6dba6e-5b745dd8dd-nr58h"] Jul 15 12:38:09.116629 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:09.116603 2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-6dba6e-5b745dd8dd-nr58h"] Jul 15 12:38:09.185448 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:09.185420 2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2874298c-fa1e-4cce-8c03-b643e6d33537" path="/var/lib/kubelet/pods/2874298c-fa1e-4cce-8c03-b643e6d33537/volumes" Jul 15 12:38:11.481179 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:11.481143 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:38:32.683621 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:32.683586 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:38:35.588929 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:35.588889 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-137ff6-595f65c858-kh9vv"] Jul 15 12:38:35.589502 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:35.589478 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2874298c-fa1e-4cce-8c03-b643e6d33537" containerName="maas-api" Jul 15 12:38:35.589628 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:35.589568 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="2874298c-fa1e-4cce-8c03-b643e6d33537" containerName="maas-api" Jul 15 12:38:35.589628 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:35.589617 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b69d907-3a2f-4bcb-94b3-a9ff0c62cd9a" containerName="revoke-keys" Jul 15 12:38:35.589628 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:35.589626 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b69d907-3a2f-4bcb-94b3-a9ff0c62cd9a" containerName="revoke-keys" Jul 15 12:38:35.589836 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:35.589772 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="2874298c-fa1e-4cce-8c03-b643e6d33537" containerName="maas-api" Jul 15 12:38:35.589836 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:35.589788 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b69d907-3a2f-4bcb-94b3-a9ff0c62cd9a" containerName="revoke-keys" Jul 15 12:38:35.592952 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:35.592930 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-137ff6-595f65c858-kh9vv" Jul 15 12:38:35.595575 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:35.595540 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-137ff6-serving-cert\"" Jul 15 12:38:35.595776 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:35.595751 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-hgh4d\"" Jul 15 12:38:35.595910 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:35.595663 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 12:38:35.596968 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:35.596945 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 15 12:38:35.604468 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:35.604447 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-137ff6-595f65c858-kh9vv"] Jul 15 12:38:35.670964 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:35.670923 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9c9mp\" (UniqueName: \"kubernetes.io/projected/8de545db-0883-406e-9e83-518106cff937-kube-api-access-9c9mp\") pod \"maas-api-e2e-api-b-137ff6-595f65c858-kh9vv\" (UID: \"8de545db-0883-406e-9e83-518106cff937\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-137ff6-595f65c858-kh9vv" Jul 15 12:38:35.671152 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:35.670972 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8de545db-0883-406e-9e83-518106cff937-openshift-service-ca\") pod \"maas-api-e2e-api-b-137ff6-595f65c858-kh9vv\" (UID: \"8de545db-0883-406e-9e83-518106cff937\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-137ff6-595f65c858-kh9vv" Jul 15 12:38:35.671152 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:35.671007 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8de545db-0883-406e-9e83-518106cff937-maas-api-tls\") pod \"maas-api-e2e-api-b-137ff6-595f65c858-kh9vv\" (UID: \"8de545db-0883-406e-9e83-518106cff937\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-137ff6-595f65c858-kh9vv" Jul 15 12:38:35.772182 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:35.772151 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9c9mp\" (UniqueName: \"kubernetes.io/projected/8de545db-0883-406e-9e83-518106cff937-kube-api-access-9c9mp\") pod \"maas-api-e2e-api-b-137ff6-595f65c858-kh9vv\" (UID: \"8de545db-0883-406e-9e83-518106cff937\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-137ff6-595f65c858-kh9vv" Jul 15 12:38:35.772182 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:35.772193 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8de545db-0883-406e-9e83-518106cff937-openshift-service-ca\") pod \"maas-api-e2e-api-b-137ff6-595f65c858-kh9vv\" (UID: \"8de545db-0883-406e-9e83-518106cff937\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-137ff6-595f65c858-kh9vv" Jul 15 12:38:35.772424 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:35.772324 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8de545db-0883-406e-9e83-518106cff937-maas-api-tls\") pod \"maas-api-e2e-api-b-137ff6-595f65c858-kh9vv\" (UID: \"8de545db-0883-406e-9e83-518106cff937\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-137ff6-595f65c858-kh9vv" Jul 15 12:38:35.772956 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:35.772923 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8de545db-0883-406e-9e83-518106cff937-openshift-service-ca\") pod \"maas-api-e2e-api-b-137ff6-595f65c858-kh9vv\" (UID: \"8de545db-0883-406e-9e83-518106cff937\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-137ff6-595f65c858-kh9vv" Jul 15 12:38:35.775223 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:35.775201 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8de545db-0883-406e-9e83-518106cff937-maas-api-tls\") pod \"maas-api-e2e-api-b-137ff6-595f65c858-kh9vv\" (UID: \"8de545db-0883-406e-9e83-518106cff937\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-137ff6-595f65c858-kh9vv" Jul 15 12:38:35.781530 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:35.781508 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9c9mp\" (UniqueName: \"kubernetes.io/projected/8de545db-0883-406e-9e83-518106cff937-kube-api-access-9c9mp\") pod \"maas-api-e2e-api-b-137ff6-595f65c858-kh9vv\" (UID: \"8de545db-0883-406e-9e83-518106cff937\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-137ff6-595f65c858-kh9vv" Jul 15 12:38:35.906812 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:35.906713 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-137ff6-595f65c858-kh9vv" Jul 15 12:38:36.042141 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:36.042113 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-137ff6-595f65c858-kh9vv"] Jul 15 12:38:36.043882 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:38:36.043857 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8de545db_0883_406e_9e83_518106cff937.slice/crio-8e5dce4b6ae24c6794c66a230f780aec05626455dd56a09f120e54fa5f801a00 WatchSource:0}: Error finding container 8e5dce4b6ae24c6794c66a230f780aec05626455dd56a09f120e54fa5f801a00: Status 404 returned error can't find the container with id 8e5dce4b6ae24c6794c66a230f780aec05626455dd56a09f120e54fa5f801a00 Jul 15 12:38:36.961044 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:36.961009 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-137ff6-595f65c858-kh9vv" event={"ID":"8de545db-0883-406e-9e83-518106cff937","Type":"ContainerStarted","Data":"9deabc73ece72cc51b0c6e4dca21904aa9b95853fa70164f3928660ab5fc447a"} Jul 15 12:38:36.961445 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:36.961074 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-137ff6-595f65c858-kh9vv" event={"ID":"8de545db-0883-406e-9e83-518106cff937","Type":"ContainerStarted","Data":"8e5dce4b6ae24c6794c66a230f780aec05626455dd56a09f120e54fa5f801a00"} Jul 15 12:38:36.961445 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:36.961101 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-137ff6-595f65c858-kh9vv" Jul 15 12:38:36.979836 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:36.979781 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-137ff6-595f65c858-kh9vv" podStartSLOduration=1.573798618 podStartE2EDuration="1.979761352s" totalImagesPullingTime="405.962734ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:38:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:38:36.977224585 +0000 UTC m=+2388.434601534" watchObservedRunningTime="2026-07-15 12:38:36.979761352 +0000 UTC m=+2388.437138279" Jul 15 12:38:42.972228 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:42.972196 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-137ff6-595f65c858-kh9vv" Jul 15 12:38:44.278788 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:44.278753 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:38:48.144956 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:48.144920 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-858a49-mxgwk"] Jul 15 12:38:48.148810 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:48.148790 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-858a49-mxgwk" Jul 15 12:38:48.151647 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:48.151629 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tf4tp\"" Jul 15 12:38:48.156374 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:48.156351 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-858a49-mxgwk"] Jul 15 12:38:48.287105 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:48.287066 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd5f415f-a6aa-4c9b-8537-54130aae80b9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-858a49-mxgwk\" (UID: \"dd5f415f-a6aa-4c9b-8537-54130aae80b9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-858a49-mxgwk" Jul 15 12:38:48.387773 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:48.387715 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd5f415f-a6aa-4c9b-8537-54130aae80b9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-858a49-mxgwk\" (UID: \"dd5f415f-a6aa-4c9b-8537-54130aae80b9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-858a49-mxgwk" Jul 15 12:38:48.388343 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:48.388322 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd5f415f-a6aa-4c9b-8537-54130aae80b9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-858a49-mxgwk\" (UID: \"dd5f415f-a6aa-4c9b-8537-54130aae80b9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-858a49-mxgwk" Jul 15 12:38:48.460783 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:48.460618 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-858a49-mxgwk" Jul 15 12:38:48.594614 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:48.594588 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-858a49-mxgwk"] Jul 15 12:38:48.596066 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:38:48.596034 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd5f415f_a6aa_4c9b_8537_54130aae80b9.slice/crio-bb2aeac5a4ce49036837cb7c9232d44f5e05e12d985acd14424feca05b08ef23 WatchSource:0}: Error finding container bb2aeac5a4ce49036837cb7c9232d44f5e05e12d985acd14424feca05b08ef23: Status 404 returned error can't find the container with id bb2aeac5a4ce49036837cb7c9232d44f5e05e12d985acd14424feca05b08ef23 Jul 15 12:38:49.033486 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:49.033453 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-858a49-mxgwk" event={"ID":"dd5f415f-a6aa-4c9b-8537-54130aae80b9","Type":"ContainerStarted","Data":"645c751a9de2110e631c85f35bbc5c4ba1d8fe566a9158225f3841af5105fb45"} Jul 15 12:38:49.033666 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:49.033511 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-858a49-mxgwk" event={"ID":"dd5f415f-a6aa-4c9b-8537-54130aae80b9","Type":"ContainerStarted","Data":"bb2aeac5a4ce49036837cb7c9232d44f5e05e12d985acd14424feca05b08ef23"} Jul 15 12:38:49.050842 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:49.050796 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-858a49-mxgwk" podStartSLOduration=1.05078061 podStartE2EDuration="1.05078061s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:38:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:38:49.049289424 +0000 UTC m=+2400.506666362" watchObservedRunningTime="2026-07-15 12:38:49.05078061 +0000 UTC m=+2400.508157588" Jul 15 12:38:49.278654 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:49.278577 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-4xrn6_85ce5adf-d361-4258-b4ca-7f13d35fe13d/console-operator/1.log" Jul 15 12:38:49.284557 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:49.284533 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mcrkx_ea415947-4742-4cf7-8b71-e22b418a131f/ovn-acl-logging/0.log" Jul 15 12:38:49.285499 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:49.285477 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-4xrn6_85ce5adf-d361-4258-b4ca-7f13d35fe13d/console-operator/1.log" Jul 15 12:38:49.291195 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:49.291177 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mcrkx_ea415947-4742-4cf7-8b71-e22b418a131f/ovn-acl-logging/0.log" Jul 15 12:38:50.040513 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:50.040476 2587 generic.go:356] "Generic (PLEG): container finished" podID="dd5f415f-a6aa-4c9b-8537-54130aae80b9" containerID="645c751a9de2110e631c85f35bbc5c4ba1d8fe566a9158225f3841af5105fb45" exitCode=0 Jul 15 12:38:50.040700 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:50.040526 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-858a49-mxgwk" event={"ID":"dd5f415f-a6aa-4c9b-8537-54130aae80b9","Type":"ContainerDied","Data":"645c751a9de2110e631c85f35bbc5c4ba1d8fe566a9158225f3841af5105fb45"} Jul 15 12:38:51.183682 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:51.183655 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-858a49-mxgwk" Jul 15 12:38:51.310935 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:51.310858 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd5f415f-a6aa-4c9b-8537-54130aae80b9-maas-api-service-ca\") pod \"dd5f415f-a6aa-4c9b-8537-54130aae80b9\" (UID: \"dd5f415f-a6aa-4c9b-8537-54130aae80b9\") " Jul 15 12:38:51.311182 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:51.311156 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd5f415f-a6aa-4c9b-8537-54130aae80b9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "dd5f415f-a6aa-4c9b-8537-54130aae80b9" (UID: "dd5f415f-a6aa-4c9b-8537-54130aae80b9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:38:51.311340 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:51.311324 2587 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd5f415f-a6aa-4c9b-8537-54130aae80b9-maas-api-service-ca\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:38:52.057499 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:52.057471 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-858a49-mxgwk" Jul 15 12:38:52.057690 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:52.057464 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-858a49-mxgwk" event={"ID":"dd5f415f-a6aa-4c9b-8537-54130aae80b9","Type":"ContainerDied","Data":"bb2aeac5a4ce49036837cb7c9232d44f5e05e12d985acd14424feca05b08ef23"} Jul 15 12:38:52.057690 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:38:52.057592 2587 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bb2aeac5a4ce49036837cb7c9232d44f5e05e12d985acd14424feca05b08ef23" Jul 15 12:39:01.384075 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:39:01.384033 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:39:24.731988 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:39:24.731880 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-137ff6-595f65c858-kh9vv"] Jul 15 12:39:24.732556 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:39:24.732103 2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-137ff6-595f65c858-kh9vv" podUID="8de545db-0883-406e-9e83-518106cff937" containerName="maas-api" containerID="cri-o://9deabc73ece72cc51b0c6e4dca21904aa9b95853fa70164f3928660ab5fc447a" gracePeriod=30 Jul 15 12:39:24.995528 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:39:24.995468 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-137ff6-595f65c858-kh9vv" Jul 15 12:39:25.108805 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:39:25.108771 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9c9mp\" (UniqueName: \"kubernetes.io/projected/8de545db-0883-406e-9e83-518106cff937-kube-api-access-9c9mp\") pod \"8de545db-0883-406e-9e83-518106cff937\" (UID: \"8de545db-0883-406e-9e83-518106cff937\") " Jul 15 12:39:25.108992 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:39:25.108819 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8de545db-0883-406e-9e83-518106cff937-openshift-service-ca\") pod \"8de545db-0883-406e-9e83-518106cff937\" (UID: \"8de545db-0883-406e-9e83-518106cff937\") " Jul 15 12:39:25.108992 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:39:25.108843 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8de545db-0883-406e-9e83-518106cff937-maas-api-tls\") pod \"8de545db-0883-406e-9e83-518106cff937\" (UID: \"8de545db-0883-406e-9e83-518106cff937\") " Jul 15 12:39:25.109151 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:39:25.109128 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8de545db-0883-406e-9e83-518106cff937-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8de545db-0883-406e-9e83-518106cff937" (UID: "8de545db-0883-406e-9e83-518106cff937"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:39:25.110918 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:39:25.110884 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8de545db-0883-406e-9e83-518106cff937-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8de545db-0883-406e-9e83-518106cff937" (UID: "8de545db-0883-406e-9e83-518106cff937"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 12:39:25.111021 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:39:25.110957 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8de545db-0883-406e-9e83-518106cff937-kube-api-access-9c9mp" (OuterVolumeSpecName: "kube-api-access-9c9mp") pod "8de545db-0883-406e-9e83-518106cff937" (UID: "8de545db-0883-406e-9e83-518106cff937"). InnerVolumeSpecName "kube-api-access-9c9mp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:39:25.209543 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:39:25.209516 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9c9mp\" (UniqueName: \"kubernetes.io/projected/8de545db-0883-406e-9e83-518106cff937-kube-api-access-9c9mp\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:39:25.209543 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:39:25.209542 2587 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8de545db-0883-406e-9e83-518106cff937-openshift-service-ca\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:39:25.209714 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:39:25.209553 2587 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8de545db-0883-406e-9e83-518106cff937-maas-api-tls\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:39:25.240432 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:39:25.240406 2587 generic.go:356] "Generic (PLEG): container finished" podID="8de545db-0883-406e-9e83-518106cff937" containerID="9deabc73ece72cc51b0c6e4dca21904aa9b95853fa70164f3928660ab5fc447a" exitCode=0 Jul 15 12:39:25.240578 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:39:25.240462 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-137ff6-595f65c858-kh9vv" Jul 15 12:39:25.240578 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:39:25.240495 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-137ff6-595f65c858-kh9vv" event={"ID":"8de545db-0883-406e-9e83-518106cff937","Type":"ContainerDied","Data":"9deabc73ece72cc51b0c6e4dca21904aa9b95853fa70164f3928660ab5fc447a"} Jul 15 12:39:25.240578 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:39:25.240553 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-137ff6-595f65c858-kh9vv" event={"ID":"8de545db-0883-406e-9e83-518106cff937","Type":"ContainerDied","Data":"8e5dce4b6ae24c6794c66a230f780aec05626455dd56a09f120e54fa5f801a00"} Jul 15 12:39:25.240707 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:39:25.240583 2587 scope.go:117] "RemoveContainer" containerID="9deabc73ece72cc51b0c6e4dca21904aa9b95853fa70164f3928660ab5fc447a" Jul 15 12:39:25.252946 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:39:25.252925 2587 scope.go:117] "RemoveContainer" containerID="9deabc73ece72cc51b0c6e4dca21904aa9b95853fa70164f3928660ab5fc447a" Jul 15 12:39:25.253209 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:39:25.253188 2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9deabc73ece72cc51b0c6e4dca21904aa9b95853fa70164f3928660ab5fc447a\": container with ID starting with 9deabc73ece72cc51b0c6e4dca21904aa9b95853fa70164f3928660ab5fc447a not found: ID does not exist" containerID="9deabc73ece72cc51b0c6e4dca21904aa9b95853fa70164f3928660ab5fc447a" Jul 15 12:39:25.253261 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:39:25.253217 2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9deabc73ece72cc51b0c6e4dca21904aa9b95853fa70164f3928660ab5fc447a"} err="failed to get container status \"9deabc73ece72cc51b0c6e4dca21904aa9b95853fa70164f3928660ab5fc447a\": rpc error: code = NotFound desc = could not find container \"9deabc73ece72cc51b0c6e4dca21904aa9b95853fa70164f3928660ab5fc447a\": container with ID starting with 9deabc73ece72cc51b0c6e4dca21904aa9b95853fa70164f3928660ab5fc447a not found: ID does not exist" Jul 15 12:39:25.260021 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:39:25.259997 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-137ff6-595f65c858-kh9vv"] Jul 15 12:39:25.265937 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:39:25.265915 2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-137ff6-595f65c858-kh9vv"] Jul 15 12:39:27.185035 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:39:27.185002 2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8de545db-0883-406e-9e83-518106cff937" path="/var/lib/kubelet/pods/8de545db-0883-406e-9e83-518106cff937/volumes" Jul 15 12:39:27.783006 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:39:27.782960 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:39:51.675111 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:39:51.675080 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:39:52.134706 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:39:52.134633 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-b2e819-7c74db7fdf-f6294"] Jul 15 12:39:52.135158 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:39:52.135136 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8de545db-0883-406e-9e83-518106cff937" containerName="maas-api" Jul 15 12:39:52.135249 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:39:52.135160 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="8de545db-0883-406e-9e83-518106cff937" containerName="maas-api" Jul 15 12:39:52.135249 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:39:52.135180 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd5f415f-a6aa-4c9b-8537-54130aae80b9" containerName="revoke-keys" Jul 15 12:39:52.135249 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:39:52.135189 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd5f415f-a6aa-4c9b-8537-54130aae80b9" containerName="revoke-keys" Jul 15 12:39:52.135413 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:39:52.135292 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="8de545db-0883-406e-9e83-518106cff937" containerName="maas-api" Jul 15 12:39:52.135413 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:39:52.135306 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd5f415f-a6aa-4c9b-8537-54130aae80b9" containerName="revoke-keys" Jul 15 12:39:52.139611 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:39:52.139587 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b2e819-7c74db7fdf-f6294" Jul 15 12:39:52.142488 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:39:52.142460 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-b2e819-serving-cert\"" Jul 15 12:39:52.142593 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:39:52.142547 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 12:39:52.142593 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:39:52.142566 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 15 12:39:52.142855 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:39:52.142841 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-hgh4d\"" Jul 15 12:39:52.148409 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:39:52.148390 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-b2e819-7c74db7fdf-f6294"] Jul 15 12:39:52.245770 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:39:52.245716 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f17da08-ffa5-453f-a1c3-c9bbbbb8b73a-maas-api-tls\") pod \"maas-api-e2e-shared-b-b2e819-7c74db7fdf-f6294\" (UID: \"5f17da08-ffa5-453f-a1c3-c9bbbbb8b73a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b2e819-7c74db7fdf-f6294" Jul 15 12:39:52.245932 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:39:52.245894 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f17da08-ffa5-453f-a1c3-c9bbbbb8b73a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-b2e819-7c74db7fdf-f6294\" (UID: \"5f17da08-ffa5-453f-a1c3-c9bbbbb8b73a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b2e819-7c74db7fdf-f6294" Jul 15 12:39:52.245985 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:39:52.245944 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqcgz\" (UniqueName: \"kubernetes.io/projected/5f17da08-ffa5-453f-a1c3-c9bbbbb8b73a-kube-api-access-pqcgz\") pod \"maas-api-e2e-shared-b-b2e819-7c74db7fdf-f6294\" (UID: \"5f17da08-ffa5-453f-a1c3-c9bbbbb8b73a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b2e819-7c74db7fdf-f6294" Jul 15 12:39:52.346752 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:39:52.346695 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqcgz\" (UniqueName: \"kubernetes.io/projected/5f17da08-ffa5-453f-a1c3-c9bbbbb8b73a-kube-api-access-pqcgz\") pod \"maas-api-e2e-shared-b-b2e819-7c74db7fdf-f6294\" (UID: \"5f17da08-ffa5-453f-a1c3-c9bbbbb8b73a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b2e819-7c74db7fdf-f6294" Jul 15 12:39:52.346941 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:39:52.346806 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f17da08-ffa5-453f-a1c3-c9bbbbb8b73a-maas-api-tls\") pod \"maas-api-e2e-shared-b-b2e819-7c74db7fdf-f6294\" (UID: \"5f17da08-ffa5-453f-a1c3-c9bbbbb8b73a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b2e819-7c74db7fdf-f6294" Jul 15 12:39:52.346941 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:39:52.346920 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f17da08-ffa5-453f-a1c3-c9bbbbb8b73a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-b2e819-7c74db7fdf-f6294\" (UID: \"5f17da08-ffa5-453f-a1c3-c9bbbbb8b73a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b2e819-7c74db7fdf-f6294" Jul 15 12:39:52.347066 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:39:52.346959 2587 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-b2e819-serving-cert: secret "maas-api-e2e-shared-b-b2e819-serving-cert" not found Jul 15 12:39:52.347066 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:39:52.347043 2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f17da08-ffa5-453f-a1c3-c9bbbbb8b73a-maas-api-tls podName:5f17da08-ffa5-453f-a1c3-c9bbbbb8b73a nodeName:}" failed. No retries permitted until 2026-07-15 12:39:52.84701964 +0000 UTC m=+2464.304396554 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5f17da08-ffa5-453f-a1c3-c9bbbbb8b73a-maas-api-tls") pod "maas-api-e2e-shared-b-b2e819-7c74db7fdf-f6294" (UID: "5f17da08-ffa5-453f-a1c3-c9bbbbb8b73a") : secret "maas-api-e2e-shared-b-b2e819-serving-cert" not found Jul 15 12:39:52.347536 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:39:52.347516 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f17da08-ffa5-453f-a1c3-c9bbbbb8b73a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-b2e819-7c74db7fdf-f6294\" (UID: \"5f17da08-ffa5-453f-a1c3-c9bbbbb8b73a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b2e819-7c74db7fdf-f6294" Jul 15 12:39:52.357001 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:39:52.356981 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqcgz\" (UniqueName: \"kubernetes.io/projected/5f17da08-ffa5-453f-a1c3-c9bbbbb8b73a-kube-api-access-pqcgz\") pod \"maas-api-e2e-shared-b-b2e819-7c74db7fdf-f6294\" (UID: \"5f17da08-ffa5-453f-a1c3-c9bbbbb8b73a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b2e819-7c74db7fdf-f6294" Jul 15 12:39:52.851200 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:39:52.851159 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f17da08-ffa5-453f-a1c3-c9bbbbb8b73a-maas-api-tls\") pod \"maas-api-e2e-shared-b-b2e819-7c74db7fdf-f6294\" (UID: \"5f17da08-ffa5-453f-a1c3-c9bbbbb8b73a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b2e819-7c74db7fdf-f6294" Jul 15 12:39:52.853537 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:39:52.853512 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f17da08-ffa5-453f-a1c3-c9bbbbb8b73a-maas-api-tls\") pod \"maas-api-e2e-shared-b-b2e819-7c74db7fdf-f6294\" (UID: \"5f17da08-ffa5-453f-a1c3-c9bbbbb8b73a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b2e819-7c74db7fdf-f6294" Jul 15 12:39:53.051792 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:39:53.051765 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b2e819-7c74db7fdf-f6294" Jul 15 12:39:53.252001 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:39:53.251971 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-b2e819-7c74db7fdf-f6294"] Jul 15 12:39:53.253324 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:39:53.253298 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f17da08_ffa5_453f_a1c3_c9bbbbb8b73a.slice/crio-6495cf8a439ab4f897a3ec5e17e7d5905deb7a8130e432ef546c169cf44d616d WatchSource:0}: Error finding container 6495cf8a439ab4f897a3ec5e17e7d5905deb7a8130e432ef546c169cf44d616d: Status 404 returned error can't find the container with id 6495cf8a439ab4f897a3ec5e17e7d5905deb7a8130e432ef546c169cf44d616d Jul 15 12:39:53.399485 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:39:53.399404 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b2e819-7c74db7fdf-f6294" event={"ID":"5f17da08-ffa5-453f-a1c3-c9bbbbb8b73a","Type":"ContainerStarted","Data":"6495cf8a439ab4f897a3ec5e17e7d5905deb7a8130e432ef546c169cf44d616d"} Jul 15 12:39:54.412836 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:39:54.412794 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b2e819-7c74db7fdf-f6294" event={"ID":"5f17da08-ffa5-453f-a1c3-c9bbbbb8b73a","Type":"ContainerStarted","Data":"e8923ba8b48596398783a46efb9f7cbd5a958fe3afce13578748dad0b419db62"} Jul 15 12:39:54.413272 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:39:54.412889 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b2e819-7c74db7fdf-f6294" Jul 15 12:39:54.432272 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:39:54.432223 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b2e819-7c74db7fdf-f6294" podStartSLOduration=2.090958859 podStartE2EDuration="2.432209538s" totalImagesPullingTime="341.250679ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:39:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:39:54.430235305 +0000 UTC m=+2465.887612241" watchObservedRunningTime="2026-07-15 12:39:54.432209538 +0000 UTC m=+2465.889586463" Jul 15 12:40:00.424297 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:40:00.424261 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b2e819-7c74db7fdf-f6294" Jul 15 12:40:02.477847 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:40:02.477811 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:40:25.881635 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:40:25.881601 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:41:01.477013 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:41:01.476981 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:41:18.283858 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:41:18.283820 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:41:36.489445 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:41:36.489412 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:41:52.481282 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:41:52.481246 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:42:11.878074 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:42:11.877999 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:42:29.076838 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:42:29.076805 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:42:42.979384 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:42:42.979350 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:43:00.982256 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:43:00.982220 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:43:11.889530 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:43:11.889491 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:43:29.880821 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:43:29.880787 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:43:49.313489 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:43:49.313408 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-4xrn6_85ce5adf-d361-4258-b4ca-7f13d35fe13d/console-operator/1.log" Jul 15 12:43:49.319325 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:43:49.319302 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mcrkx_ea415947-4742-4cf7-8b71-e22b418a131f/ovn-acl-logging/0.log" Jul 15 12:43:49.322247 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:43:49.322228 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-4xrn6_85ce5adf-d361-4258-b4ca-7f13d35fe13d/console-operator/1.log" Jul 15 12:43:49.328660 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:43:49.328637 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mcrkx_ea415947-4742-4cf7-8b71-e22b418a131f/ovn-acl-logging/0.log" Jul 15 12:43:50.379876 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:43:50.379847 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:44:09.786666 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:44:09.786631 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:44:22.183011 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:44:22.182981 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:45:00.138761 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:00.138665 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-b2e819-29735325-xbplr"] Jul 15 12:45:00.142657 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:00.142638 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a9c7da-29735325-6m8s7"] Jul 15 12:45:00.142798 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:00.142779 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-b2e819-29735325-xbplr" Jul 15 12:45:00.145359 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:00.145341 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tf4tp\"" Jul 15 12:45:00.146608 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:00.146593 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735325-6c2f8"] Jul 15 12:45:00.146783 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:00.146760 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a9c7da-29735325-6m8s7" Jul 15 12:45:00.152043 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:00.151691 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-b2e819-29735325-xbplr"] Jul 15 12:45:00.152043 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:00.151827 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735325-6c2f8" Jul 15 12:45:00.155109 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:00.155055 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a9c7da-29735325-6m8s7"] Jul 15 12:45:00.164748 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:00.157620 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-b2e819-29735325-xbplr" Jul 15 12:45:00.164748 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:00.160795 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735325-6c2f8"] Jul 15 12:45:00.196250 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:00.196225 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a9c7da-29735325-6m8s7" Jul 15 12:45:00.202658 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:00.202589 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735325-6c2f8" Jul 15 12:45:00.352167 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:00.352127 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-b2e819-29735325-xbplr"] Jul 15 12:45:00.353394 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:45:00.353354 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb61bff8b_1999_4d29_9791_935bf444685f.slice/crio-95b5d84ae8c5b513688e5eb3b14828da959428d996f10a1d960934993e8346f3 WatchSource:0}: Error finding container 95b5d84ae8c5b513688e5eb3b14828da959428d996f10a1d960934993e8346f3: Status 404 returned error can't find the container with id 95b5d84ae8c5b513688e5eb3b14828da959428d996f10a1d960934993e8346f3 Jul 15 12:45:00.389150 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:00.389117 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a9c7da-29735325-6m8s7"] Jul 15 12:45:00.389816 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:45:00.389793 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc664f563_6f22_4e6a_8858_69c9b30f266b.slice/crio-47f654d849717284329aaa38547b5b6a220b9cd063419521c27a2189d35fba3d WatchSource:0}: Error finding container 47f654d849717284329aaa38547b5b6a220b9cd063419521c27a2189d35fba3d: Status 404 returned error can't find the container with id 47f654d849717284329aaa38547b5b6a220b9cd063419521c27a2189d35fba3d Jul 15 12:45:00.399668 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:00.399645 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735325-6c2f8"] Jul 15 12:45:00.402633 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:45:00.402606 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod553495ae_bebd_4a85_a12d_13f498d024ab.slice/crio-e7656c5559cb74c355b5d267894d44ba7e4e13219588d3f781b547e939a48831 WatchSource:0}: Error finding container e7656c5559cb74c355b5d267894d44ba7e4e13219588d3f781b547e939a48831: Status 404 returned error can't find the container with id e7656c5559cb74c355b5d267894d44ba7e4e13219588d3f781b547e939a48831 Jul 15 12:45:01.135411 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:01.135367 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a9c7da-29735325-6m8s7" event={"ID":"c664f563-6f22-4e6a-8858-69c9b30f266b","Type":"ContainerStarted","Data":"c9142074dfa09fa01d720cd33990cfbd7439a7f029fefc255180620956d9c7c7"} Jul 15 12:45:01.135600 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:01.135423 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a9c7da-29735325-6m8s7" event={"ID":"c664f563-6f22-4e6a-8858-69c9b30f266b","Type":"ContainerStarted","Data":"47f654d849717284329aaa38547b5b6a220b9cd063419521c27a2189d35fba3d"} Jul 15 12:45:01.136965 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:01.136937 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-b2e819-29735325-xbplr" event={"ID":"b61bff8b-1999-4d29-9791-935bf444685f","Type":"ContainerStarted","Data":"68ef092dfea39b3288c85d425ec88de2eb9a06daa1081ebe61c90b27dd7410a8"} Jul 15 12:45:01.137092 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:01.136989 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-b2e819-29735325-xbplr" event={"ID":"b61bff8b-1999-4d29-9791-935bf444685f","Type":"ContainerStarted","Data":"95b5d84ae8c5b513688e5eb3b14828da959428d996f10a1d960934993e8346f3"} Jul 15 12:45:01.138430 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:01.138407 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735325-6c2f8" event={"ID":"553495ae-bebd-4a85-a12d-13f498d024ab","Type":"ContainerStarted","Data":"e5ad54e88068169fc20b71850259c716855cf6ad0365a8751aebcce7fe392fc0"} Jul 15 12:45:01.138555 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:01.138449 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735325-6c2f8" event={"ID":"553495ae-bebd-4a85-a12d-13f498d024ab","Type":"ContainerStarted","Data":"e7656c5559cb74c355b5d267894d44ba7e4e13219588d3f781b547e939a48831"} Jul 15 12:45:01.153417 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:01.153379 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a9c7da-29735325-6m8s7" podStartSLOduration=1.153365669 podStartE2EDuration="1.153365669s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:45:01.152136733 +0000 UTC m=+2772.609513660" watchObservedRunningTime="2026-07-15 12:45:01.153365669 +0000 UTC m=+2772.610742571" Jul 15 12:45:01.166435 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:01.166395 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-b2e819-29735325-xbplr" podStartSLOduration=1.166381622 podStartE2EDuration="1.166381622s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:45:01.165184859 +0000 UTC m=+2772.622561785" watchObservedRunningTime="2026-07-15 12:45:01.166381622 +0000 UTC m=+2772.623758541" Jul 15 12:45:01.180694 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:01.180656 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735325-6c2f8" podStartSLOduration=1.180644225 podStartE2EDuration="1.180644225s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:45:01.178903928 +0000 UTC m=+2772.636280855" watchObservedRunningTime="2026-07-15 12:45:01.180644225 +0000 UTC m=+2772.638021150" Jul 15 12:45:02.146475 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:02.146439 2587 generic.go:356] "Generic (PLEG): container finished" podID="b61bff8b-1999-4d29-9791-935bf444685f" containerID="68ef092dfea39b3288c85d425ec88de2eb9a06daa1081ebe61c90b27dd7410a8" exitCode=0 Jul 15 12:45:02.146767 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:02.146525 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-b2e819-29735325-xbplr" event={"ID":"b61bff8b-1999-4d29-9791-935bf444685f","Type":"ContainerDied","Data":"68ef092dfea39b3288c85d425ec88de2eb9a06daa1081ebe61c90b27dd7410a8"} Jul 15 12:45:02.148081 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:02.148050 2587 generic.go:356] "Generic (PLEG): container finished" podID="553495ae-bebd-4a85-a12d-13f498d024ab" containerID="e5ad54e88068169fc20b71850259c716855cf6ad0365a8751aebcce7fe392fc0" exitCode=0 Jul 15 12:45:02.148174 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:02.148079 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735325-6c2f8" event={"ID":"553495ae-bebd-4a85-a12d-13f498d024ab","Type":"ContainerDied","Data":"e5ad54e88068169fc20b71850259c716855cf6ad0365a8751aebcce7fe392fc0"} Jul 15 12:45:02.149434 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:02.149416 2587 generic.go:356] "Generic (PLEG): container finished" podID="c664f563-6f22-4e6a-8858-69c9b30f266b" containerID="c9142074dfa09fa01d720cd33990cfbd7439a7f029fefc255180620956d9c7c7" exitCode=0 Jul 15 12:45:02.149509 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:02.149441 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a9c7da-29735325-6m8s7" event={"ID":"c664f563-6f22-4e6a-8858-69c9b30f266b","Type":"ContainerDied","Data":"c9142074dfa09fa01d720cd33990cfbd7439a7f029fefc255180620956d9c7c7"} Jul 15 12:45:03.309287 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:03.309266 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735325-6c2f8" Jul 15 12:45:03.353157 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:03.353104 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-b2e819-29735325-xbplr" Jul 15 12:45:03.356419 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:03.356393 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a9c7da-29735325-6m8s7" Jul 15 12:45:04.163216 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:04.163174 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a9c7da-29735325-6m8s7" Jul 15 12:45:04.163216 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:04.163211 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a9c7da-29735325-6m8s7" event={"ID":"c664f563-6f22-4e6a-8858-69c9b30f266b","Type":"ContainerDied","Data":"47f654d849717284329aaa38547b5b6a220b9cd063419521c27a2189d35fba3d"} Jul 15 12:45:04.163450 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:04.163286 2587 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="47f654d849717284329aaa38547b5b6a220b9cd063419521c27a2189d35fba3d" Jul 15 12:45:04.164608 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:04.164588 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-b2e819-29735325-xbplr" Jul 15 12:45:04.164719 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:04.164636 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-b2e819-29735325-xbplr" event={"ID":"b61bff8b-1999-4d29-9791-935bf444685f","Type":"ContainerDied","Data":"95b5d84ae8c5b513688e5eb3b14828da959428d996f10a1d960934993e8346f3"} Jul 15 12:45:04.164719 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:04.164682 2587 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="95b5d84ae8c5b513688e5eb3b14828da959428d996f10a1d960934993e8346f3" Jul 15 12:45:04.165994 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:04.165971 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735325-6c2f8" event={"ID":"553495ae-bebd-4a85-a12d-13f498d024ab","Type":"ContainerDied","Data":"e7656c5559cb74c355b5d267894d44ba7e4e13219588d3f781b547e939a48831"} Jul 15 12:45:04.165994 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:04.165987 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735325-6c2f8" Jul 15 12:45:04.166133 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:04.166010 2587 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e7656c5559cb74c355b5d267894d44ba7e4e13219588d3f781b547e939a48831" Jul 15 12:45:04.593711 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:04.593677 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:45:25.289944 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:25.289862 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:45:42.091353 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:42.091315 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:45:48.923287 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:48.923251 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7615-140094018573744"] Jul 15 12:45:48.923653 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:48.923624 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c664f563-6f22-4e6a-8858-69c9b30f266b" containerName="cleanup" Jul 15 12:45:48.923653 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:48.923635 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="c664f563-6f22-4e6a-8858-69c9b30f266b" containerName="cleanup" Jul 15 12:45:48.923653 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:48.923655 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="553495ae-bebd-4a85-a12d-13f498d024ab" containerName="cleanup" Jul 15 12:45:48.923784 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:48.923660 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="553495ae-bebd-4a85-a12d-13f498d024ab" containerName="cleanup" Jul 15 12:45:48.923784 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:48.923668 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b61bff8b-1999-4d29-9791-935bf444685f" containerName="cleanup" Jul 15 12:45:48.923784 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:48.923673 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="b61bff8b-1999-4d29-9791-935bf444685f" containerName="cleanup" Jul 15 12:45:48.923784 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:48.923742 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="553495ae-bebd-4a85-a12d-13f498d024ab" containerName="cleanup" Jul 15 12:45:48.923784 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:48.923753 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="c664f563-6f22-4e6a-8858-69c9b30f266b" containerName="cleanup" Jul 15 12:45:48.923784 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:48.923760 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="b61bff8b-1999-4d29-9791-935bf444685f" containerName="cleanup" Jul 15 12:45:48.926934 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:48.926916 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7615-140094018573744" Jul 15 12:45:48.929624 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:48.929604 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-7xt2g\"" Jul 15 12:45:48.942405 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:48.942382 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7615-140094018573744"] Jul 15 12:45:48.971657 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:48.971631 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gl2p\" (UniqueName: \"kubernetes.io/projected/34651fc5-1170-4e59-a7e1-2ccd5216a041-kube-api-access-5gl2p\") pod \"test-curl-7615-140094018573744\" (UID: \"34651fc5-1170-4e59-a7e1-2ccd5216a041\") " pod="opendatahub/test-curl-7615-140094018573744" Jul 15 12:45:49.072921 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:49.072891 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gl2p\" (UniqueName: \"kubernetes.io/projected/34651fc5-1170-4e59-a7e1-2ccd5216a041-kube-api-access-5gl2p\") pod \"test-curl-7615-140094018573744\" (UID: \"34651fc5-1170-4e59-a7e1-2ccd5216a041\") " pod="opendatahub/test-curl-7615-140094018573744" Jul 15 12:45:49.080785 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:49.080765 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gl2p\" (UniqueName: \"kubernetes.io/projected/34651fc5-1170-4e59-a7e1-2ccd5216a041-kube-api-access-5gl2p\") pod \"test-curl-7615-140094018573744\" (UID: \"34651fc5-1170-4e59-a7e1-2ccd5216a041\") " pod="opendatahub/test-curl-7615-140094018573744" Jul 15 12:45:49.238803 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:49.238699 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-7xt2g\"" Jul 15 12:45:49.246987 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:49.246966 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7615-140094018573744" Jul 15 12:45:49.394924 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:49.394895 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7615-140094018573744"] Jul 15 12:45:49.398017 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:45:49.397991 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod34651fc5_1170_4e59_a7e1_2ccd5216a041.slice/crio-a8120e245c40a8847fc64d206cd6aafb105b714384f0e59c211b2d5f255f0569 WatchSource:0}: Error finding container a8120e245c40a8847fc64d206cd6aafb105b714384f0e59c211b2d5f255f0569: Status 404 returned error can't find the container with id a8120e245c40a8847fc64d206cd6aafb105b714384f0e59c211b2d5f255f0569 Jul 15 12:45:49.399382 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:49.399362 2587 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 12:45:49.421784 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:49.421759 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7615-140094018573744" event={"ID":"34651fc5-1170-4e59-a7e1-2ccd5216a041","Type":"ContainerStarted","Data":"a8120e245c40a8847fc64d206cd6aafb105b714384f0e59c211b2d5f255f0569"} Jul 15 12:45:50.429159 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:50.429060 2587 generic.go:356] "Generic (PLEG): container finished" podID="34651fc5-1170-4e59-a7e1-2ccd5216a041" containerID="9d675ea2778e4f89e5484c0969df1b5d92acd34d1a835bb76b484a0e26d4915f" exitCode=0 Jul 15 12:45:50.429159 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:50.429140 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7615-140094018573744" event={"ID":"34651fc5-1170-4e59-a7e1-2ccd5216a041","Type":"ContainerDied","Data":"9d675ea2778e4f89e5484c0969df1b5d92acd34d1a835bb76b484a0e26d4915f"} Jul 15 12:45:51.569522 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:51.569499 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7615-140094018573744" Jul 15 12:45:51.612723 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:51.612653 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7615-140094018573744_34651fc5-1170-4e59-a7e1-2ccd5216a041/test-curl-7615-140094018573744/0.log" Jul 15 12:45:51.635375 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:51.635349 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7615-140094018573744"] Jul 15 12:45:51.639173 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:51.639148 2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7615-140094018573744"] Jul 15 12:45:51.696860 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:51.696800 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5gl2p\" (UniqueName: \"kubernetes.io/projected/34651fc5-1170-4e59-a7e1-2ccd5216a041-kube-api-access-5gl2p\") pod \"34651fc5-1170-4e59-a7e1-2ccd5216a041\" (UID: \"34651fc5-1170-4e59-a7e1-2ccd5216a041\") " Jul 15 12:45:51.698909 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:51.698887 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34651fc5-1170-4e59-a7e1-2ccd5216a041-kube-api-access-5gl2p" (OuterVolumeSpecName: "kube-api-access-5gl2p") pod "34651fc5-1170-4e59-a7e1-2ccd5216a041" (UID: "34651fc5-1170-4e59-a7e1-2ccd5216a041"). InnerVolumeSpecName "kube-api-access-5gl2p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:45:51.769342 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:51.769318 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7615-140094018574768"] Jul 15 12:45:51.769684 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:51.769672 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34651fc5-1170-4e59-a7e1-2ccd5216a041" containerName="test-curl-7615-140094018573744" Jul 15 12:45:51.769759 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:51.769686 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="34651fc5-1170-4e59-a7e1-2ccd5216a041" containerName="test-curl-7615-140094018573744" Jul 15 12:45:51.769801 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:51.769784 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="34651fc5-1170-4e59-a7e1-2ccd5216a041" containerName="test-curl-7615-140094018573744" Jul 15 12:45:51.772810 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:51.772793 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7615-140094018574768" Jul 15 12:45:51.786232 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:51.786215 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7615-140094018574768"] Jul 15 12:45:51.797987 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:51.797966 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5gl2p\" (UniqueName: \"kubernetes.io/projected/34651fc5-1170-4e59-a7e1-2ccd5216a041-kube-api-access-5gl2p\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:45:51.898994 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:51.898970 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2fgt\" (UniqueName: \"kubernetes.io/projected/8c4e4087-f1e8-40ed-b609-83a1abc0e54f-kube-api-access-b2fgt\") pod \"test-curl-7615-140094018574768\" (UID: \"8c4e4087-f1e8-40ed-b609-83a1abc0e54f\") " pod="opendatahub/test-curl-7615-140094018574768" Jul 15 12:45:51.999677 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:51.999613 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2fgt\" (UniqueName: \"kubernetes.io/projected/8c4e4087-f1e8-40ed-b609-83a1abc0e54f-kube-api-access-b2fgt\") pod \"test-curl-7615-140094018574768\" (UID: \"8c4e4087-f1e8-40ed-b609-83a1abc0e54f\") " pod="opendatahub/test-curl-7615-140094018574768" Jul 15 12:45:52.007975 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:52.007950 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2fgt\" (UniqueName: \"kubernetes.io/projected/8c4e4087-f1e8-40ed-b609-83a1abc0e54f-kube-api-access-b2fgt\") pod \"test-curl-7615-140094018574768\" (UID: \"8c4e4087-f1e8-40ed-b609-83a1abc0e54f\") " pod="opendatahub/test-curl-7615-140094018574768" Jul 15 12:45:52.081076 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:52.081053 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7615-140094018574768" Jul 15 12:45:52.216785 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:52.216762 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7615-140094018574768"] Jul 15 12:45:52.217467 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:45:52.217438 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8c4e4087_f1e8_40ed_b609_83a1abc0e54f.slice/crio-b29f73fe99ed92da191b8d9ebe198cbacb7e890eca6062e122b747ebb9269882 WatchSource:0}: Error finding container b29f73fe99ed92da191b8d9ebe198cbacb7e890eca6062e122b747ebb9269882: Status 404 returned error can't find the container with id b29f73fe99ed92da191b8d9ebe198cbacb7e890eca6062e122b747ebb9269882 Jul 15 12:45:52.441931 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:52.441892 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7615-140094018574768" event={"ID":"8c4e4087-f1e8-40ed-b609-83a1abc0e54f","Type":"ContainerStarted","Data":"b29f73fe99ed92da191b8d9ebe198cbacb7e890eca6062e122b747ebb9269882"} Jul 15 12:45:52.443049 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:52.443030 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7615-140094018573744" Jul 15 12:45:52.443138 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:52.443036 2587 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a8120e245c40a8847fc64d206cd6aafb105b714384f0e59c211b2d5f255f0569" Jul 15 12:45:52.453266 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:45:52.453242 2587 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7615-140094018573744\" is forbidden: User \"system:node:ip-10-0-135-243.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-243.ec2.internal' and this object" podUID="34651fc5-1170-4e59-a7e1-2ccd5216a041" pod="opendatahub/test-curl-7615-140094018573744" Jul 15 12:45:53.185558 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:53.185525 2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34651fc5-1170-4e59-a7e1-2ccd5216a041" path="/var/lib/kubelet/pods/34651fc5-1170-4e59-a7e1-2ccd5216a041/volumes" Jul 15 12:45:53.450707 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:53.450629 2587 generic.go:356] "Generic (PLEG): container finished" podID="8c4e4087-f1e8-40ed-b609-83a1abc0e54f" containerID="7466b217c9deb887ae5b649e8c2ce2eb4829b3ea0dc4a4f797ca8290f7eba523" exitCode=0 Jul 15 12:45:53.450707 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:53.450671 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7615-140094018574768" event={"ID":"8c4e4087-f1e8-40ed-b609-83a1abc0e54f","Type":"ContainerDied","Data":"7466b217c9deb887ae5b649e8c2ce2eb4829b3ea0dc4a4f797ca8290f7eba523"} Jul 15 12:45:54.590904 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:54.590875 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7615-140094018574768" Jul 15 12:45:54.616086 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:54.616055 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7615-140094018574768_8c4e4087-f1e8-40ed-b609-83a1abc0e54f/test-curl-7615-140094018574768/0.log" Jul 15 12:45:54.642835 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:54.642786 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7615-140094018574768"] Jul 15 12:45:54.645519 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:54.645498 2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7615-140094018574768"] Jul 15 12:45:54.720893 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:54.720812 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b2fgt\" (UniqueName: \"kubernetes.io/projected/8c4e4087-f1e8-40ed-b609-83a1abc0e54f-kube-api-access-b2fgt\") pod \"8c4e4087-f1e8-40ed-b609-83a1abc0e54f\" (UID: \"8c4e4087-f1e8-40ed-b609-83a1abc0e54f\") " Jul 15 12:45:54.722871 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:54.722850 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c4e4087-f1e8-40ed-b609-83a1abc0e54f-kube-api-access-b2fgt" (OuterVolumeSpecName: "kube-api-access-b2fgt") pod "8c4e4087-f1e8-40ed-b609-83a1abc0e54f" (UID: "8c4e4087-f1e8-40ed-b609-83a1abc0e54f"). InnerVolumeSpecName "kube-api-access-b2fgt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:45:54.812181 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:54.812150 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7615-140094018766576"] Jul 15 12:45:54.812536 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:54.812523 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c4e4087-f1e8-40ed-b609-83a1abc0e54f" containerName="test-curl-7615-140094018574768" Jul 15 12:45:54.812589 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:54.812539 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c4e4087-f1e8-40ed-b609-83a1abc0e54f" containerName="test-curl-7615-140094018574768" Jul 15 12:45:54.812622 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:54.812606 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c4e4087-f1e8-40ed-b609-83a1abc0e54f" containerName="test-curl-7615-140094018574768" Jul 15 12:45:54.815531 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:54.815515 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7615-140094018766576" Jul 15 12:45:54.821535 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:54.821507 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b2fgt\" (UniqueName: \"kubernetes.io/projected/8c4e4087-f1e8-40ed-b609-83a1abc0e54f-kube-api-access-b2fgt\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:45:54.824703 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:54.824677 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7615-140094018766576"] Jul 15 12:45:54.922601 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:54.922565 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46m5k\" (UniqueName: \"kubernetes.io/projected/30db6ca3-a7af-41eb-9a44-03145bcf377e-kube-api-access-46m5k\") pod \"test-curl-7615-140094018766576\" (UID: \"30db6ca3-a7af-41eb-9a44-03145bcf377e\") " pod="opendatahub/test-curl-7615-140094018766576" Jul 15 12:45:55.023554 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:55.023464 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46m5k\" (UniqueName: \"kubernetes.io/projected/30db6ca3-a7af-41eb-9a44-03145bcf377e-kube-api-access-46m5k\") pod \"test-curl-7615-140094018766576\" (UID: \"30db6ca3-a7af-41eb-9a44-03145bcf377e\") " pod="opendatahub/test-curl-7615-140094018766576" Jul 15 12:45:55.032527 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:55.032493 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-46m5k\" (UniqueName: \"kubernetes.io/projected/30db6ca3-a7af-41eb-9a44-03145bcf377e-kube-api-access-46m5k\") pod \"test-curl-7615-140094018766576\" (UID: \"30db6ca3-a7af-41eb-9a44-03145bcf377e\") " pod="opendatahub/test-curl-7615-140094018766576" Jul 15 12:45:55.125287 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:55.125255 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7615-140094018766576" Jul 15 12:45:55.189514 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:55.189476 2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c4e4087-f1e8-40ed-b609-83a1abc0e54f" path="/var/lib/kubelet/pods/8c4e4087-f1e8-40ed-b609-83a1abc0e54f/volumes" Jul 15 12:45:55.265508 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:55.265483 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7615-140094018766576"] Jul 15 12:45:55.266832 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:45:55.266807 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod30db6ca3_a7af_41eb_9a44_03145bcf377e.slice/crio-601fec0393ca9c2f3f16abdce28d3605b378c104c00b70a88e5e90d96a055718 WatchSource:0}: Error finding container 601fec0393ca9c2f3f16abdce28d3605b378c104c00b70a88e5e90d96a055718: Status 404 returned error can't find the container with id 601fec0393ca9c2f3f16abdce28d3605b378c104c00b70a88e5e90d96a055718 Jul 15 12:45:55.463528 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:55.463497 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7615-140094018574768" Jul 15 12:45:55.463715 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:55.463538 2587 scope.go:117] "RemoveContainer" containerID="7466b217c9deb887ae5b649e8c2ce2eb4829b3ea0dc4a4f797ca8290f7eba523" Jul 15 12:45:55.464986 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:55.464863 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7615-140094018766576" event={"ID":"30db6ca3-a7af-41eb-9a44-03145bcf377e","Type":"ContainerStarted","Data":"601fec0393ca9c2f3f16abdce28d3605b378c104c00b70a88e5e90d96a055718"} Jul 15 12:45:56.473327 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:56.473293 2587 generic.go:356] "Generic (PLEG): container finished" podID="30db6ca3-a7af-41eb-9a44-03145bcf377e" containerID="eeb84465acd0c4700b145c67f99ffd31c5fffb539e7aacfecc051bd096f4b7ee" exitCode=0 Jul 15 12:45:56.473688 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:56.473374 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7615-140094018766576" event={"ID":"30db6ca3-a7af-41eb-9a44-03145bcf377e","Type":"ContainerDied","Data":"eeb84465acd0c4700b145c67f99ffd31c5fffb539e7aacfecc051bd096f4b7ee"} Jul 15 12:45:57.616608 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:57.616582 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7615-140094018766576" Jul 15 12:45:57.635941 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:57.635910 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7615-140094018766576_30db6ca3-a7af-41eb-9a44-03145bcf377e/test-curl-7615-140094018766576/0.log" Jul 15 12:45:57.660111 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:57.660079 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7615-140094018766576"] Jul 15 12:45:57.662480 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:57.662459 2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7615-140094018766576"] Jul 15 12:45:57.750265 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:57.750176 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-46m5k\" (UniqueName: \"kubernetes.io/projected/30db6ca3-a7af-41eb-9a44-03145bcf377e-kube-api-access-46m5k\") pod \"30db6ca3-a7af-41eb-9a44-03145bcf377e\" (UID: \"30db6ca3-a7af-41eb-9a44-03145bcf377e\") " Jul 15 12:45:57.752258 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:57.752234 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30db6ca3-a7af-41eb-9a44-03145bcf377e-kube-api-access-46m5k" (OuterVolumeSpecName: "kube-api-access-46m5k") pod "30db6ca3-a7af-41eb-9a44-03145bcf377e" (UID: "30db6ca3-a7af-41eb-9a44-03145bcf377e"). InnerVolumeSpecName "kube-api-access-46m5k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:45:57.790378 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:57.790353 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7615-140094018766576"] Jul 15 12:45:57.790773 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:57.790759 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30db6ca3-a7af-41eb-9a44-03145bcf377e" containerName="test-curl-7615-140094018766576" Jul 15 12:45:57.790827 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:57.790774 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="30db6ca3-a7af-41eb-9a44-03145bcf377e" containerName="test-curl-7615-140094018766576" Jul 15 12:45:57.790863 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:57.790845 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="30db6ca3-a7af-41eb-9a44-03145bcf377e" containerName="test-curl-7615-140094018766576" Jul 15 12:45:57.793934 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:57.793919 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7615-140094018766576" Jul 15 12:45:57.802666 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:57.802640 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7615-140094018766576"] Jul 15 12:45:57.851403 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:57.851212 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pfhr\" (UniqueName: \"kubernetes.io/projected/54d3975e-a7ef-4dfa-92c7-7ea012ab6707-kube-api-access-5pfhr\") pod \"test-curl-7615-140094018766576\" (UID: \"54d3975e-a7ef-4dfa-92c7-7ea012ab6707\") " pod="opendatahub/test-curl-7615-140094018766576" Jul 15 12:45:57.851403 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:57.851348 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-46m5k\" (UniqueName: \"kubernetes.io/projected/30db6ca3-a7af-41eb-9a44-03145bcf377e-kube-api-access-46m5k\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:45:57.952619 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:57.952582 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pfhr\" (UniqueName: \"kubernetes.io/projected/54d3975e-a7ef-4dfa-92c7-7ea012ab6707-kube-api-access-5pfhr\") pod \"test-curl-7615-140094018766576\" (UID: \"54d3975e-a7ef-4dfa-92c7-7ea012ab6707\") " pod="opendatahub/test-curl-7615-140094018766576" Jul 15 12:45:57.960969 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:57.960938 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pfhr\" (UniqueName: \"kubernetes.io/projected/54d3975e-a7ef-4dfa-92c7-7ea012ab6707-kube-api-access-5pfhr\") pod \"test-curl-7615-140094018766576\" (UID: \"54d3975e-a7ef-4dfa-92c7-7ea012ab6707\") " pod="opendatahub/test-curl-7615-140094018766576" Jul 15 12:45:58.104243 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:58.104209 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7615-140094018766576" Jul 15 12:45:58.242573 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:58.242544 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7615-140094018766576"] Jul 15 12:45:58.243619 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:45:58.243594 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod54d3975e_a7ef_4dfa_92c7_7ea012ab6707.slice/crio-6a5bb912d1c2b543110ab4d19e5dfe562b4cd423f687fd1c8880fb1184f6892b WatchSource:0}: Error finding container 6a5bb912d1c2b543110ab4d19e5dfe562b4cd423f687fd1c8880fb1184f6892b: Status 404 returned error can't find the container with id 6a5bb912d1c2b543110ab4d19e5dfe562b4cd423f687fd1c8880fb1184f6892b Jul 15 12:45:58.486697 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:58.486617 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7615-140094018766576" Jul 15 12:45:58.486697 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:58.486645 2587 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="601fec0393ca9c2f3f16abdce28d3605b378c104c00b70a88e5e90d96a055718" Jul 15 12:45:58.487959 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:58.487928 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7615-140094018766576" event={"ID":"54d3975e-a7ef-4dfa-92c7-7ea012ab6707","Type":"ContainerStarted","Data":"6a5bb912d1c2b543110ab4d19e5dfe562b4cd423f687fd1c8880fb1184f6892b"} Jul 15 12:45:58.513750 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:58.511027 2587 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7615-140094018766576" oldPodUID="30db6ca3-a7af-41eb-9a44-03145bcf377e" podUID="54d3975e-a7ef-4dfa-92c7-7ea012ab6707" Jul 15 12:45:59.186058 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:59.186027 2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30db6ca3-a7af-41eb-9a44-03145bcf377e" path="/var/lib/kubelet/pods/30db6ca3-a7af-41eb-9a44-03145bcf377e/volumes" Jul 15 12:45:59.381768 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:59.381719 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:45:59.496167 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:59.496071 2587 generic.go:356] "Generic (PLEG): container finished" podID="54d3975e-a7ef-4dfa-92c7-7ea012ab6707" containerID="c78c60f5c931832c2a63040d83af015a30c5eb4ec33023f8ec5ea5c38b62940a" exitCode=0 Jul 15 12:45:59.496167 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:45:59.496143 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7615-140094018766576" event={"ID":"54d3975e-a7ef-4dfa-92c7-7ea012ab6707","Type":"ContainerDied","Data":"c78c60f5c931832c2a63040d83af015a30c5eb4ec33023f8ec5ea5c38b62940a"} Jul 15 12:46:00.643576 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:00.643549 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7615-140094018766576" Jul 15 12:46:00.666852 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:00.666808 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7615-140094018766576_54d3975e-a7ef-4dfa-92c7-7ea012ab6707/test-curl-7615-140094018766576/0.log" Jul 15 12:46:00.690647 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:00.690617 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7615-140094018766576"] Jul 15 12:46:00.696028 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:00.695992 2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7615-140094018766576"] Jul 15 12:46:00.780087 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:00.780010 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5pfhr\" (UniqueName: \"kubernetes.io/projected/54d3975e-a7ef-4dfa-92c7-7ea012ab6707-kube-api-access-5pfhr\") pod \"54d3975e-a7ef-4dfa-92c7-7ea012ab6707\" (UID: \"54d3975e-a7ef-4dfa-92c7-7ea012ab6707\") " Jul 15 12:46:00.782053 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:00.782029 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54d3975e-a7ef-4dfa-92c7-7ea012ab6707-kube-api-access-5pfhr" (OuterVolumeSpecName: "kube-api-access-5pfhr") pod "54d3975e-a7ef-4dfa-92c7-7ea012ab6707" (UID: "54d3975e-a7ef-4dfa-92c7-7ea012ab6707"). InnerVolumeSpecName "kube-api-access-5pfhr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:46:00.835486 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:00.835453 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7615-140094017729856"] Jul 15 12:46:00.835890 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:00.835874 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54d3975e-a7ef-4dfa-92c7-7ea012ab6707" containerName="test-curl-7615-140094018766576" Jul 15 12:46:00.835949 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:00.835892 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="54d3975e-a7ef-4dfa-92c7-7ea012ab6707" containerName="test-curl-7615-140094018766576" Jul 15 12:46:00.835992 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:00.835959 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="54d3975e-a7ef-4dfa-92c7-7ea012ab6707" containerName="test-curl-7615-140094018766576" Jul 15 12:46:00.839047 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:00.839030 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7615-140094017729856" Jul 15 12:46:00.845168 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:00.845145 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7615-140094017729856"] Jul 15 12:46:00.881085 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:00.881059 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5pfhr\" (UniqueName: \"kubernetes.io/projected/54d3975e-a7ef-4dfa-92c7-7ea012ab6707-kube-api-access-5pfhr\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:46:00.982120 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:00.982087 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5n9lx\" (UniqueName: \"kubernetes.io/projected/995c6b11-4af5-4386-8e65-2c42e0f67f4a-kube-api-access-5n9lx\") pod \"test-curl-7615-140094017729856\" (UID: \"995c6b11-4af5-4386-8e65-2c42e0f67f4a\") " pod="opendatahub/test-curl-7615-140094017729856" Jul 15 12:46:01.082507 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:01.082474 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5n9lx\" (UniqueName: \"kubernetes.io/projected/995c6b11-4af5-4386-8e65-2c42e0f67f4a-kube-api-access-5n9lx\") pod \"test-curl-7615-140094017729856\" (UID: \"995c6b11-4af5-4386-8e65-2c42e0f67f4a\") " pod="opendatahub/test-curl-7615-140094017729856" Jul 15 12:46:01.092603 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:01.092577 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5n9lx\" (UniqueName: \"kubernetes.io/projected/995c6b11-4af5-4386-8e65-2c42e0f67f4a-kube-api-access-5n9lx\") pod \"test-curl-7615-140094017729856\" (UID: \"995c6b11-4af5-4386-8e65-2c42e0f67f4a\") " pod="opendatahub/test-curl-7615-140094017729856" Jul 15 12:46:01.148922 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:01.148891 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7615-140094017729856" Jul 15 12:46:01.186668 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:01.186632 2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54d3975e-a7ef-4dfa-92c7-7ea012ab6707" path="/var/lib/kubelet/pods/54d3975e-a7ef-4dfa-92c7-7ea012ab6707/volumes" Jul 15 12:46:01.289250 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:01.289221 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7615-140094017729856"] Jul 15 12:46:01.290633 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:46:01.290604 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod995c6b11_4af5_4386_8e65_2c42e0f67f4a.slice/crio-f7de26728b1752a94b6382b268a5f242e12957ef20ed7f3b6cd66efc9ea8264c WatchSource:0}: Error finding container f7de26728b1752a94b6382b268a5f242e12957ef20ed7f3b6cd66efc9ea8264c: Status 404 returned error can't find the container with id f7de26728b1752a94b6382b268a5f242e12957ef20ed7f3b6cd66efc9ea8264c Jul 15 12:46:01.510037 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:01.509947 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7615-140094018766576" Jul 15 12:46:01.510037 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:01.509993 2587 scope.go:117] "RemoveContainer" containerID="c78c60f5c931832c2a63040d83af015a30c5eb4ec33023f8ec5ea5c38b62940a" Jul 15 12:46:01.511351 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:01.511173 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7615-140094017729856" event={"ID":"995c6b11-4af5-4386-8e65-2c42e0f67f4a","Type":"ContainerStarted","Data":"f7de26728b1752a94b6382b268a5f242e12957ef20ed7f3b6cd66efc9ea8264c"} Jul 15 12:46:02.519624 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:02.519591 2587 generic.go:356] "Generic (PLEG): container finished" podID="995c6b11-4af5-4386-8e65-2c42e0f67f4a" containerID="10fea6039730fec4e968560cdd763dce9a839e2607beeee8d52644f2d1ed3ff4" exitCode=0 Jul 15 12:46:02.520015 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:02.519683 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7615-140094017729856" event={"ID":"995c6b11-4af5-4386-8e65-2c42e0f67f4a","Type":"ContainerDied","Data":"10fea6039730fec4e968560cdd763dce9a839e2607beeee8d52644f2d1ed3ff4"} Jul 15 12:46:03.664702 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:03.664673 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7615-140094017729856" Jul 15 12:46:03.686974 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:03.686939 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7615-140094017729856_995c6b11-4af5-4386-8e65-2c42e0f67f4a/test-curl-7615-140094017729856/0.log" Jul 15 12:46:03.714273 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:03.714242 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7615-140094017729856"] Jul 15 12:46:03.717624 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:03.717598 2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7615-140094017729856"] Jul 15 12:46:03.807494 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:03.807463 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5n9lx\" (UniqueName: \"kubernetes.io/projected/995c6b11-4af5-4386-8e65-2c42e0f67f4a-kube-api-access-5n9lx\") pod \"995c6b11-4af5-4386-8e65-2c42e0f67f4a\" (UID: \"995c6b11-4af5-4386-8e65-2c42e0f67f4a\") " Jul 15 12:46:03.809532 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:03.809500 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/995c6b11-4af5-4386-8e65-2c42e0f67f4a-kube-api-access-5n9lx" (OuterVolumeSpecName: "kube-api-access-5n9lx") pod "995c6b11-4af5-4386-8e65-2c42e0f67f4a" (UID: "995c6b11-4af5-4386-8e65-2c42e0f67f4a"). InnerVolumeSpecName "kube-api-access-5n9lx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:46:03.908457 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:03.908423 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5n9lx\" (UniqueName: \"kubernetes.io/projected/995c6b11-4af5-4386-8e65-2c42e0f67f4a-kube-api-access-5n9lx\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:46:04.533883 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:04.533847 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7615-140094017729856" Jul 15 12:46:04.534091 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:04.533897 2587 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f7de26728b1752a94b6382b268a5f242e12957ef20ed7f3b6cd66efc9ea8264c" Jul 15 12:46:04.543810 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:46:04.543783 2587 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7615-140094017729856\" is forbidden: User \"system:node:ip-10-0-135-243.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-243.ec2.internal' and this object" podUID="995c6b11-4af5-4386-8e65-2c42e0f67f4a" pod="opendatahub/test-curl-7615-140094017729856" Jul 15 12:46:05.188880 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:05.186997 2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="995c6b11-4af5-4386-8e65-2c42e0f67f4a" path="/var/lib/kubelet/pods/995c6b11-4af5-4386-8e65-2c42e0f67f4a/volumes" Jul 15 12:46:17.522523 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:17.522493 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a9c7da-29735325-6m8s7"] Jul 15 12:46:17.524885 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:17.524860 2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a9c7da-29735325-6m8s7"] Jul 15 12:46:19.187327 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:19.187291 2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c664f563-6f22-4e6a-8858-69c9b30f266b" path="/var/lib/kubelet/pods/c664f563-6f22-4e6a-8858-69c9b30f266b/volumes" Jul 15 12:46:20.879602 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:20.879567 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:46:43.763964 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:43.763887 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-b2e819-7c74db7fdf-f6294"] Jul 15 12:46:43.764358 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:43.764083 2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b2e819-7c74db7fdf-f6294" podUID="5f17da08-ffa5-453f-a1c3-c9bbbbb8b73a" containerName="maas-api" containerID="cri-o://e8923ba8b48596398783a46efb9f7cbd5a958fe3afce13578748dad0b419db62" gracePeriod=30 Jul 15 12:46:43.782609 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:43.782580 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-b2e819-29735325-xbplr"] Jul 15 12:46:43.785179 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:43.785160 2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-b2e819-29735325-xbplr"] Jul 15 12:46:44.021903 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:44.021846 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b2e819-7c74db7fdf-f6294" Jul 15 12:46:44.052043 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:44.052017 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pqcgz\" (UniqueName: \"kubernetes.io/projected/5f17da08-ffa5-453f-a1c3-c9bbbbb8b73a-kube-api-access-pqcgz\") pod \"5f17da08-ffa5-453f-a1c3-c9bbbbb8b73a\" (UID: \"5f17da08-ffa5-453f-a1c3-c9bbbbb8b73a\") " Jul 15 12:46:44.052190 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:44.052068 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f17da08-ffa5-453f-a1c3-c9bbbbb8b73a-openshift-service-ca\") pod \"5f17da08-ffa5-453f-a1c3-c9bbbbb8b73a\" (UID: \"5f17da08-ffa5-453f-a1c3-c9bbbbb8b73a\") " Jul 15 12:46:44.052190 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:44.052116 2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f17da08-ffa5-453f-a1c3-c9bbbbb8b73a-maas-api-tls\") pod \"5f17da08-ffa5-453f-a1c3-c9bbbbb8b73a\" (UID: \"5f17da08-ffa5-453f-a1c3-c9bbbbb8b73a\") " Jul 15 12:46:44.052487 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:44.052463 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f17da08-ffa5-453f-a1c3-c9bbbbb8b73a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5f17da08-ffa5-453f-a1c3-c9bbbbb8b73a" (UID: "5f17da08-ffa5-453f-a1c3-c9bbbbb8b73a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 12:46:44.054303 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:44.054272 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f17da08-ffa5-453f-a1c3-c9bbbbb8b73a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5f17da08-ffa5-453f-a1c3-c9bbbbb8b73a" (UID: "5f17da08-ffa5-453f-a1c3-c9bbbbb8b73a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 12:46:44.054447 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:44.054424 2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f17da08-ffa5-453f-a1c3-c9bbbbb8b73a-kube-api-access-pqcgz" (OuterVolumeSpecName: "kube-api-access-pqcgz") pod "5f17da08-ffa5-453f-a1c3-c9bbbbb8b73a" (UID: "5f17da08-ffa5-453f-a1c3-c9bbbbb8b73a"). InnerVolumeSpecName "kube-api-access-pqcgz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 12:46:44.152931 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:44.152894 2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pqcgz\" (UniqueName: \"kubernetes.io/projected/5f17da08-ffa5-453f-a1c3-c9bbbbb8b73a-kube-api-access-pqcgz\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:46:44.152931 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:44.152923 2587 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f17da08-ffa5-453f-a1c3-c9bbbbb8b73a-openshift-service-ca\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:46:44.152931 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:44.152933 2587 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f17da08-ffa5-453f-a1c3-c9bbbbb8b73a-maas-api-tls\") on node \"ip-10-0-135-243.ec2.internal\" DevicePath \"\"" Jul 15 12:46:44.778999 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:44.778958 2587 generic.go:356] "Generic (PLEG): container finished" podID="5f17da08-ffa5-453f-a1c3-c9bbbbb8b73a" containerID="e8923ba8b48596398783a46efb9f7cbd5a958fe3afce13578748dad0b419db62" exitCode=0 Jul 15 12:46:44.779443 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:44.779016 2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b2e819-7c74db7fdf-f6294" Jul 15 12:46:44.779443 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:44.779047 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b2e819-7c74db7fdf-f6294" event={"ID":"5f17da08-ffa5-453f-a1c3-c9bbbbb8b73a","Type":"ContainerDied","Data":"e8923ba8b48596398783a46efb9f7cbd5a958fe3afce13578748dad0b419db62"} Jul 15 12:46:44.779443 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:44.779126 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b2e819-7c74db7fdf-f6294" event={"ID":"5f17da08-ffa5-453f-a1c3-c9bbbbb8b73a","Type":"ContainerDied","Data":"6495cf8a439ab4f897a3ec5e17e7d5905deb7a8130e432ef546c169cf44d616d"} Jul 15 12:46:44.779443 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:44.779148 2587 scope.go:117] "RemoveContainer" containerID="e8923ba8b48596398783a46efb9f7cbd5a958fe3afce13578748dad0b419db62" Jul 15 12:46:44.789091 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:44.789072 2587 scope.go:117] "RemoveContainer" containerID="e8923ba8b48596398783a46efb9f7cbd5a958fe3afce13578748dad0b419db62" Jul 15 12:46:44.789367 ip-10-0-135-243 kubenswrapper[2587]: E0715 12:46:44.789347 2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8923ba8b48596398783a46efb9f7cbd5a958fe3afce13578748dad0b419db62\": container with ID starting with e8923ba8b48596398783a46efb9f7cbd5a958fe3afce13578748dad0b419db62 not found: ID does not exist" containerID="e8923ba8b48596398783a46efb9f7cbd5a958fe3afce13578748dad0b419db62" Jul 15 12:46:44.789443 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:44.789375 2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8923ba8b48596398783a46efb9f7cbd5a958fe3afce13578748dad0b419db62"} err="failed to get container status \"e8923ba8b48596398783a46efb9f7cbd5a958fe3afce13578748dad0b419db62\": rpc error: code = NotFound desc = could not find container \"e8923ba8b48596398783a46efb9f7cbd5a958fe3afce13578748dad0b419db62\": container with ID starting with e8923ba8b48596398783a46efb9f7cbd5a958fe3afce13578748dad0b419db62 not found: ID does not exist" Jul 15 12:46:44.801939 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:44.801917 2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-b2e819-7c74db7fdf-f6294"] Jul 15 12:46:44.805923 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:44.805901 2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-b2e819-7c74db7fdf-f6294"] Jul 15 12:46:45.185772 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:45.185721 2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f17da08-ffa5-453f-a1c3-c9bbbbb8b73a" path="/var/lib/kubelet/pods/5f17da08-ffa5-453f-a1c3-c9bbbbb8b73a/volumes" Jul 15 12:46:45.186068 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:45.186054 2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b61bff8b-1999-4d29-9791-935bf444685f" path="/var/lib/kubelet/pods/b61bff8b-1999-4d29-9791-935bf444685f/volumes" Jul 15 12:46:47.081223 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:46:47.081189 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7bz5l"] Jul 15 12:47:05.360859 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:05.360814 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-bh8kq_41bdfb9d-6267-4e66-8925-31ffacf48d39/manager/0.log" Jul 15 12:47:05.725015 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:05.724986 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-j4h5d_15f67716-be1c-4a3f-9dc9-d35980060c82/server/0.log" Jul 15 12:47:05.847931 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:05.847897 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-p2pr6_14ba6a32-b02c-4afd-9adb-4547d982708f/manager/0.log" Jul 15 12:47:06.228813 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:06.228782 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-694c4fcbc8-mdm8d_04eb66ea-bd75-40be-8683-2072ee07eb79/copy-manifests/0.log" Jul 15 12:47:06.237689 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:06.237658 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-694c4fcbc8-mdm8d_04eb66ea-bd75-40be-8683-2072ee07eb79/manager/0.log" Jul 15 12:47:06.925625 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:06.925593 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29735325-6c2f8_553495ae-bebd-4a85-a12d-13f498d024ab/cleanup/0.log" Jul 15 12:47:07.258590 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:07.258503 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-a-858a49-mxgwk_dd5f415f-a6aa-4c9b-8537-54130aae80b9/revoke-keys/0.log" Jul 15 12:47:07.597231 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:07.597204 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-a-dbcba5-jbnvz_9b69d907-3a2f-4bcb-94b3-a9ff0c62cd9a/revoke-keys/0.log" Jul 15 12:47:07.933099 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:07.933016 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-145feecc-jz8rk_faa078fb-5623-4357-9149-3c02bc392c4b/revoke-keys/0.log" Jul 15 12:47:09.178107 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:09.178076 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tcccd_760fbdcd-8e40-4d90-8d93-7402a4aafee6/util/0.log" Jul 15 12:47:09.183742 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:09.183715 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tcccd_760fbdcd-8e40-4d90-8d93-7402a4aafee6/pull/0.log" Jul 15 12:47:09.188748 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:09.188709 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tcccd_760fbdcd-8e40-4d90-8d93-7402a4aafee6/extract/0.log" Jul 15 12:47:09.303861 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:09.303831 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l8c6h_825aeecb-52ad-4bd5-bba0-b4438705430d/util/0.log" Jul 15 12:47:09.310331 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:09.310306 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l8c6h_825aeecb-52ad-4bd5-bba0-b4438705430d/pull/0.log" Jul 15 12:47:09.316100 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:09.316077 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l8c6h_825aeecb-52ad-4bd5-bba0-b4438705430d/extract/0.log" Jul 15 12:47:09.426032 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:09.426006 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qspjl_4e4e75e4-4576-4bab-922e-ffe34d7890ff/util/0.log" Jul 15 12:47:09.431948 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:09.431886 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qspjl_4e4e75e4-4576-4bab-922e-ffe34d7890ff/pull/0.log" Jul 15 12:47:09.437234 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:09.437207 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qspjl_4e4e75e4-4576-4bab-922e-ffe34d7890ff/extract/0.log" Jul 15 12:47:09.553305 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:09.553270 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8khc_70f2106a-f040-4eb3-ae55-4c46d899932b/util/0.log" Jul 15 12:47:09.558937 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:09.558880 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8khc_70f2106a-f040-4eb3-ae55-4c46d899932b/pull/0.log" Jul 15 12:47:09.564424 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:09.564407 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8khc_70f2106a-f040-4eb3-ae55-4c46d899932b/extract/0.log" Jul 15 12:47:09.826481 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:09.826452 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-jscrc_51f92a56-4f2d-4608-8c65-6958e300c98b/manager/0.log" Jul 15 12:47:10.294649 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:10.294620 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-96jwd_3ca440f5-57ad-4103-a99c-964e0bd478ca/manager/0.log" Jul 15 12:47:10.419097 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:10.419069 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-7bz5l_a7a6f20e-072d-4ede-9cd9-9e5c82860905/limitador/0.log" Jul 15 12:47:11.036882 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:11.036852 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-4lxxm_6dd415bd-a802-4b9d-8a67-b81b97863da4/discovery/0.log" Jul 15 12:47:11.151641 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:11.151613 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d7f886b65-qd67k_a347f0d8-4a4b-4f5d-83b8-082a96f4aa7a/kube-auth-proxy/0.log" Jul 15 12:47:12.354123 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:12.354090 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-9gcsv_e667ff0b-0f12-4fa1-bb39-1e7bb96b9c93/main/0.log" Jul 15 12:47:21.270967 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:21.270926 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-vn6hr_c26f517e-49fe-4da5-a790-e8b77483317a/global-pull-secret-syncer/0.log" Jul 15 12:47:21.397648 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:21.397617 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-x9kt6_20ea3927-e5c6-47c0-9ff6-43b7cad247c0/konnectivity-agent/0.log" Jul 15 12:47:21.440340 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:21.440308 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-243.ec2.internal_31cf20077539c85a825a4f13af6feab9/haproxy/0.log" Jul 15 12:47:26.247122 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:26.247084 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tcccd_760fbdcd-8e40-4d90-8d93-7402a4aafee6/extract/0.log" Jul 15 12:47:26.285243 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:26.285210 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tcccd_760fbdcd-8e40-4d90-8d93-7402a4aafee6/util/0.log" Jul 15 12:47:26.310135 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:26.310101 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tcccd_760fbdcd-8e40-4d90-8d93-7402a4aafee6/pull/0.log" Jul 15 12:47:26.335175 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:26.335147 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l8c6h_825aeecb-52ad-4bd5-bba0-b4438705430d/extract/0.log" Jul 15 12:47:26.354082 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:26.354061 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l8c6h_825aeecb-52ad-4bd5-bba0-b4438705430d/util/0.log" Jul 15 12:47:26.374149 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:26.374127 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l8c6h_825aeecb-52ad-4bd5-bba0-b4438705430d/pull/0.log" Jul 15 12:47:26.417318 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:26.417294 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qspjl_4e4e75e4-4576-4bab-922e-ffe34d7890ff/extract/0.log" Jul 15 12:47:26.446838 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:26.446814 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qspjl_4e4e75e4-4576-4bab-922e-ffe34d7890ff/util/0.log" Jul 15 12:47:26.475228 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:26.475206 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qspjl_4e4e75e4-4576-4bab-922e-ffe34d7890ff/pull/0.log" Jul 15 12:47:26.509506 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:26.509444 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8khc_70f2106a-f040-4eb3-ae55-4c46d899932b/extract/0.log" Jul 15 12:47:26.535123 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:26.535097 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8khc_70f2106a-f040-4eb3-ae55-4c46d899932b/util/0.log" Jul 15 12:47:26.555942 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:26.555922 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8khc_70f2106a-f040-4eb3-ae55-4c46d899932b/pull/0.log" Jul 15 12:47:27.049476 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:27.049444 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-jscrc_51f92a56-4f2d-4608-8c65-6958e300c98b/manager/0.log" Jul 15 12:47:27.313607 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:27.313527 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-96jwd_3ca440f5-57ad-4103-a99c-964e0bd478ca/manager/0.log" Jul 15 12:47:27.334496 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:27.334469 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-7bz5l_a7a6f20e-072d-4ede-9cd9-9e5c82860905/limitador/0.log" Jul 15 12:47:29.085617 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:29.085564 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7db5bbc3-42f4-4da9-8b50-ef83c65b502d/alertmanager/0.log" Jul 15 12:47:29.124420 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:29.124395 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7db5bbc3-42f4-4da9-8b50-ef83c65b502d/config-reloader/0.log" Jul 15 12:47:29.147270 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:29.147241 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7db5bbc3-42f4-4da9-8b50-ef83c65b502d/kube-rbac-proxy-web/0.log" Jul 15 12:47:29.166929 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:29.166907 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7db5bbc3-42f4-4da9-8b50-ef83c65b502d/kube-rbac-proxy/0.log" Jul 15 12:47:29.186494 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:29.186460 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7db5bbc3-42f4-4da9-8b50-ef83c65b502d/kube-rbac-proxy-metric/0.log" Jul 15 12:47:29.206901 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:29.206878 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7db5bbc3-42f4-4da9-8b50-ef83c65b502d/prom-label-proxy/0.log" Jul 15 12:47:29.229076 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:29.229053 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7db5bbc3-42f4-4da9-8b50-ef83c65b502d/init-config-reloader/0.log" Jul 15 12:47:29.442268 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:29.442237 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7vz7j_cec409e8-1241-411e-834f-7d82377b1f8b/node-exporter/0.log" Jul 15 12:47:29.467804 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:29.467771 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7vz7j_cec409e8-1241-411e-834f-7d82377b1f8b/kube-rbac-proxy/0.log" Jul 15 12:47:29.488118 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:29.488096 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7vz7j_cec409e8-1241-411e-834f-7d82377b1f8b/init-textfile/0.log" Jul 15 12:47:29.904309 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:29.904276 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-9pvkq_d1c5ae78-48ab-4245-912c-3497e6c503b5/prometheus-operator/0.log" Jul 15 12:47:29.919132 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:29.919108 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-9pvkq_d1c5ae78-48ab-4245-912c-3497e6c503b5/kube-rbac-proxy/0.log" Jul 15 12:47:29.949421 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:29.949392 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-bbrmm_a4a87ac3-6521-499a-8566-67c0f2b073e4/prometheus-operator-admission-webhook/0.log" Jul 15 12:47:29.970771 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:29.970749 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7c9f77bc7-ltx55_1a4cf141-579e-47bd-b911-b19e20f0599c/telemeter-client/0.log" Jul 15 12:47:29.990583 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:29.990557 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7c9f77bc7-ltx55_1a4cf141-579e-47bd-b911-b19e20f0599c/reload/0.log" Jul 15 12:47:30.012812 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:30.012789 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7c9f77bc7-ltx55_1a4cf141-579e-47bd-b911-b19e20f0599c/kube-rbac-proxy/0.log" Jul 15 12:47:30.869602 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:30.869571 2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-d79rw/perf-node-gather-daemonset-kqlq9"] Jul 15 12:47:30.870008 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:30.869982 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="995c6b11-4af5-4386-8e65-2c42e0f67f4a" containerName="test-curl-7615-140094017729856" Jul 15 12:47:30.870008 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:30.869996 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="995c6b11-4af5-4386-8e65-2c42e0f67f4a" containerName="test-curl-7615-140094017729856" Jul 15 12:47:30.870080 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:30.870015 2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f17da08-ffa5-453f-a1c3-c9bbbbb8b73a" containerName="maas-api" Jul 15 12:47:30.870080 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:30.870021 2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f17da08-ffa5-453f-a1c3-c9bbbbb8b73a" containerName="maas-api" Jul 15 12:47:30.870143 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:30.870083 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="995c6b11-4af5-4386-8e65-2c42e0f67f4a" containerName="test-curl-7615-140094017729856" Jul 15 12:47:30.870143 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:30.870092 2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f17da08-ffa5-453f-a1c3-c9bbbbb8b73a" containerName="maas-api" Jul 15 12:47:30.873318 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:30.873297 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-d79rw/perf-node-gather-daemonset-kqlq9" Jul 15 12:47:30.876036 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:30.876016 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-d79rw\"/\"kube-root-ca.crt\"" Jul 15 12:47:30.877338 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:30.877313 2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-d79rw\"/\"openshift-service-ca.crt\"" Jul 15 12:47:30.877338 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:30.877326 2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-d79rw\"/\"default-dockercfg-l7gjh\"" Jul 15 12:47:30.885391 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:30.885365 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-d79rw/perf-node-gather-daemonset-kqlq9"] Jul 15 12:47:30.953835 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:30.953803 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/57a41795-a558-4ecd-be74-25809affa0c5-podres\") pod \"perf-node-gather-daemonset-kqlq9\" (UID: \"57a41795-a558-4ecd-be74-25809affa0c5\") " pod="openshift-must-gather-d79rw/perf-node-gather-daemonset-kqlq9" Jul 15 12:47:30.954015 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:30.953857 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/57a41795-a558-4ecd-be74-25809affa0c5-lib-modules\") pod \"perf-node-gather-daemonset-kqlq9\" (UID: \"57a41795-a558-4ecd-be74-25809affa0c5\") " pod="openshift-must-gather-d79rw/perf-node-gather-daemonset-kqlq9" Jul 15 12:47:30.954015 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:30.953923 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgd8c\" (UniqueName: \"kubernetes.io/projected/57a41795-a558-4ecd-be74-25809affa0c5-kube-api-access-lgd8c\") pod \"perf-node-gather-daemonset-kqlq9\" (UID: \"57a41795-a558-4ecd-be74-25809affa0c5\") " pod="openshift-must-gather-d79rw/perf-node-gather-daemonset-kqlq9" Jul 15 12:47:30.954015 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:30.953947 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/57a41795-a558-4ecd-be74-25809affa0c5-sys\") pod \"perf-node-gather-daemonset-kqlq9\" (UID: \"57a41795-a558-4ecd-be74-25809affa0c5\") " pod="openshift-must-gather-d79rw/perf-node-gather-daemonset-kqlq9" Jul 15 12:47:30.954015 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:30.953969 2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/57a41795-a558-4ecd-be74-25809affa0c5-proc\") pod \"perf-node-gather-daemonset-kqlq9\" (UID: \"57a41795-a558-4ecd-be74-25809affa0c5\") " pod="openshift-must-gather-d79rw/perf-node-gather-daemonset-kqlq9" Jul 15 12:47:31.054798 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:31.054768 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/57a41795-a558-4ecd-be74-25809affa0c5-podres\") pod \"perf-node-gather-daemonset-kqlq9\" (UID: \"57a41795-a558-4ecd-be74-25809affa0c5\") " pod="openshift-must-gather-d79rw/perf-node-gather-daemonset-kqlq9" Jul 15 12:47:31.054987 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:31.054814 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/57a41795-a558-4ecd-be74-25809affa0c5-lib-modules\") pod \"perf-node-gather-daemonset-kqlq9\" (UID: \"57a41795-a558-4ecd-be74-25809affa0c5\") " pod="openshift-must-gather-d79rw/perf-node-gather-daemonset-kqlq9" Jul 15 12:47:31.054987 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:31.054852 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgd8c\" (UniqueName: \"kubernetes.io/projected/57a41795-a558-4ecd-be74-25809affa0c5-kube-api-access-lgd8c\") pod \"perf-node-gather-daemonset-kqlq9\" (UID: \"57a41795-a558-4ecd-be74-25809affa0c5\") " pod="openshift-must-gather-d79rw/perf-node-gather-daemonset-kqlq9" Jul 15 12:47:31.054987 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:31.054871 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/57a41795-a558-4ecd-be74-25809affa0c5-sys\") pod \"perf-node-gather-daemonset-kqlq9\" (UID: \"57a41795-a558-4ecd-be74-25809affa0c5\") " pod="openshift-must-gather-d79rw/perf-node-gather-daemonset-kqlq9" Jul 15 12:47:31.054987 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:31.054886 2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/57a41795-a558-4ecd-be74-25809affa0c5-proc\") pod \"perf-node-gather-daemonset-kqlq9\" (UID: \"57a41795-a558-4ecd-be74-25809affa0c5\") " pod="openshift-must-gather-d79rw/perf-node-gather-daemonset-kqlq9" Jul 15 12:47:31.054987 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:31.054965 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/57a41795-a558-4ecd-be74-25809affa0c5-podres\") pod \"perf-node-gather-daemonset-kqlq9\" (UID: \"57a41795-a558-4ecd-be74-25809affa0c5\") " pod="openshift-must-gather-d79rw/perf-node-gather-daemonset-kqlq9" Jul 15 12:47:31.054987 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:31.054987 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/57a41795-a558-4ecd-be74-25809affa0c5-proc\") pod \"perf-node-gather-daemonset-kqlq9\" (UID: \"57a41795-a558-4ecd-be74-25809affa0c5\") " pod="openshift-must-gather-d79rw/perf-node-gather-daemonset-kqlq9" Jul 15 12:47:31.055205 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:31.054991 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/57a41795-a558-4ecd-be74-25809affa0c5-sys\") pod \"perf-node-gather-daemonset-kqlq9\" (UID: \"57a41795-a558-4ecd-be74-25809affa0c5\") " pod="openshift-must-gather-d79rw/perf-node-gather-daemonset-kqlq9" Jul 15 12:47:31.055205 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:31.055034 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/57a41795-a558-4ecd-be74-25809affa0c5-lib-modules\") pod \"perf-node-gather-daemonset-kqlq9\" (UID: \"57a41795-a558-4ecd-be74-25809affa0c5\") " pod="openshift-must-gather-d79rw/perf-node-gather-daemonset-kqlq9" Jul 15 12:47:31.064825 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:31.064796 2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgd8c\" (UniqueName: \"kubernetes.io/projected/57a41795-a558-4ecd-be74-25809affa0c5-kube-api-access-lgd8c\") pod \"perf-node-gather-daemonset-kqlq9\" (UID: \"57a41795-a558-4ecd-be74-25809affa0c5\") " pod="openshift-must-gather-d79rw/perf-node-gather-daemonset-kqlq9" Jul 15 12:47:31.184948 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:31.184871 2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-d79rw/perf-node-gather-daemonset-kqlq9" Jul 15 12:47:31.323691 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:31.323665 2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-d79rw/perf-node-gather-daemonset-kqlq9"] Jul 15 12:47:31.325189 ip-10-0-135-243 kubenswrapper[2587]: W0715 12:47:31.325164 2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod57a41795_a558_4ecd_be74_25809affa0c5.slice/crio-d0fe0e519d703cf2dd3f5d545b2915b21a74d0428cb61c62c288b7f468eb1ae9 WatchSource:0}: Error finding container d0fe0e519d703cf2dd3f5d545b2915b21a74d0428cb61c62c288b7f468eb1ae9: Status 404 returned error can't find the container with id d0fe0e519d703cf2dd3f5d545b2915b21a74d0428cb61c62c288b7f468eb1ae9 Jul 15 12:47:31.410638 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:31.410612 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-8fjlk_379eba97-49a2-4f00-afba-0e65b34fd260/networking-console-plugin/0.log" Jul 15 12:47:32.000838 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:32.000803 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-4xrn6_85ce5adf-d361-4258-b4ca-7f13d35fe13d/console-operator/1.log" Jul 15 12:47:32.009990 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:32.009949 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-4xrn6_85ce5adf-d361-4258-b4ca-7f13d35fe13d/console-operator/2.log" Jul 15 12:47:32.051886 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:32.051853 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-d79rw/perf-node-gather-daemonset-kqlq9" event={"ID":"57a41795-a558-4ecd-be74-25809affa0c5","Type":"ContainerStarted","Data":"12217dd795edb327db8787c6c505e89aa3064eb30092074be29c1ca03deaa65f"} Jul 15 12:47:32.052056 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:32.051906 2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-d79rw/perf-node-gather-daemonset-kqlq9" event={"ID":"57a41795-a558-4ecd-be74-25809affa0c5","Type":"ContainerStarted","Data":"d0fe0e519d703cf2dd3f5d545b2915b21a74d0428cb61c62c288b7f468eb1ae9"} Jul 15 12:47:32.052056 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:32.051950 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-d79rw/perf-node-gather-daemonset-kqlq9" Jul 15 12:47:32.069918 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:32.069868 2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-d79rw/perf-node-gather-daemonset-kqlq9" podStartSLOduration=2.06985596 podStartE2EDuration="2.06985596s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 12:47:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:47:32.068775249 +0000 UTC m=+2923.526152174" watchObservedRunningTime="2026-07-15 12:47:32.06985596 +0000 UTC m=+2923.527232885" Jul 15 12:47:32.557146 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:32.557105 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-7kt4q_56b02112-6aa5-42dd-aeab-0a296a6314b7/download-server/0.log" Jul 15 12:47:33.144563 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:33.144534 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-x26m7_d094f781-6c7b-435a-aef1-4f2e30138a3e/volume-data-source-validator/0.log" Jul 15 12:47:34.287213 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:34.287183 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-k5m9c_8db3f4f8-7a28-477d-9a5b-8df04d5a282a/dns/0.log" Jul 15 12:47:34.320134 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:34.320111 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-k5m9c_8db3f4f8-7a28-477d-9a5b-8df04d5a282a/kube-rbac-proxy/0.log" Jul 15 12:47:34.410956 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:34.410926 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fnhqk_b0228db4-1648-4484-8e9c-023c68c1e1e5/dns-node-resolver/0.log" Jul 15 12:47:35.028199 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:35.028163 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5bb6n_d09ffd58-1f50-4daa-8ce4-0c40ad65306b/node-ca/0.log" Jul 15 12:47:36.215452 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:36.215422 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-4lxxm_6dd415bd-a802-4b9d-8a67-b81b97863da4/discovery/0.log" Jul 15 12:47:36.235769 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:36.235745 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d7f886b65-qd67k_a347f0d8-4a4b-4f5d-83b8-082a96f4aa7a/kube-auth-proxy/0.log" Jul 15 12:47:37.185714 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:37.185685 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-26qs2_01fb3758-749d-4e30-b529-2250f27f4700/serve-healthcheck-canary/0.log" Jul 15 12:47:37.877837 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:37.877803 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8j9km_b1be73b4-e61c-4c4b-9830-749d3c5a544b/kube-rbac-proxy/0.log" Jul 15 12:47:37.901481 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:37.901452 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8j9km_b1be73b4-e61c-4c4b-9830-749d3c5a544b/exporter/0.log" Jul 15 12:47:37.926165 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:37.926138 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8j9km_b1be73b4-e61c-4c4b-9830-749d3c5a544b/extractor/0.log" Jul 15 12:47:38.074220 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:38.074197 2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-d79rw/perf-node-gather-daemonset-kqlq9" Jul 15 12:47:40.252047 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:40.252018 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-bh8kq_41bdfb9d-6267-4e66-8925-31ffacf48d39/manager/0.log" Jul 15 12:47:40.671584 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:40.671556 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-j4h5d_15f67716-be1c-4a3f-9dc9-d35980060c82/server/0.log" Jul 15 12:47:40.778837 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:40.778809 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-p2pr6_14ba6a32-b02c-4afd-9adb-4547d982708f/manager/0.log" Jul 15 12:47:41.006760 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:41.006634 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-694c4fcbc8-mdm8d_04eb66ea-bd75-40be-8683-2072ee07eb79/manager/0.log" Jul 15 12:47:41.027176 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:41.027148 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-694c4fcbc8-mdm8d_04eb66ea-bd75-40be-8683-2072ee07eb79/copy-manifests/0.log" Jul 15 12:47:41.990293 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:41.990258 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-4lxxm_6dd415bd-a802-4b9d-8a67-b81b97863da4/discovery/0.log" Jul 15 12:47:42.001469 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:42.001447 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d7f886b65-qd67k_a347f0d8-4a4b-4f5d-83b8-082a96f4aa7a/kube-auth-proxy/0.log" Jul 15 12:47:42.286216 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:42.286184 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-kwdh4_3ddb7416-fddb-4328-9b6d-d076a13f940a/openshift-lws-operator/0.log" Jul 15 12:47:43.520248 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:43.520207 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-4lxxm_6dd415bd-a802-4b9d-8a67-b81b97863da4/discovery/0.log" Jul 15 12:47:43.532039 ip-10-0-135-243 kubenswrapper[2587]: I0715 12:47:43.531891 2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d7f886b65-qd67k_a347f0d8-4a4b-4f5d-83b8-082a96f4aa7a/kube-auth-proxy/0.log"