Jul 07 15:13:01.873912 ip-10-0-138-11 systemd[1]: Starting Kubernetes Kubelet... Jul 07 15:13:02.265770 ip-10-0-138-11 kubenswrapper[2549]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 15:13:02.265770 ip-10-0-138-11 kubenswrapper[2549]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 07 15:13:02.265770 ip-10-0-138-11 kubenswrapper[2549]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 15:13:02.265770 ip-10-0-138-11 kubenswrapper[2549]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 07 15:13:02.265770 ip-10-0-138-11 kubenswrapper[2549]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 15:13:02.266675 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.266548 2549 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 07 15:13:02.273125 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273101 2549 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 15:13:02.273125 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273119 2549 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 15:13:02.273125 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273123 2549 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 15:13:02.273125 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273126 2549 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 15:13:02.273125 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273129 2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 15:13:02.273125 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273132 2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 15:13:02.273125 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273135 2549 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 15:13:02.273382 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273138 2549 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 15:13:02.273382 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273140 2549 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 15:13:02.273382 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273143 2549 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 15:13:02.273382 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273147 2549 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 15:13:02.273382 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273149 2549 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 15:13:02.273382 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273152 2549 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 15:13:02.273382 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273155 2549 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 15:13:02.273382 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273157 2549 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 15:13:02.273382 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273160 2549 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 15:13:02.273382 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273163 2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 15:13:02.273382 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273165 2549 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 15:13:02.273382 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273168 2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 15:13:02.273382 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273170 2549 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 15:13:02.273382 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273173 2549 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 15:13:02.273382 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273176 2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 15:13:02.273382 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273178 2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 15:13:02.273382 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273181 2549 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 15:13:02.273382 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273184 2549 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 15:13:02.273382 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273186 2549 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 15:13:02.273382 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273189 2549 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 15:13:02.273873 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273192 2549 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 15:13:02.273873 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273195 2549 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 15:13:02.273873 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273197 2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 15:13:02.273873 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273203 2549 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 15:13:02.273873 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273206 2549 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 15:13:02.273873 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273208 2549 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 15:13:02.273873 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273211 2549 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 15:13:02.273873 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273214 2549 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 15:13:02.273873 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273224 2549 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 15:13:02.273873 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273227 2549 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 15:13:02.273873 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273229 2549 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 15:13:02.273873 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273232 2549 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 15:13:02.273873 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273234 2549 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 15:13:02.273873 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273237 2549 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 15:13:02.273873 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273240 2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 15:13:02.273873 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273243 2549 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 15:13:02.273873 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273246 2549 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 15:13:02.273873 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273249 2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 15:13:02.273873 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273254 2549 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 15:13:02.274341 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273258 2549 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 15:13:02.274341 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273261 2549 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 15:13:02.274341 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273264 2549 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 15:13:02.274341 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273267 2549 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 15:13:02.274341 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273270 2549 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 15:13:02.274341 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273273 2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 15:13:02.274341 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273275 2549 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 15:13:02.274341 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273278 2549 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 15:13:02.274341 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273281 2549 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 15:13:02.274341 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273283 2549 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 15:13:02.274341 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273286 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 15:13:02.274341 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273288 2549 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 15:13:02.274341 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273291 2549 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 15:13:02.274341 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273293 2549 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 15:13:02.274341 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273296 2549 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 15:13:02.274341 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273299 2549 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 15:13:02.274341 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273302 2549 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 15:13:02.274341 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273305 2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 15:13:02.274341 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273308 2549 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 15:13:02.274341 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273311 2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 15:13:02.274859 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273313 2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 15:13:02.274859 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273315 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 15:13:02.274859 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273318 2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 15:13:02.274859 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273320 2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 15:13:02.274859 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273323 2549 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 15:13:02.274859 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273325 2549 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 15:13:02.274859 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273330 2549 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 15:13:02.274859 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273333 2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 15:13:02.274859 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273337 2549 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 15:13:02.274859 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273341 2549 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 15:13:02.274859 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273344 2549 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 15:13:02.274859 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273347 2549 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 15:13:02.274859 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273349 2549 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 15:13:02.274859 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273352 2549 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 15:13:02.274859 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273355 2549 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 15:13:02.274859 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273358 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 15:13:02.274859 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273361 2549 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 15:13:02.274859 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273364 2549 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 15:13:02.274859 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273367 2549 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 15:13:02.275319 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273372 2549 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 15:13:02.275319 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273375 2549 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 15:13:02.275319 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273378 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 15:13:02.275319 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273382 2549 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 15:13:02.275319 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273385 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 15:13:02.275319 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273388 2549 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 15:13:02.275319 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273412 2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 15:13:02.275319 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273416 2549 feature_gate.go:328] unrecognized feature gate: Example Jul 07 15:13:02.275319 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273419 2549 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 15:13:02.275319 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273422 2549 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 15:13:02.275319 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273425 2549 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 15:13:02.275319 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273428 2549 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 15:13:02.275319 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273430 2549 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 15:13:02.275319 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.273433 2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 15:13:02.275319 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274202 2549 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 15:13:02.275319 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274211 2549 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 15:13:02.275319 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274214 2549 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 15:13:02.275319 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274218 2549 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 15:13:02.275319 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274221 2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 15:13:02.275319 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274224 2549 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 15:13:02.275849 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274227 2549 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 15:13:02.275849 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274230 2549 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 15:13:02.275849 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274233 2549 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 15:13:02.275849 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274235 2549 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 15:13:02.275849 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274238 2549 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 15:13:02.275849 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274240 2549 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 15:13:02.275849 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274243 2549 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 15:13:02.275849 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274246 2549 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 15:13:02.275849 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274248 2549 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 15:13:02.275849 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274251 2549 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 15:13:02.275849 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274253 2549 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 15:13:02.275849 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274256 2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 15:13:02.275849 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274259 2549 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 15:13:02.275849 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274262 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 15:13:02.275849 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274264 2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 15:13:02.275849 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274268 2549 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 15:13:02.275849 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274270 2549 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 15:13:02.275849 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274273 2549 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 15:13:02.275849 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274275 2549 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 15:13:02.276319 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274278 2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 15:13:02.276319 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274280 2549 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 15:13:02.276319 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274283 2549 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 15:13:02.276319 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274285 2549 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 15:13:02.276319 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274287 2549 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 15:13:02.276319 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274290 2549 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 15:13:02.276319 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274292 2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 15:13:02.276319 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274295 2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 15:13:02.276319 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274297 2549 feature_gate.go:328] unrecognized feature gate: Example Jul 07 15:13:02.276319 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274300 2549 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 15:13:02.276319 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274302 2549 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 15:13:02.276319 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274305 2549 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 15:13:02.276319 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274308 2549 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 15:13:02.276319 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274311 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 15:13:02.276319 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274315 2549 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 15:13:02.276319 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274318 2549 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 15:13:02.276319 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274321 2549 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 15:13:02.276319 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274324 2549 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 15:13:02.276319 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274327 2549 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 15:13:02.276319 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274329 2549 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 15:13:02.276813 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274332 2549 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 15:13:02.276813 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274337 2549 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 15:13:02.276813 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274340 2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 15:13:02.276813 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274343 2549 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 15:13:02.276813 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274346 2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 15:13:02.276813 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274350 2549 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 15:13:02.276813 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274353 2549 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 15:13:02.276813 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274356 2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 15:13:02.276813 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274359 2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 15:13:02.276813 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274361 2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 15:13:02.276813 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274364 2549 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 15:13:02.276813 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274366 2549 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 15:13:02.276813 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274370 2549 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 15:13:02.276813 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274373 2549 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 15:13:02.276813 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274376 2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 15:13:02.276813 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274379 2549 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 15:13:02.276813 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274382 2549 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 15:13:02.276813 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274385 2549 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 15:13:02.276813 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274388 2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 15:13:02.277289 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274391 2549 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 15:13:02.277289 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274393 2549 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 15:13:02.277289 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274396 2549 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 15:13:02.277289 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274399 2549 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 15:13:02.277289 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274402 2549 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 15:13:02.277289 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274405 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 15:13:02.277289 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274407 2549 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 15:13:02.277289 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274410 2549 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 15:13:02.277289 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274414 2549 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 15:13:02.277289 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274417 2549 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 15:13:02.277289 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274419 2549 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 15:13:02.277289 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274422 2549 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 15:13:02.277289 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274424 2549 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 15:13:02.277289 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274427 2549 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 15:13:02.277289 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274430 2549 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 15:13:02.277289 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274432 2549 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 15:13:02.277289 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274434 2549 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 15:13:02.277289 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274437 2549 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 15:13:02.277289 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274439 2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 15:13:02.277289 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274443 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 15:13:02.277791 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274446 2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 15:13:02.277791 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274448 2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 15:13:02.277791 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274450 2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 15:13:02.277791 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274453 2549 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 15:13:02.277791 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274455 2549 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 15:13:02.277791 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274473 2549 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 15:13:02.277791 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274477 2549 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 15:13:02.277791 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274480 2549 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 15:13:02.277791 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274483 2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 15:13:02.277791 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274486 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 15:13:02.277791 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274488 2549 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 15:13:02.277791 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274491 2549 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 15:13:02.277791 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274493 2549 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 15:13:02.277791 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274496 2549 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 15:13:02.277791 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274498 2549 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 15:13:02.277791 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274903 2549 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 15:13:02.277791 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274909 2549 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 15:13:02.277791 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274912 2549 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 15:13:02.277791 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274914 2549 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 15:13:02.277791 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274917 2549 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 15:13:02.278295 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274920 2549 feature_gate.go:328] unrecognized feature gate: Example Jul 07 15:13:02.278295 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274923 2549 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 15:13:02.278295 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274925 2549 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 15:13:02.278295 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274928 2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 15:13:02.278295 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274930 2549 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 15:13:02.278295 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274933 2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 15:13:02.278295 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274936 2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 15:13:02.278295 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274938 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 15:13:02.278295 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274941 2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 15:13:02.278295 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274943 2549 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 15:13:02.278295 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274945 2549 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 15:13:02.278295 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274948 2549 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 15:13:02.278295 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274951 2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 15:13:02.278295 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274955 2549 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 15:13:02.278295 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274959 2549 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 15:13:02.278295 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274961 2549 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 15:13:02.278295 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274965 2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 15:13:02.278295 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274967 2549 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 15:13:02.278295 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274970 2549 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 15:13:02.278777 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274973 2549 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 15:13:02.278777 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274976 2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 15:13:02.278777 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274978 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 15:13:02.278777 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274982 2549 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 15:13:02.278777 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274984 2549 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 15:13:02.278777 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274987 2549 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 15:13:02.278777 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274989 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 15:13:02.278777 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274992 2549 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 15:13:02.278777 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.274994 2549 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 15:13:02.278777 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275003 2549 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 15:13:02.278777 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275005 2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 15:13:02.278777 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275008 2549 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 15:13:02.278777 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275010 2549 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 15:13:02.278777 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275013 2549 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 15:13:02.278777 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275016 2549 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 15:13:02.278777 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275019 2549 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 15:13:02.278777 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275023 2549 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 15:13:02.278777 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275026 2549 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 15:13:02.278777 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275028 2549 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 15:13:02.278777 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275031 2549 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 15:13:02.279260 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275033 2549 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 15:13:02.279260 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275036 2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 15:13:02.279260 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275041 2549 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 15:13:02.279260 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275045 2549 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 15:13:02.279260 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275048 2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 15:13:02.279260 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275052 2549 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 15:13:02.279260 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275055 2549 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 15:13:02.279260 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275058 2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 15:13:02.279260 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275060 2549 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 15:13:02.279260 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275063 2549 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 15:13:02.279260 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275066 2549 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 15:13:02.279260 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275068 2549 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 15:13:02.279260 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275071 2549 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 15:13:02.279260 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275074 2549 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 15:13:02.279260 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275076 2549 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 15:13:02.279260 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275079 2549 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 15:13:02.279260 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275082 2549 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 15:13:02.279260 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275084 2549 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 15:13:02.279260 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275087 2549 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 15:13:02.279730 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275089 2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 15:13:02.279730 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275092 2549 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 15:13:02.279730 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275096 2549 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 15:13:02.279730 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275099 2549 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 15:13:02.279730 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275101 2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 15:13:02.279730 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275104 2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 15:13:02.279730 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275107 2549 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 15:13:02.279730 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275109 2549 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 15:13:02.279730 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275112 2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 15:13:02.279730 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275115 2549 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 15:13:02.279730 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275117 2549 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 15:13:02.279730 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275120 2549 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 15:13:02.279730 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275123 2549 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 15:13:02.279730 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275125 2549 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 15:13:02.279730 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275128 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 15:13:02.279730 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275130 2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 15:13:02.279730 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275133 2549 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 15:13:02.279730 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275136 2549 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 15:13:02.279730 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275138 2549 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 15:13:02.279730 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275141 2549 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 15:13:02.280248 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275144 2549 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 15:13:02.280248 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275146 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 15:13:02.280248 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275149 2549 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 15:13:02.280248 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275151 2549 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 15:13:02.280248 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275154 2549 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 15:13:02.280248 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275156 2549 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 15:13:02.280248 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275159 2549 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 15:13:02.280248 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275163 2549 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 15:13:02.280248 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275166 2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 15:13:02.280248 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275168 2549 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 15:13:02.280248 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275171 2549 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 15:13:02.280248 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275173 2549 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 15:13:02.280248 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275176 2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 15:13:02.280248 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275178 2549 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 15:13:02.280248 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275182 2549 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 15:13:02.280248 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275185 2549 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 15:13:02.280248 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275254 2549 flags.go:64] FLAG: --address="0.0.0.0" Jul 07 15:13:02.280248 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275261 2549 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 07 15:13:02.280248 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275266 2549 flags.go:64] FLAG: --anonymous-auth="true" Jul 07 15:13:02.280248 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275270 2549 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 07 15:13:02.280248 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275274 2549 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 07 15:13:02.280766 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275277 2549 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 07 15:13:02.280766 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275281 2549 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 07 15:13:02.280766 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275285 2549 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 07 15:13:02.280766 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275288 2549 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 07 15:13:02.280766 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275291 2549 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 07 15:13:02.280766 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275295 2549 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 07 15:13:02.280766 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275298 2549 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 07 15:13:02.280766 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275301 2549 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 07 15:13:02.280766 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275304 2549 flags.go:64] FLAG: --cgroup-root="" Jul 07 15:13:02.280766 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275307 2549 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 07 15:13:02.280766 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275309 2549 flags.go:64] FLAG: --client-ca-file="" Jul 07 15:13:02.280766 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275313 2549 flags.go:64] FLAG: --cloud-provider="external" Jul 07 15:13:02.280766 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275316 2549 flags.go:64] FLAG: --cluster-dns="[]" Jul 07 15:13:02.280766 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275321 2549 flags.go:64] FLAG: --cluster-domain="" Jul 07 15:13:02.280766 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275324 2549 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 07 15:13:02.280766 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275327 2549 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 07 15:13:02.280766 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275330 2549 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 07 15:13:02.280766 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275334 2549 flags.go:64] FLAG: --container-log-max-files="5" Jul 07 15:13:02.280766 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275337 2549 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 07 15:13:02.280766 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275340 2549 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 07 15:13:02.280766 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275343 2549 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 07 15:13:02.280766 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275346 2549 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 07 15:13:02.280766 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275350 2549 flags.go:64] FLAG: --contention-profiling="false" Jul 07 15:13:02.280766 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275352 2549 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 07 15:13:02.281334 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275355 2549 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 07 15:13:02.281334 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275360 2549 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 07 15:13:02.281334 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275363 2549 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 07 15:13:02.281334 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275368 2549 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 07 15:13:02.281334 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275371 2549 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 07 15:13:02.281334 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275374 2549 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 07 15:13:02.281334 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275376 2549 flags.go:64] FLAG: --enable-load-reader="false" Jul 07 15:13:02.281334 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275379 2549 flags.go:64] FLAG: --enable-server="true" Jul 07 15:13:02.281334 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275382 2549 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 07 15:13:02.281334 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275389 2549 flags.go:64] FLAG: --event-burst="100" Jul 07 15:13:02.281334 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275392 2549 flags.go:64] FLAG: --event-qps="50" Jul 07 15:13:02.281334 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275395 2549 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 07 15:13:02.281334 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275398 2549 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 07 15:13:02.281334 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275401 2549 flags.go:64] FLAG: --eviction-hard="" Jul 07 15:13:02.281334 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275405 2549 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 07 15:13:02.281334 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275408 2549 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 07 15:13:02.281334 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275411 2549 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 07 15:13:02.281334 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275414 2549 flags.go:64] FLAG: --eviction-soft="" Jul 07 15:13:02.281334 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275417 2549 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 07 15:13:02.281334 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275419 2549 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 07 15:13:02.281334 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275423 2549 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 07 15:13:02.281334 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275426 2549 flags.go:64] FLAG: --experimental-mounter-path="" Jul 07 15:13:02.281334 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275429 2549 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 07 15:13:02.281334 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275431 2549 flags.go:64] FLAG: --fail-swap-on="true" Jul 07 15:13:02.281334 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275434 2549 flags.go:64] FLAG: --feature-gates="" Jul 07 15:13:02.282002 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275438 2549 flags.go:64] FLAG: --file-check-frequency="20s" Jul 07 15:13:02.282002 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275441 2549 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 07 15:13:02.282002 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275444 2549 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 07 15:13:02.282002 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275447 2549 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 07 15:13:02.282002 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275450 2549 flags.go:64] FLAG: --healthz-port="10248" Jul 07 15:13:02.282002 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275453 2549 flags.go:64] FLAG: --help="false" Jul 07 15:13:02.282002 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275455 2549 flags.go:64] FLAG: --hostname-override="ip-10-0-138-11.ec2.internal" Jul 07 15:13:02.282002 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275460 2549 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 07 15:13:02.282002 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275465 2549 flags.go:64] FLAG: --http-check-frequency="20s" Jul 07 15:13:02.282002 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275468 2549 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 07 15:13:02.282002 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275471 2549 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 07 15:13:02.282002 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275474 2549 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 07 15:13:02.282002 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275477 2549 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 07 15:13:02.282002 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275480 2549 flags.go:64] FLAG: --image-service-endpoint="" Jul 07 15:13:02.282002 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275483 2549 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 07 15:13:02.282002 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275486 2549 flags.go:64] FLAG: --kube-api-burst="100" Jul 07 15:13:02.282002 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275489 2549 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 07 15:13:02.282002 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275492 2549 flags.go:64] FLAG: --kube-api-qps="50" Jul 07 15:13:02.282002 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275495 2549 flags.go:64] FLAG: --kube-reserved="" Jul 07 15:13:02.282002 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275497 2549 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 07 15:13:02.282002 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275500 2549 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 07 15:13:02.282002 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275503 2549 flags.go:64] FLAG: --kubelet-cgroups="" Jul 07 15:13:02.282002 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275506 2549 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 07 15:13:02.282002 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275509 2549 flags.go:64] FLAG: --lock-file="" Jul 07 15:13:02.282613 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275512 2549 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 07 15:13:02.282613 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275515 2549 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 07 15:13:02.282613 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275518 2549 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 07 15:13:02.282613 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275523 2549 flags.go:64] FLAG: --log-json-split-stream="false" Jul 07 15:13:02.282613 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275527 2549 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 07 15:13:02.282613 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275530 2549 flags.go:64] FLAG: --log-text-split-stream="false" Jul 07 15:13:02.282613 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275532 2549 flags.go:64] FLAG: --logging-format="text" Jul 07 15:13:02.282613 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275535 2549 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 07 15:13:02.282613 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275538 2549 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 07 15:13:02.282613 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275541 2549 flags.go:64] FLAG: --manifest-url="" Jul 07 15:13:02.282613 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275544 2549 flags.go:64] FLAG: --manifest-url-header="" Jul 07 15:13:02.282613 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275548 2549 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 07 15:13:02.282613 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275551 2549 flags.go:64] FLAG: --max-open-files="1000000" Jul 07 15:13:02.282613 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275555 2549 flags.go:64] FLAG: --max-pods="110" Jul 07 15:13:02.282613 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275558 2549 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 07 15:13:02.282613 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275564 2549 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 07 15:13:02.282613 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275568 2549 flags.go:64] FLAG: --memory-manager-policy="None" Jul 07 15:13:02.282613 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275571 2549 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 07 15:13:02.282613 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275574 2549 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 07 15:13:02.282613 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275577 2549 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 07 15:13:02.282613 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275594 2549 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 07 15:13:02.282613 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275601 2549 flags.go:64] FLAG: --node-status-max-images="50" Jul 07 15:13:02.282613 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275604 2549 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 07 15:13:02.282613 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275607 2549 flags.go:64] FLAG: --oom-score-adj="-999" Jul 07 15:13:02.282613 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275610 2549 flags.go:64] FLAG: --pod-cidr="" Jul 07 15:13:02.283286 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275613 2549 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 07 15:13:02.283286 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275618 2549 flags.go:64] FLAG: --pod-manifest-path="" Jul 07 15:13:02.283286 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275621 2549 flags.go:64] FLAG: --pod-max-pids="-1" Jul 07 15:13:02.283286 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275624 2549 flags.go:64] FLAG: --pods-per-core="0" Jul 07 15:13:02.283286 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275627 2549 flags.go:64] FLAG: --port="10250" Jul 07 15:13:02.283286 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275630 2549 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 07 15:13:02.283286 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275633 2549 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0abf0ca7891d87995" Jul 07 15:13:02.283286 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275637 2549 flags.go:64] FLAG: --qos-reserved="" Jul 07 15:13:02.283286 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275639 2549 flags.go:64] FLAG: --read-only-port="10255" Jul 07 15:13:02.283286 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275642 2549 flags.go:64] FLAG: --register-node="true" Jul 07 15:13:02.283286 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275645 2549 flags.go:64] FLAG: --register-with-taints="" Jul 07 15:13:02.283286 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275648 2549 flags.go:64] FLAG: --registry-burst="10" Jul 07 15:13:02.283286 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275652 2549 flags.go:64] FLAG: --registry-qps="5" Jul 07 15:13:02.283286 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275654 2549 flags.go:64] FLAG: --reserved-cpus="" Jul 07 15:13:02.283286 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275657 2549 flags.go:64] FLAG: --reserved-memory="" Jul 07 15:13:02.283286 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275661 2549 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 07 15:13:02.283286 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275664 2549 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 07 15:13:02.283286 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275667 2549 flags.go:64] FLAG: --rotate-certificates="false" Jul 07 15:13:02.283286 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275670 2549 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 07 15:13:02.283286 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275672 2549 flags.go:64] FLAG: --runonce="false" Jul 07 15:13:02.283286 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275675 2549 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 07 15:13:02.283286 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275678 2549 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 07 15:13:02.283286 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275681 2549 flags.go:64] FLAG: --seccomp-default="false" Jul 07 15:13:02.283286 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275686 2549 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 07 15:13:02.283286 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275690 2549 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 07 15:13:02.283998 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275693 2549 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 07 15:13:02.283998 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275696 2549 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 07 15:13:02.283998 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275699 2549 flags.go:64] FLAG: --storage-driver-password="root" Jul 07 15:13:02.283998 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275702 2549 flags.go:64] FLAG: --storage-driver-secure="false" Jul 07 15:13:02.283998 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275705 2549 flags.go:64] FLAG: --storage-driver-table="stats" Jul 07 15:13:02.283998 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275708 2549 flags.go:64] FLAG: --storage-driver-user="root" Jul 07 15:13:02.283998 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275711 2549 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 07 15:13:02.283998 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275714 2549 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 07 15:13:02.283998 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275716 2549 flags.go:64] FLAG: --system-cgroups="" Jul 07 15:13:02.283998 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275719 2549 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 07 15:13:02.283998 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275724 2549 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 07 15:13:02.283998 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275727 2549 flags.go:64] FLAG: --tls-cert-file="" Jul 07 15:13:02.283998 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275730 2549 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 07 15:13:02.283998 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275735 2549 flags.go:64] FLAG: --tls-min-version="" Jul 07 15:13:02.283998 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275738 2549 flags.go:64] FLAG: --tls-private-key-file="" Jul 07 15:13:02.283998 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275740 2549 flags.go:64] FLAG: --topology-manager-policy="none" Jul 07 15:13:02.283998 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275743 2549 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 07 15:13:02.283998 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275746 2549 flags.go:64] FLAG: --topology-manager-scope="container" Jul 07 15:13:02.283998 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275749 2549 flags.go:64] FLAG: --v="2" Jul 07 15:13:02.283998 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275753 2549 flags.go:64] FLAG: --version="false" Jul 07 15:13:02.283998 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275761 2549 flags.go:64] FLAG: --vmodule="" Jul 07 15:13:02.283998 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275765 2549 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 07 15:13:02.283998 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.275768 2549 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 07 15:13:02.283998 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275887 2549 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 15:13:02.284599 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275891 2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 15:13:02.284599 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275894 2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 15:13:02.284599 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275896 2549 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 15:13:02.284599 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275899 2549 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 15:13:02.284599 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275902 2549 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 15:13:02.284599 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275904 2549 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 15:13:02.284599 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275909 2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 15:13:02.284599 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275912 2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 15:13:02.284599 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275916 2549 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 15:13:02.284599 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275919 2549 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 15:13:02.284599 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275923 2549 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 15:13:02.284599 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275925 2549 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 15:13:02.284599 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275928 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 15:13:02.284599 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275932 2549 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 15:13:02.284599 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275936 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 15:13:02.284599 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275939 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 15:13:02.284599 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275942 2549 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 15:13:02.284599 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275945 2549 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 15:13:02.284599 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275948 2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 15:13:02.285058 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275951 2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 15:13:02.285058 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275954 2549 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 15:13:02.285058 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275956 2549 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 15:13:02.285058 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275959 2549 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 15:13:02.285058 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275962 2549 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 15:13:02.285058 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275964 2549 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 15:13:02.285058 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275967 2549 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 15:13:02.285058 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275969 2549 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 15:13:02.285058 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275972 2549 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 15:13:02.285058 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275974 2549 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 15:13:02.285058 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275983 2549 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 15:13:02.285058 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275986 2549 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 15:13:02.285058 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275988 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 15:13:02.285058 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275991 2549 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 15:13:02.285058 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275993 2549 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 15:13:02.285058 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275996 2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 15:13:02.285058 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.275998 2549 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 15:13:02.285058 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276001 2549 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 15:13:02.285058 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276003 2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 15:13:02.285058 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276009 2549 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 15:13:02.285549 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276011 2549 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 15:13:02.285549 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276014 2549 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 15:13:02.285549 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276016 2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 15:13:02.285549 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276019 2549 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 15:13:02.285549 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276021 2549 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 15:13:02.285549 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276025 2549 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 15:13:02.285549 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276029 2549 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 15:13:02.285549 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276032 2549 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 15:13:02.285549 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276035 2549 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 15:13:02.285549 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276037 2549 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 15:13:02.285549 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276040 2549 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 15:13:02.285549 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276043 2549 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 15:13:02.285549 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276045 2549 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 15:13:02.285549 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276048 2549 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 15:13:02.285549 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276050 2549 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 15:13:02.285549 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276053 2549 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 15:13:02.285549 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276055 2549 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 15:13:02.285549 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276058 2549 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 15:13:02.285549 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276061 2549 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 15:13:02.285549 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276064 2549 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 15:13:02.286092 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276066 2549 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 15:13:02.286092 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276069 2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 15:13:02.286092 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276071 2549 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 15:13:02.286092 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276075 2549 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 15:13:02.286092 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276077 2549 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 15:13:02.286092 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276080 2549 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 15:13:02.286092 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276082 2549 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 15:13:02.286092 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276084 2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 15:13:02.286092 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276087 2549 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 15:13:02.286092 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276090 2549 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 15:13:02.286092 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276092 2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 15:13:02.286092 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276096 2549 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 15:13:02.286092 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276099 2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 15:13:02.286092 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276101 2549 feature_gate.go:328] unrecognized feature gate: Example Jul 07 15:13:02.286092 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276104 2549 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 15:13:02.286092 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276106 2549 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 15:13:02.286092 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276109 2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 15:13:02.286092 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276111 2549 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 15:13:02.286092 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276114 2549 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 15:13:02.286092 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276117 2549 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 15:13:02.286613 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276119 2549 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 15:13:02.286613 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276122 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 15:13:02.286613 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276124 2549 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 15:13:02.286613 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276127 2549 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 15:13:02.286613 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276129 2549 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 15:13:02.286613 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276132 2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 15:13:02.286613 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276135 2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 15:13:02.286613 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276137 2549 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 15:13:02.286613 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276140 2549 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 15:13:02.286613 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276142 2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 15:13:02.286613 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276145 2549 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 15:13:02.286613 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276147 2549 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 15:13:02.286613 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276150 2549 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 15:13:02.286613 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276153 2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 15:13:02.286613 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276155 2549 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 15:13:02.286613 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276158 2549 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 15:13:02.286613 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276162 2549 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 15:13:02.286613 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276164 2549 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 15:13:02.286613 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.276167 2549 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 15:13:02.287086 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.276175 2549 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 15:13:02.287086 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.282896 2549 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 07 15:13:02.287086 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.282914 2549 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 07 15:13:02.287086 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.282929 2549 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 07 15:13:02.287086 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.282935 2549 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 07 15:13:02.287086 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283000 2549 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 15:13:02.287086 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283006 2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 15:13:02.287086 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283009 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 15:13:02.287086 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283012 2549 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 15:13:02.287086 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283015 2549 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 15:13:02.287086 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283018 2549 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 15:13:02.287086 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283021 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 15:13:02.287086 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283024 2549 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 15:13:02.287086 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283026 2549 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 15:13:02.287086 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283029 2549 feature_gate.go:328] unrecognized feature gate: Example Jul 07 15:13:02.287569 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283031 2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 15:13:02.287569 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283034 2549 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 15:13:02.287569 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283037 2549 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 15:13:02.287569 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283040 2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 15:13:02.287569 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283042 2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 15:13:02.287569 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283045 2549 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 15:13:02.287569 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283048 2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 15:13:02.287569 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283050 2549 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 15:13:02.287569 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283053 2549 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 15:13:02.287569 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283055 2549 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 15:13:02.287569 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283058 2549 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 15:13:02.287569 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283060 2549 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 15:13:02.287569 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283064 2549 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 15:13:02.287569 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283068 2549 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 15:13:02.287569 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283071 2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 15:13:02.287569 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283074 2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 15:13:02.287569 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283077 2549 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 15:13:02.287569 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283081 2549 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 15:13:02.287569 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283084 2549 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 15:13:02.288069 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283088 2549 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 15:13:02.288069 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283091 2549 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 15:13:02.288069 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283094 2549 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 15:13:02.288069 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283098 2549 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 15:13:02.288069 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283100 2549 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 15:13:02.288069 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283103 2549 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 15:13:02.288069 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283106 2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 15:13:02.288069 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283109 2549 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 15:13:02.288069 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283111 2549 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 15:13:02.288069 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283114 2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 15:13:02.288069 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283116 2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 15:13:02.288069 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283119 2549 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 15:13:02.288069 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283122 2549 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 15:13:02.288069 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283125 2549 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 15:13:02.288069 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283127 2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 15:13:02.288069 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283129 2549 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 15:13:02.288069 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283132 2549 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 15:13:02.288069 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283135 2549 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 15:13:02.288069 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283138 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 15:13:02.288069 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283140 2549 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 15:13:02.288886 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283143 2549 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 15:13:02.288886 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283145 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 15:13:02.288886 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283148 2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 15:13:02.288886 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283151 2549 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 15:13:02.288886 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283153 2549 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 15:13:02.288886 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283156 2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 15:13:02.288886 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283158 2549 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 15:13:02.288886 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283162 2549 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 15:13:02.288886 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283164 2549 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 15:13:02.288886 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283167 2549 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 15:13:02.288886 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283169 2549 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 15:13:02.288886 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283171 2549 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 15:13:02.288886 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283175 2549 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 15:13:02.288886 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283179 2549 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 15:13:02.288886 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283181 2549 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 15:13:02.288886 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283184 2549 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 15:13:02.288886 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283187 2549 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 15:13:02.288886 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283189 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 15:13:02.288886 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283192 2549 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 15:13:02.288886 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283194 2549 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 15:13:02.289612 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283197 2549 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 15:13:02.289612 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283199 2549 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 15:13:02.289612 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283202 2549 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 15:13:02.289612 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283205 2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 15:13:02.289612 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283207 2549 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 15:13:02.289612 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283210 2549 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 15:13:02.289612 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283213 2549 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 15:13:02.289612 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283215 2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 15:13:02.289612 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283218 2549 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 15:13:02.289612 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283220 2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 15:13:02.289612 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283223 2549 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 15:13:02.289612 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283225 2549 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 15:13:02.289612 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283228 2549 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 15:13:02.289612 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283230 2549 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 15:13:02.289612 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283233 2549 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 15:13:02.289612 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283235 2549 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 15:13:02.289612 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283238 2549 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 15:13:02.289612 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283241 2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 15:13:02.289612 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283243 2549 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 15:13:02.290168 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283247 2549 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 15:13:02.290168 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283251 2549 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 15:13:02.290168 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283254 2549 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 15:13:02.290168 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283256 2549 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 15:13:02.290168 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283259 2549 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 15:13:02.290168 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283262 2549 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 15:13:02.290168 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283265 2549 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 15:13:02.290168 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283272 2549 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 15:13:02.290168 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283275 2549 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 15:13:02.290168 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283278 2549 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 15:13:02.290168 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283281 2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 15:13:02.290168 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.283286 2549 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 15:13:02.290168 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283406 2549 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 15:13:02.290168 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283410 2549 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 15:13:02.290168 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283413 2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 15:13:02.290576 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283416 2549 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 15:13:02.290576 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283419 2549 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 15:13:02.290576 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283422 2549 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 15:13:02.290576 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283425 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 15:13:02.290576 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283428 2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 15:13:02.290576 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283431 2549 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 15:13:02.290576 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283434 2549 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 15:13:02.290576 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283438 2549 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 15:13:02.290576 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283441 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 15:13:02.290576 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283444 2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 15:13:02.290576 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283447 2549 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 15:13:02.290576 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283450 2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 15:13:02.290576 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283453 2549 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 15:13:02.290576 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283456 2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 15:13:02.290576 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283459 2549 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 15:13:02.290576 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283461 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 15:13:02.290576 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283464 2549 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 15:13:02.290576 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283466 2549 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 15:13:02.290576 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283469 2549 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 15:13:02.291421 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283471 2549 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 15:13:02.291421 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283474 2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 15:13:02.291421 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283477 2549 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 15:13:02.291421 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283480 2549 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 15:13:02.291421 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283483 2549 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 15:13:02.291421 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283485 2549 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 15:13:02.291421 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283488 2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 15:13:02.291421 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283491 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 15:13:02.291421 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283493 2549 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 15:13:02.291421 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283496 2549 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 15:13:02.291421 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283498 2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 15:13:02.291421 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283501 2549 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 15:13:02.291421 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283503 2549 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 15:13:02.291421 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283506 2549 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 15:13:02.291421 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283508 2549 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 15:13:02.291421 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283511 2549 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 15:13:02.291421 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283513 2549 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 15:13:02.291421 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283516 2549 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 15:13:02.291421 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283518 2549 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 15:13:02.292014 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283521 2549 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 15:13:02.292014 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283523 2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 15:13:02.292014 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283525 2549 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 15:13:02.292014 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283529 2549 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 15:13:02.292014 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283533 2549 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 15:13:02.292014 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283536 2549 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 15:13:02.292014 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283538 2549 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 15:13:02.292014 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283541 2549 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 15:13:02.292014 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283543 2549 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 15:13:02.292014 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283546 2549 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 15:13:02.292014 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283548 2549 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 15:13:02.292014 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283550 2549 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 15:13:02.292014 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283553 2549 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 15:13:02.292014 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283555 2549 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 15:13:02.292014 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283558 2549 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 15:13:02.292014 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283561 2549 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 15:13:02.292014 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283564 2549 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 15:13:02.292014 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283566 2549 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 15:13:02.292014 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283569 2549 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 15:13:02.292014 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283571 2549 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 15:13:02.292746 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283574 2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 15:13:02.292746 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283577 2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 15:13:02.292746 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283596 2549 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 15:13:02.292746 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283599 2549 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 15:13:02.292746 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283602 2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 15:13:02.292746 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283604 2549 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 15:13:02.292746 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283607 2549 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 15:13:02.292746 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283610 2549 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 15:13:02.292746 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283612 2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 15:13:02.292746 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283615 2549 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 15:13:02.292746 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283617 2549 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 15:13:02.292746 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283620 2549 feature_gate.go:328] unrecognized feature gate: Example Jul 07 15:13:02.292746 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283623 2549 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 15:13:02.292746 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283625 2549 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 15:13:02.292746 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283627 2549 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 15:13:02.292746 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283630 2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 15:13:02.292746 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283632 2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 15:13:02.292746 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283635 2549 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 15:13:02.292746 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283637 2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 15:13:02.292746 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283640 2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 15:13:02.293405 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283642 2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 15:13:02.293405 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283646 2549 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 15:13:02.293405 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283649 2549 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 15:13:02.293405 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283652 2549 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 15:13:02.293405 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283654 2549 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 15:13:02.293405 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283657 2549 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 15:13:02.293405 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283660 2549 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 15:13:02.293405 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283663 2549 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 15:13:02.293405 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283666 2549 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 15:13:02.293405 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283669 2549 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 15:13:02.293405 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283671 2549 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 15:13:02.293405 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283674 2549 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 15:13:02.293405 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283677 2549 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 15:13:02.293405 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283680 2549 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 15:13:02.293405 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283683 2549 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 15:13:02.293405 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283686 2549 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 15:13:02.293405 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283688 2549 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 15:13:02.293405 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:02.283690 2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 15:13:02.294085 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.283695 2549 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 15:13:02.294085 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.283813 2549 server.go:962] "Client rotation is on, will bootstrap in background" Jul 07 15:13:02.294085 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.285664 2549 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 07 15:13:02.294085 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.287241 2549 server.go:1019] "Starting client certificate rotation" Jul 07 15:13:02.294085 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.287360 2549 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 15:13:02.294085 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.287409 2549 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 15:13:02.294085 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.289848 2549 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 15:13:02.294812 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.294794 2549 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 15:13:02.307910 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.307892 2549 log.go:25] "Validated CRI v1 runtime API" Jul 07 15:13:02.309863 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.309849 2549 log.go:25] "Validated CRI v1 image API" Jul 07 15:13:02.310644 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.310628 2549 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 07 15:13:02.313114 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.313094 2549 fs.go:135] Filesystem UUIDs: map[0a7d060b-b66a-42fd-9580-6c86e62480e3:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 9f4b4064-78e5-43cb-8288-7bca3a760d91:/dev/nvme0n1p3] Jul 07 15:13:02.313172 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.313112 2549 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 07 15:13:02.318198 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.318085 2549 manager.go:216] Machine: {Timestamp:2026-07-07 15:13:02.316750164 +0000 UTC m=+0.324227604 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100755 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec21c62900f38f8c21cfa3e30cf45730 SystemUUID:ec21c629-00f3-8f8c-21cf-a3e30cf45730 BootID:dc6e2fd0-3f26-46bd-8c12-ef2d6d19d466 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:dc:05:d5:98:fb Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:dc:05:d5:98:fb Speed:0 Mtu:9001} {Name:ovs-system MacAddress:f2:52:cf:b0:ca:2e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 07 15:13:02.318198 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.318196 2549 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 07 15:13:02.318306 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.318277 2549 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 07 15:13:02.321081 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.321052 2549 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 07 15:13:02.321235 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.321084 2549 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-11.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 07 15:13:02.322435 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.322424 2549 topology_manager.go:138] "Creating topology manager with none policy" Jul 07 15:13:02.322473 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.322438 2549 container_manager_linux.go:309] "Creating device plugin manager" Jul 07 15:13:02.322473 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.322446 2549 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 15:13:02.323062 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.323051 2549 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 15:13:02.324271 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.324260 2549 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 07 15:13:02.326911 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.326899 2549 state_mem.go:36] "Initialized new in-memory state store" Jul 07 15:13:02.327072 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.327063 2549 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 07 15:13:02.327844 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.327835 2549 kubelet.go:486] "Attempting to sync node with API server" Jul 07 15:13:02.327883 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.327850 2549 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 07 15:13:02.327883 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.327869 2549 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 07 15:13:02.327883 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.327879 2549 kubelet.go:398] "Adding apiserver pod source" Jul 07 15:13:02.327963 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.327889 2549 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 07 15:13:02.328256 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.328245 2549 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 15:13:02.328302 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.328269 2549 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 15:13:02.330539 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.330522 2549 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 07 15:13:02.332351 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.332338 2549 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 07 15:13:02.332396 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.332368 2549 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 07 15:13:02.332576 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.332560 2549 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 15:13:02.333434 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.333422 2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 07 15:13:02.333478 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.333438 2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 07 15:13:02.333478 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.333445 2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 07 15:13:02.333478 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.333450 2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 07 15:13:02.333478 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.333460 2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 07 15:13:02.333478 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.333465 2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 07 15:13:02.333478 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.333473 2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 07 15:13:02.333478 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.333478 2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 07 15:13:02.333678 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.333484 2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 07 15:13:02.333678 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.333495 2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 07 15:13:02.333678 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.333510 2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 07 15:13:02.333678 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.333516 2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 07 15:13:02.334237 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.334227 2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 07 15:13:02.334237 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.334236 2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 07 15:13:02.337317 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.337305 2549 server.go:1268] "Started kubelet" Jul 07 15:13:02.337404 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.337381 2549 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 07 15:13:02.337451 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.337389 2549 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 07 15:13:02.337451 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.337440 2549 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 07 15:13:02.338042 ip-10-0-138-11 systemd[1]: Started Kubernetes Kubelet. Jul 07 15:13:02.341014 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.340995 2549 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 07 15:13:02.342657 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.342640 2549 server.go:310] "Adding debug handlers to kubelet server" Jul 07 15:13:02.342902 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:02.342881 2549 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 07 15:13:02.347673 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.347648 2549 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 07 15:13:02.347767 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.347676 2549 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 07 15:13:02.347819 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:02.347781 2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-11.ec2.internal\" not found" Jul 07 15:13:02.348286 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.348257 2549 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 07 15:13:02.348286 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.348276 2549 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 07 15:13:02.348410 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.348362 2549 reconstruct.go:97] "Volume reconstruction finished" Jul 07 15:13:02.348410 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.348371 2549 reconciler.go:29] "Reconciler: start to sync state" Jul 07 15:13:02.348410 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.348386 2549 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 07 15:13:02.348757 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.348740 2549 factory.go:55] Registering systemd factory Jul 07 15:13:02.348757 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.348758 2549 factory.go:223] Registration of the systemd container factory successfully Jul 07 15:13:02.348919 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.348907 2549 factory.go:153] Registering CRI-O factory Jul 07 15:13:02.348969 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.348922 2549 factory.go:223] Registration of the crio container factory successfully Jul 07 15:13:02.349023 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.348968 2549 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 07 15:13:02.349023 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.348990 2549 factory.go:103] Registering Raw factory Jul 07 15:13:02.349023 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.349002 2549 manager.go:1196] Started watching for new ooms in manager Jul 07 15:13:02.349604 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.349572 2549 manager.go:349] Starting recovery of all containers Jul 07 15:13:02.351423 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:02.351399 2549 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 07 15:13:02.351493 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:02.351458 2549 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-11.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 07 15:13:02.351667 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.351649 2549 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-11.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 07 15:13:02.351733 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:02.351711 2549 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 07 15:13:02.351819 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:02.351792 2549 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-11.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 07 15:13:02.352822 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:02.351774 2549 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-11.ec2.internal.18c00aab146fecca default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-11.ec2.internal,UID:ip-10-0-138-11.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-11.ec2.internal,},FirstTimestamp:2026-07-07 15:13:02.337281226 +0000 UTC m=+0.344758665,LastTimestamp:2026-07-07 15:13:02.337281226 +0000 UTC m=+0.344758665,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-11.ec2.internal,}" Jul 07 15:13:02.359267 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.359250 2549 manager.go:354] Recovery completed Jul 07 15:13:02.363382 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.363370 2549 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 15:13:02.365307 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.365282 2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-11.ec2.internal" event="NodeHasSufficientMemory" Jul 07 15:13:02.365361 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.365315 2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-11.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 15:13:02.365361 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.365325 2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-11.ec2.internal" event="NodeHasSufficientPID" Jul 07 15:13:02.365897 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.365885 2549 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 07 15:13:02.365897 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.365895 2549 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 07 15:13:02.365977 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.365911 2549 state_mem.go:36] "Initialized new in-memory state store" Jul 07 15:13:02.365977 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.365915 2549 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 15:13:02.366554 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.366539 2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-11.ec2.internal" event="NodeHasSufficientMemory" Jul 07 15:13:02.366637 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.366563 2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-11.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 15:13:02.366637 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.366593 2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-11.ec2.internal" event="NodeHasSufficientPID" Jul 07 15:13:02.366925 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.366910 2549 policy_none.go:49] "None policy: Start" Jul 07 15:13:02.366925 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.366926 2549 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 07 15:13:02.367019 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.366936 2549 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 07 15:13:02.367832 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.367822 2549 policy_none.go:47] "Start" Jul 07 15:13:02.369476 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:02.369405 2549 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-11.ec2.internal.18c00aab161b7559 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-11.ec2.internal,UID:ip-10-0-138-11.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-138-11.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-138-11.ec2.internal,},FirstTimestamp:2026-07-07 15:13:02.365300057 +0000 UTC m=+0.372777497,LastTimestamp:2026-07-07 15:13:02.365300057 +0000 UTC m=+0.372777497,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-11.ec2.internal,}" Jul 07 15:13:02.381589 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:02.381428 2549 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-11.ec2.internal.18c00aab161bc3ec default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-11.ec2.internal,UID:ip-10-0-138-11.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-138-11.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-138-11.ec2.internal,},FirstTimestamp:2026-07-07 15:13:02.365320172 +0000 UTC m=+0.372797612,LastTimestamp:2026-07-07 15:13:02.365320172 +0000 UTC m=+0.372797612,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-11.ec2.internal,}" Jul 07 15:13:02.392787 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:02.392715 2549 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-11.ec2.internal.18c00aab161be6fb default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-11.ec2.internal,UID:ip-10-0-138-11.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-138-11.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-138-11.ec2.internal,},FirstTimestamp:2026-07-07 15:13:02.365329147 +0000 UTC m=+0.372806586,LastTimestamp:2026-07-07 15:13:02.365329147 +0000 UTC m=+0.372806586,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-11.ec2.internal,}" Jul 07 15:13:02.398732 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.398721 2549 manager.go:337] "Starting Device Plugin manager" Jul 07 15:13:02.398803 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:02.398756 2549 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 07 15:13:02.398803 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.398766 2549 server.go:87] "Starting device plugin registration server" Jul 07 15:13:02.399008 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.398997 2549 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 07 15:13:02.399041 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.399015 2549 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 07 15:13:02.399121 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.399101 2549 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 07 15:13:02.399195 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.399177 2549 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 07 15:13:02.399195 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.399188 2549 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 07 15:13:02.399650 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:02.399634 2549 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 07 15:13:02.399717 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:02.399672 2549 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-11.ec2.internal\" not found" Jul 07 15:13:02.407077 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:02.407016 2549 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-138-11.ec2.internal.18c00aab161b7559\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-11.ec2.internal.18c00aab161b7559 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-11.ec2.internal,UID:ip-10-0-138-11.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-138-11.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-138-11.ec2.internal,},FirstTimestamp:2026-07-07 15:13:02.365300057 +0000 UTC m=+0.372777497,LastTimestamp:2026-07-07 15:13:02.36655196 +0000 UTC m=+0.374029399,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-11.ec2.internal,}" Jul 07 15:13:02.416745 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:02.416684 2549 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-138-11.ec2.internal.18c00aab161bc3ec\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-11.ec2.internal.18c00aab161bc3ec default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-11.ec2.internal,UID:ip-10-0-138-11.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-138-11.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-138-11.ec2.internal,},FirstTimestamp:2026-07-07 15:13:02.365320172 +0000 UTC m=+0.372797612,LastTimestamp:2026-07-07 15:13:02.366567789 +0000 UTC m=+0.374045228,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-11.ec2.internal,}" Jul 07 15:13:02.424795 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:02.424733 2549 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-138-11.ec2.internal.18c00aab161be6fb\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-11.ec2.internal.18c00aab161be6fb default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-11.ec2.internal,UID:ip-10-0-138-11.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-138-11.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-138-11.ec2.internal,},FirstTimestamp:2026-07-07 15:13:02.365329147 +0000 UTC m=+0.372806586,LastTimestamp:2026-07-07 15:13:02.366597909 +0000 UTC m=+0.374075348,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-11.ec2.internal,}" Jul 07 15:13:02.433493 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:02.433419 2549 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-11.ec2.internal.18c00aab18f305f5 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-11.ec2.internal,UID:ip-10-0-138-11.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-138-11.ec2.internal,},FirstTimestamp:2026-07-07 15:13:02.412981749 +0000 UTC m=+0.420459181,LastTimestamp:2026-07-07 15:13:02.412981749 +0000 UTC m=+0.420459181,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-11.ec2.internal,}" Jul 07 15:13:02.446536 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.446511 2549 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-h2w7v" Jul 07 15:13:02.465783 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.465756 2549 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-h2w7v" Jul 07 15:13:02.499261 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.499234 2549 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 15:13:02.500168 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.500148 2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-11.ec2.internal" event="NodeHasSufficientMemory" Jul 07 15:13:02.500236 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.500188 2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-11.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 15:13:02.500236 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.500201 2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-11.ec2.internal" event="NodeHasSufficientPID" Jul 07 15:13:02.500236 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.500227 2549 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-11.ec2.internal" Jul 07 15:13:02.521413 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.521358 2549 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-11.ec2.internal" Jul 07 15:13:02.521413 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:02.521384 2549 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-11.ec2.internal\": node \"ip-10-0-138-11.ec2.internal\" not found" Jul 07 15:13:02.526519 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.526491 2549 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 07 15:13:02.527872 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.527853 2549 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 07 15:13:02.527952 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.527894 2549 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 07 15:13:02.527952 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.527915 2549 kubelet.go:2448] "Starting kubelet main sync loop" Jul 07 15:13:02.527952 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:02.527943 2549 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 07 15:13:02.530004 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.529988 2549 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 15:13:02.545147 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:02.545131 2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-11.ec2.internal\" not found" Jul 07 15:13:02.628042 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.628007 2549 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-11.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-11.ec2.internal"] Jul 07 15:13:02.628154 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.628083 2549 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 15:13:02.629826 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.629811 2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-11.ec2.internal" event="NodeHasSufficientMemory" Jul 07 15:13:02.629882 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.629838 2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-11.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 15:13:02.629882 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.629848 2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-11.ec2.internal" event="NodeHasSufficientPID" Jul 07 15:13:02.631028 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.631016 2549 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 15:13:02.631129 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.631116 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-11.ec2.internal" Jul 07 15:13:02.631163 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.631143 2549 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 15:13:02.631708 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.631680 2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-11.ec2.internal" event="NodeHasSufficientMemory" Jul 07 15:13:02.631791 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.631712 2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-11.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 15:13:02.631791 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.631724 2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-11.ec2.internal" event="NodeHasSufficientPID" Jul 07 15:13:02.631791 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.631730 2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-11.ec2.internal" event="NodeHasSufficientMemory" Jul 07 15:13:02.631791 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.631746 2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-11.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 15:13:02.631791 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.631755 2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-11.ec2.internal" event="NodeHasSufficientPID" Jul 07 15:13:02.632730 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.632716 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-11.ec2.internal" Jul 07 15:13:02.632778 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.632740 2549 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 15:13:02.633280 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.633266 2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-11.ec2.internal" event="NodeHasSufficientMemory" Jul 07 15:13:02.633366 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.633293 2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-11.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 15:13:02.633366 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.633306 2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-11.ec2.internal" event="NodeHasSufficientPID" Jul 07 15:13:02.645661 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:02.645645 2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-11.ec2.internal\" not found" Jul 07 15:13:02.649869 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.649848 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b12720c0f68435524972635f743dfbf4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-11.ec2.internal\" (UID: \"b12720c0f68435524972635f743dfbf4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-11.ec2.internal" Jul 07 15:13:02.649957 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.649871 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f3ea74c0531243ed68ab1cca77160244-config\") pod \"kube-apiserver-proxy-ip-10-0-138-11.ec2.internal\" (UID: \"f3ea74c0531243ed68ab1cca77160244\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-11.ec2.internal" Jul 07 15:13:02.649957 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.649888 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b12720c0f68435524972635f743dfbf4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-11.ec2.internal\" (UID: \"b12720c0f68435524972635f743dfbf4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-11.ec2.internal" Jul 07 15:13:02.650072 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:02.650056 2549 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-11.ec2.internal\" not found" node="ip-10-0-138-11.ec2.internal" Jul 07 15:13:02.653778 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:02.653765 2549 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-11.ec2.internal\" not found" node="ip-10-0-138-11.ec2.internal" Jul 07 15:13:02.746536 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:02.746505 2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-11.ec2.internal\" not found" Jul 07 15:13:02.750992 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.750968 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b12720c0f68435524972635f743dfbf4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-11.ec2.internal\" (UID: \"b12720c0f68435524972635f743dfbf4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-11.ec2.internal" Jul 07 15:13:02.751134 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.751000 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b12720c0f68435524972635f743dfbf4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-11.ec2.internal\" (UID: \"b12720c0f68435524972635f743dfbf4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-11.ec2.internal" Jul 07 15:13:02.751134 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.751022 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f3ea74c0531243ed68ab1cca77160244-config\") pod \"kube-apiserver-proxy-ip-10-0-138-11.ec2.internal\" (UID: \"f3ea74c0531243ed68ab1cca77160244\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-11.ec2.internal" Jul 07 15:13:02.751134 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.751064 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f3ea74c0531243ed68ab1cca77160244-config\") pod \"kube-apiserver-proxy-ip-10-0-138-11.ec2.internal\" (UID: \"f3ea74c0531243ed68ab1cca77160244\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-11.ec2.internal" Jul 07 15:13:02.751134 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.751074 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b12720c0f68435524972635f743dfbf4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-11.ec2.internal\" (UID: \"b12720c0f68435524972635f743dfbf4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-11.ec2.internal" Jul 07 15:13:02.751134 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.751065 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b12720c0f68435524972635f743dfbf4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-11.ec2.internal\" (UID: \"b12720c0f68435524972635f743dfbf4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-11.ec2.internal" Jul 07 15:13:02.847379 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:02.847308 2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-11.ec2.internal\" not found" Jul 07 15:13:02.947718 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:02.947680 2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-11.ec2.internal\" not found" Jul 07 15:13:02.952928 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.952912 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-11.ec2.internal" Jul 07 15:13:02.956610 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:02.956490 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-11.ec2.internal" Jul 07 15:13:03.048027 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:03.047988 2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-11.ec2.internal\" not found" Jul 07 15:13:03.148437 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:03.148359 2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-11.ec2.internal\" not found" Jul 07 15:13:03.248721 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:03.248681 2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-11.ec2.internal\" not found" Jul 07 15:13:03.286977 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:03.286945 2549 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 07 15:13:03.287437 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:03.287079 2549 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 07 15:13:03.349486 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:03.349454 2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-11.ec2.internal\" not found" Jul 07 15:13:03.408205 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:03.408115 2549 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 15:13:03.409413 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:03.409391 2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-11.ec2.internal" event="NodeHasSufficientMemory" Jul 07 15:13:03.409413 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:03.409426 2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-11.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 15:13:03.409608 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:03.409440 2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-11.ec2.internal" event="NodeHasSufficientPID" Jul 07 15:13:03.450056 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:03.450027 2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-11.ec2.internal\" not found" Jul 07 15:13:03.467186 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:03.467144 2549 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 15:08:02 +0000 UTC" deadline="2028-02-04 00:40:14.927179279 +0000 UTC" Jul 07 15:13:03.467186 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:03.467187 2549 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13833h27m11.459995602s" Jul 07 15:13:03.502779 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:03.502755 2549 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 15:13:03.548336 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:03.548306 2549 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-11.ec2.internal" Jul 07 15:13:03.558163 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:03.558140 2549 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 15:13:03.558831 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:03.558790 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb12720c0f68435524972635f743dfbf4.slice/crio-7338f91c0b883cc9ecf51bd838ca3e87989f81b20d2524eb1097ed9d30aaa089 WatchSource:0}: Error finding container 7338f91c0b883cc9ecf51bd838ca3e87989f81b20d2524eb1097ed9d30aaa089: Status 404 returned error can't find the container with id 7338f91c0b883cc9ecf51bd838ca3e87989f81b20d2524eb1097ed9d30aaa089 Jul 07 15:13:03.559117 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:03.559098 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3ea74c0531243ed68ab1cca77160244.slice/crio-a34601e919d73207ecaae1facf02231e72bc4b30ab3e8154916cff659fdedad6 WatchSource:0}: Error finding container a34601e919d73207ecaae1facf02231e72bc4b30ab3e8154916cff659fdedad6: Status 404 returned error can't find the container with id a34601e919d73207ecaae1facf02231e72bc4b30ab3e8154916cff659fdedad6 Jul 07 15:13:03.559326 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:03.559312 2549 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-11.ec2.internal" Jul 07 15:13:03.562603 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:03.562513 2549 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 15:13:03.563002 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:03.562988 2549 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 15:13:03.566438 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:03.566421 2549 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 15:13:03.835574 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:03.835487 2549 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 15:13:04.330007 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.329931 2549 apiserver.go:52] "Watching apiserver" Jul 07 15:13:04.337210 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.337181 2549 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 07 15:13:04.337651 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.337628 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-xt54w","openshift-cluster-node-tuning-operator/tuned-2x8hc","openshift-dns/node-resolver-t8hcc","openshift-image-registry/node-ca-zd2vk","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-11.ec2.internal","openshift-multus/network-metrics-daemon-xqp6p","openshift-network-diagnostics/network-check-target-9dwk4","openshift-network-operator/iptables-alerter-sk2gc","kube-system/konnectivity-agent-dvrfw","kube-system/kube-apiserver-proxy-ip-10-0-138-11.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6dr6g","openshift-multus/multus-additional-cni-plugins-qkznn","openshift-multus/multus-mf9rg","openshift-ovn-kubernetes/ovnkube-node-9p7dn"] Jul 07 15:13:04.338985 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.338961 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9dwk4" Jul 07 15:13:04.339094 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:04.339041 2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9dwk4" podUID="95c962c7-1672-403f-9d88-acc1fc891e91" Jul 07 15:13:04.340025 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.340006 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6dr6g" Jul 07 15:13:04.341202 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.341183 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-t8hcc" Jul 07 15:13:04.342163 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.342070 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 07 15:13:04.342163 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.342084 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-zdc2v\"" Jul 07 15:13:04.342667 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.342554 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zd2vk" Jul 07 15:13:04.342782 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.342755 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 07 15:13:04.343051 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.343023 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 07 15:13:04.343724 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.343252 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-5dg7c\"" Jul 07 15:13:04.343724 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.343510 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 07 15:13:04.343724 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.343627 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 07 15:13:04.343724 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.343633 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.344975 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.344732 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 07 15:13:04.344975 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.344752 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 07 15:13:04.344975 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.344759 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 07 15:13:04.344975 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.344752 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-9j98h\"" Jul 07 15:13:04.344975 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.344875 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqp6p" Jul 07 15:13:04.344975 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:04.344930 2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xqp6p" podUID="274e9d34-35d9-452a-996f-1183641bf618" Jul 07 15:13:04.345692 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.345672 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 07 15:13:04.345772 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.345693 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 07 15:13:04.345772 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.345693 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 07 15:13:04.346204 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.346189 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dvrfw" Jul 07 15:13:04.346703 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.346686 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 07 15:13:04.346775 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.346741 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 07 15:13:04.346931 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.346915 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 07 15:13:04.347031 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.347015 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-7t64q\"" Jul 07 15:13:04.347667 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.347650 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mf9rg" Jul 07 15:13:04.348498 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.348474 2549 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 07 15:13:04.349221 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.349207 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2x8hc" Jul 07 15:13:04.350680 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.350630 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 07 15:13:04.351690 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.351659 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-sk2gc" Jul 07 15:13:04.352112 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.352093 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-hb9mw\"" Jul 07 15:13:04.353337 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.352868 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 07 15:13:04.353337 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.352939 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-pmjnd\"" Jul 07 15:13:04.353337 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.352966 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 07 15:13:04.353337 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.353101 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 07 15:13:04.353337 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.353295 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xt54w" Jul 07 15:13:04.353337 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.353306 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 07 15:13:04.354065 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.354045 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 07 15:13:04.354423 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.354406 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-n9n4h\"" Jul 07 15:13:04.354505 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:04.354468 2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xt54w" podUID="473572dc-5da2-44c1-918d-a409370c493f" Jul 07 15:13:04.355368 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.355259 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 07 15:13:04.355368 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.355279 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 07 15:13:04.356177 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.355816 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-pgtx2\"" Jul 07 15:13:04.356177 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.355928 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 07 15:13:04.356177 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.355824 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 07 15:13:04.356177 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.356053 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 07 15:13:04.356985 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.356963 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/104cc0be-5dba-411e-be9b-6742c6898f6c-var-lib-kubelet\") pod \"tuned-2x8hc\" (UID: \"104cc0be-5dba-411e-be9b-6742c6898f6c\") " pod="openshift-cluster-node-tuning-operator/tuned-2x8hc" Jul 07 15:13:04.357069 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.357002 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/104cc0be-5dba-411e-be9b-6742c6898f6c-host\") pod \"tuned-2x8hc\" (UID: \"104cc0be-5dba-411e-be9b-6742c6898f6c\") " pod="openshift-cluster-node-tuning-operator/tuned-2x8hc" Jul 07 15:13:04.357069 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.357059 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2549m\" (UniqueName: \"kubernetes.io/projected/104cc0be-5dba-411e-be9b-6742c6898f6c-kube-api-access-2549m\") pod \"tuned-2x8hc\" (UID: \"104cc0be-5dba-411e-be9b-6742c6898f6c\") " pod="openshift-cluster-node-tuning-operator/tuned-2x8hc" Jul 07 15:13:04.357159 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.357102 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/22eaf67c-4e87-4339-b266-e1f7330bf89c-ovnkube-config\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.357159 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.357137 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f9f4e27e-0a19-420b-977e-c2ac05efb164-multus-cni-dir\") pod \"multus-mf9rg\" (UID: \"f9f4e27e-0a19-420b-977e-c2ac05efb164\") " pod="openshift-multus/multus-mf9rg" Jul 07 15:13:04.357239 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.357169 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f9f4e27e-0a19-420b-977e-c2ac05efb164-host-run-netns\") pod \"multus-mf9rg\" (UID: \"f9f4e27e-0a19-420b-977e-c2ac05efb164\") " pod="openshift-multus/multus-mf9rg" Jul 07 15:13:04.357239 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.357200 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f9f4e27e-0a19-420b-977e-c2ac05efb164-host-var-lib-kubelet\") pod \"multus-mf9rg\" (UID: \"f9f4e27e-0a19-420b-977e-c2ac05efb164\") " pod="openshift-multus/multus-mf9rg" Jul 07 15:13:04.357239 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.357229 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/22eaf67c-4e87-4339-b266-e1f7330bf89c-systemd-units\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.357851 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.357281 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4p8jg\" (UniqueName: \"kubernetes.io/projected/80a72d31-7f06-45f3-8c76-9dd94fdb476a-kube-api-access-4p8jg\") pod \"node-ca-zd2vk\" (UID: \"80a72d31-7f06-45f3-8c76-9dd94fdb476a\") " pod="openshift-image-registry/node-ca-zd2vk" Jul 07 15:13:04.357925 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.357880 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da22234a-c3b5-450f-804c-6b6f6b238d3b-tmp-dir\") pod \"node-resolver-t8hcc\" (UID: \"da22234a-c3b5-450f-804c-6b6f6b238d3b\") " pod="openshift-dns/node-resolver-t8hcc" Jul 07 15:13:04.357985 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.357920 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/22eaf67c-4e87-4339-b266-e1f7330bf89c-ovn-node-metrics-cert\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.357985 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.357926 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qkznn" Jul 07 15:13:04.357985 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.357951 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnjjx\" (UniqueName: \"kubernetes.io/projected/f9f4e27e-0a19-420b-977e-c2ac05efb164-kube-api-access-vnjjx\") pod \"multus-mf9rg\" (UID: \"f9f4e27e-0a19-420b-977e-c2ac05efb164\") " pod="openshift-multus/multus-mf9rg" Jul 07 15:13:04.358280 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.357987 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/80a72d31-7f06-45f3-8c76-9dd94fdb476a-serviceca\") pod \"node-ca-zd2vk\" (UID: \"80a72d31-7f06-45f3-8c76-9dd94fdb476a\") " pod="openshift-image-registry/node-ca-zd2vk" Jul 07 15:13:04.358280 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.358019 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/22eaf67c-4e87-4339-b266-e1f7330bf89c-host-kubelet\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.358280 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.358047 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3791d99d-ae07-48b8-9311-051b2f83e54a-sys-fs\") pod \"aws-ebs-csi-driver-node-6dr6g\" (UID: \"3791d99d-ae07-48b8-9311-051b2f83e54a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6dr6g" Jul 07 15:13:04.358280 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.358078 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/104cc0be-5dba-411e-be9b-6742c6898f6c-etc-tuned\") pod \"tuned-2x8hc\" (UID: \"104cc0be-5dba-411e-be9b-6742c6898f6c\") " pod="openshift-cluster-node-tuning-operator/tuned-2x8hc" Jul 07 15:13:04.358280 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.358106 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4frnx\" (UniqueName: \"kubernetes.io/projected/3791d99d-ae07-48b8-9311-051b2f83e54a-kube-api-access-4frnx\") pod \"aws-ebs-csi-driver-node-6dr6g\" (UID: \"3791d99d-ae07-48b8-9311-051b2f83e54a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6dr6g" Jul 07 15:13:04.358280 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.358139 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/22eaf67c-4e87-4339-b266-e1f7330bf89c-run-openvswitch\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.358280 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.358162 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/22eaf67c-4e87-4339-b266-e1f7330bf89c-node-log\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.358280 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.358203 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/22eaf67c-4e87-4339-b266-e1f7330bf89c-log-socket\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.358280 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.358230 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f9f4e27e-0a19-420b-977e-c2ac05efb164-host-run-multus-certs\") pod \"multus-mf9rg\" (UID: \"f9f4e27e-0a19-420b-977e-c2ac05efb164\") " pod="openshift-multus/multus-mf9rg" Jul 07 15:13:04.358280 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.358267 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xkz2\" (UniqueName: \"kubernetes.io/projected/274e9d34-35d9-452a-996f-1183641bf618-kube-api-access-6xkz2\") pod \"network-metrics-daemon-xqp6p\" (UID: \"274e9d34-35d9-452a-996f-1183641bf618\") " pod="openshift-multus/network-metrics-daemon-xqp6p" Jul 07 15:13:04.358736 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.358295 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f9f4e27e-0a19-420b-977e-c2ac05efb164-os-release\") pod \"multus-mf9rg\" (UID: \"f9f4e27e-0a19-420b-977e-c2ac05efb164\") " pod="openshift-multus/multus-mf9rg" Jul 07 15:13:04.358736 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.358317 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f9f4e27e-0a19-420b-977e-c2ac05efb164-cni-binary-copy\") pod \"multus-mf9rg\" (UID: \"f9f4e27e-0a19-420b-977e-c2ac05efb164\") " pod="openshift-multus/multus-mf9rg" Jul 07 15:13:04.358736 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.358345 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f9f4e27e-0a19-420b-977e-c2ac05efb164-multus-socket-dir-parent\") pod \"multus-mf9rg\" (UID: \"f9f4e27e-0a19-420b-977e-c2ac05efb164\") " pod="openshift-multus/multus-mf9rg" Jul 07 15:13:04.358736 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.358374 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/104cc0be-5dba-411e-be9b-6742c6898f6c-etc-sysctl-conf\") pod \"tuned-2x8hc\" (UID: \"104cc0be-5dba-411e-be9b-6742c6898f6c\") " pod="openshift-cluster-node-tuning-operator/tuned-2x8hc" Jul 07 15:13:04.358736 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.358442 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/22eaf67c-4e87-4339-b266-e1f7330bf89c-host-run-ovn-kubernetes\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.358736 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.358470 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/da22234a-c3b5-450f-804c-6b6f6b238d3b-hosts-file\") pod \"node-resolver-t8hcc\" (UID: \"da22234a-c3b5-450f-804c-6b6f6b238d3b\") " pod="openshift-dns/node-resolver-t8hcc" Jul 07 15:13:04.358736 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.358500 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f9f4e27e-0a19-420b-977e-c2ac05efb164-host-var-lib-cni-multus\") pod \"multus-mf9rg\" (UID: \"f9f4e27e-0a19-420b-977e-c2ac05efb164\") " pod="openshift-multus/multus-mf9rg" Jul 07 15:13:04.359053 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.358825 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3791d99d-ae07-48b8-9311-051b2f83e54a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6dr6g\" (UID: \"3791d99d-ae07-48b8-9311-051b2f83e54a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6dr6g" Jul 07 15:13:04.359053 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.358973 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/104cc0be-5dba-411e-be9b-6742c6898f6c-sys\") pod \"tuned-2x8hc\" (UID: \"104cc0be-5dba-411e-be9b-6742c6898f6c\") " pod="openshift-cluster-node-tuning-operator/tuned-2x8hc" Jul 07 15:13:04.359514 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.359346 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/22eaf67c-4e87-4339-b266-e1f7330bf89c-host-cni-netd\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.359514 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.359398 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/274e9d34-35d9-452a-996f-1183641bf618-metrics-certs\") pod \"network-metrics-daemon-xqp6p\" (UID: \"274e9d34-35d9-452a-996f-1183641bf618\") " pod="openshift-multus/network-metrics-daemon-xqp6p" Jul 07 15:13:04.359514 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.359419 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f9f4e27e-0a19-420b-977e-c2ac05efb164-system-cni-dir\") pod \"multus-mf9rg\" (UID: \"f9f4e27e-0a19-420b-977e-c2ac05efb164\") " pod="openshift-multus/multus-mf9rg" Jul 07 15:13:04.359514 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.359439 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f9f4e27e-0a19-420b-977e-c2ac05efb164-host-var-lib-cni-bin\") pod \"multus-mf9rg\" (UID: \"f9f4e27e-0a19-420b-977e-c2ac05efb164\") " pod="openshift-multus/multus-mf9rg" Jul 07 15:13:04.359514 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.359493 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/22eaf67c-4e87-4339-b266-e1f7330bf89c-run-systemd\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.359819 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.359515 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f9f4e27e-0a19-420b-977e-c2ac05efb164-etc-kubernetes\") pod \"multus-mf9rg\" (UID: \"f9f4e27e-0a19-420b-977e-c2ac05efb164\") " pod="openshift-multus/multus-mf9rg" Jul 07 15:13:04.359819 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.359539 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3791d99d-ae07-48b8-9311-051b2f83e54a-registration-dir\") pod \"aws-ebs-csi-driver-node-6dr6g\" (UID: \"3791d99d-ae07-48b8-9311-051b2f83e54a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6dr6g" Jul 07 15:13:04.359819 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.359638 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3791d99d-ae07-48b8-9311-051b2f83e54a-device-dir\") pod \"aws-ebs-csi-driver-node-6dr6g\" (UID: \"3791d99d-ae07-48b8-9311-051b2f83e54a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6dr6g" Jul 07 15:13:04.359819 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.359661 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/31fc8ff5-b255-4523-9758-bf21b0409efa-agent-certs\") pod \"konnectivity-agent-dvrfw\" (UID: \"31fc8ff5-b255-4523-9758-bf21b0409efa\") " pod="kube-system/konnectivity-agent-dvrfw" Jul 07 15:13:04.359819 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.359684 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/31fc8ff5-b255-4523-9758-bf21b0409efa-konnectivity-ca\") pod \"konnectivity-agent-dvrfw\" (UID: \"31fc8ff5-b255-4523-9758-bf21b0409efa\") " pod="kube-system/konnectivity-agent-dvrfw" Jul 07 15:13:04.359819 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.359708 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/104cc0be-5dba-411e-be9b-6742c6898f6c-etc-modprobe-d\") pod \"tuned-2x8hc\" (UID: \"104cc0be-5dba-411e-be9b-6742c6898f6c\") " pod="openshift-cluster-node-tuning-operator/tuned-2x8hc" Jul 07 15:13:04.359819 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.359730 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/104cc0be-5dba-411e-be9b-6742c6898f6c-lib-modules\") pod \"tuned-2x8hc\" (UID: \"104cc0be-5dba-411e-be9b-6742c6898f6c\") " pod="openshift-cluster-node-tuning-operator/tuned-2x8hc" Jul 07 15:13:04.359819 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.359754 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/22eaf67c-4e87-4339-b266-e1f7330bf89c-host-slash\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.359819 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.359777 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/22eaf67c-4e87-4339-b266-e1f7330bf89c-var-lib-openvswitch\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.359819 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.359800 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/22eaf67c-4e87-4339-b266-e1f7330bf89c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.359819 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.359822 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f9f4e27e-0a19-420b-977e-c2ac05efb164-multus-daemon-config\") pod \"multus-mf9rg\" (UID: \"f9f4e27e-0a19-420b-977e-c2ac05efb164\") " pod="openshift-multus/multus-mf9rg" Jul 07 15:13:04.360280 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.359844 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3791d99d-ae07-48b8-9311-051b2f83e54a-socket-dir\") pod \"aws-ebs-csi-driver-node-6dr6g\" (UID: \"3791d99d-ae07-48b8-9311-051b2f83e54a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6dr6g" Jul 07 15:13:04.360280 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.359867 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ctlv\" (UniqueName: \"kubernetes.io/projected/da22234a-c3b5-450f-804c-6b6f6b238d3b-kube-api-access-9ctlv\") pod \"node-resolver-t8hcc\" (UID: \"da22234a-c3b5-450f-804c-6b6f6b238d3b\") " pod="openshift-dns/node-resolver-t8hcc" Jul 07 15:13:04.360280 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.359907 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/22eaf67c-4e87-4339-b266-e1f7330bf89c-host-cni-bin\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.360280 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.359928 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f9f4e27e-0a19-420b-977e-c2ac05efb164-host-run-k8s-cni-cncf-io\") pod \"multus-mf9rg\" (UID: \"f9f4e27e-0a19-420b-977e-c2ac05efb164\") " pod="openshift-multus/multus-mf9rg" Jul 07 15:13:04.360280 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.359951 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3791d99d-ae07-48b8-9311-051b2f83e54a-etc-selinux\") pod \"aws-ebs-csi-driver-node-6dr6g\" (UID: \"3791d99d-ae07-48b8-9311-051b2f83e54a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6dr6g" Jul 07 15:13:04.360280 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.359974 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/104cc0be-5dba-411e-be9b-6742c6898f6c-etc-systemd\") pod \"tuned-2x8hc\" (UID: \"104cc0be-5dba-411e-be9b-6742c6898f6c\") " pod="openshift-cluster-node-tuning-operator/tuned-2x8hc" Jul 07 15:13:04.360280 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.359995 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/104cc0be-5dba-411e-be9b-6742c6898f6c-run\") pod \"tuned-2x8hc\" (UID: \"104cc0be-5dba-411e-be9b-6742c6898f6c\") " pod="openshift-cluster-node-tuning-operator/tuned-2x8hc" Jul 07 15:13:04.360280 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.360036 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/22eaf67c-4e87-4339-b266-e1f7330bf89c-host-run-netns\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.360280 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.360060 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/22eaf67c-4e87-4339-b266-e1f7330bf89c-env-overrides\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.360280 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.360084 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/22eaf67c-4e87-4339-b266-e1f7330bf89c-ovnkube-script-lib\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.360280 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.360105 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f9f4e27e-0a19-420b-977e-c2ac05efb164-hostroot\") pod \"multus-mf9rg\" (UID: \"f9f4e27e-0a19-420b-977e-c2ac05efb164\") " pod="openshift-multus/multus-mf9rg" Jul 07 15:13:04.360280 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.360126 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f9f4e27e-0a19-420b-977e-c2ac05efb164-multus-conf-dir\") pod \"multus-mf9rg\" (UID: \"f9f4e27e-0a19-420b-977e-c2ac05efb164\") " pod="openshift-multus/multus-mf9rg" Jul 07 15:13:04.360280 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.360175 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgrk9\" (UniqueName: \"kubernetes.io/projected/95c962c7-1672-403f-9d88-acc1fc891e91-kube-api-access-rgrk9\") pod \"network-check-target-9dwk4\" (UID: \"95c962c7-1672-403f-9d88-acc1fc891e91\") " pod="openshift-network-diagnostics/network-check-target-9dwk4" Jul 07 15:13:04.360280 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.360198 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nk9jk\" (UniqueName: \"kubernetes.io/projected/22eaf67c-4e87-4339-b266-e1f7330bf89c-kube-api-access-nk9jk\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.360280 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.360219 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f9f4e27e-0a19-420b-977e-c2ac05efb164-cnibin\") pod \"multus-mf9rg\" (UID: \"f9f4e27e-0a19-420b-977e-c2ac05efb164\") " pod="openshift-multus/multus-mf9rg" Jul 07 15:13:04.360280 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.360283 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/104cc0be-5dba-411e-be9b-6742c6898f6c-etc-kubernetes\") pod \"tuned-2x8hc\" (UID: \"104cc0be-5dba-411e-be9b-6742c6898f6c\") " pod="openshift-cluster-node-tuning-operator/tuned-2x8hc" Jul 07 15:13:04.361018 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.360315 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/104cc0be-5dba-411e-be9b-6742c6898f6c-etc-sysctl-d\") pod \"tuned-2x8hc\" (UID: \"104cc0be-5dba-411e-be9b-6742c6898f6c\") " pod="openshift-cluster-node-tuning-operator/tuned-2x8hc" Jul 07 15:13:04.361018 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.360325 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 07 15:13:04.361018 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.360516 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/104cc0be-5dba-411e-be9b-6742c6898f6c-tmp\") pod \"tuned-2x8hc\" (UID: \"104cc0be-5dba-411e-be9b-6742c6898f6c\") " pod="openshift-cluster-node-tuning-operator/tuned-2x8hc" Jul 07 15:13:04.361018 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.360535 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 07 15:13:04.361018 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.360542 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/22eaf67c-4e87-4339-b266-e1f7330bf89c-etc-openvswitch\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.361018 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.360562 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/22eaf67c-4e87-4339-b266-e1f7330bf89c-run-ovn\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.361018 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.360619 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/80a72d31-7f06-45f3-8c76-9dd94fdb476a-host\") pod \"node-ca-zd2vk\" (UID: \"80a72d31-7f06-45f3-8c76-9dd94fdb476a\") " pod="openshift-image-registry/node-ca-zd2vk" Jul 07 15:13:04.361018 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.360638 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/104cc0be-5dba-411e-be9b-6742c6898f6c-etc-sysconfig\") pod \"tuned-2x8hc\" (UID: \"104cc0be-5dba-411e-be9b-6742c6898f6c\") " pod="openshift-cluster-node-tuning-operator/tuned-2x8hc" Jul 07 15:13:04.361018 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.360780 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-xsh26\"" Jul 07 15:13:04.366121 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.366098 2549 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 15:13:04.386392 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.386367 2549 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-7qvxg" Jul 07 15:13:04.392693 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.392671 2549 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-7qvxg" Jul 07 15:13:04.449415 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.449381 2549 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 07 15:13:04.461701 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.461670 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/31fc8ff5-b255-4523-9758-bf21b0409efa-agent-certs\") pod \"konnectivity-agent-dvrfw\" (UID: \"31fc8ff5-b255-4523-9758-bf21b0409efa\") " pod="kube-system/konnectivity-agent-dvrfw" Jul 07 15:13:04.461864 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.461711 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/31fc8ff5-b255-4523-9758-bf21b0409efa-konnectivity-ca\") pod \"konnectivity-agent-dvrfw\" (UID: \"31fc8ff5-b255-4523-9758-bf21b0409efa\") " pod="kube-system/konnectivity-agent-dvrfw" Jul 07 15:13:04.461864 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.461739 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/104cc0be-5dba-411e-be9b-6742c6898f6c-etc-modprobe-d\") pod \"tuned-2x8hc\" (UID: \"104cc0be-5dba-411e-be9b-6742c6898f6c\") " pod="openshift-cluster-node-tuning-operator/tuned-2x8hc" Jul 07 15:13:04.461864 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.461764 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/104cc0be-5dba-411e-be9b-6742c6898f6c-lib-modules\") pod \"tuned-2x8hc\" (UID: \"104cc0be-5dba-411e-be9b-6742c6898f6c\") " pod="openshift-cluster-node-tuning-operator/tuned-2x8hc" Jul 07 15:13:04.461864 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.461789 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/22eaf67c-4e87-4339-b266-e1f7330bf89c-host-slash\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.461864 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.461813 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/22eaf67c-4e87-4339-b266-e1f7330bf89c-var-lib-openvswitch\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.461864 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.461837 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/22eaf67c-4e87-4339-b266-e1f7330bf89c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.461864 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.461864 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f9f4e27e-0a19-420b-977e-c2ac05efb164-multus-daemon-config\") pod \"multus-mf9rg\" (UID: \"f9f4e27e-0a19-420b-977e-c2ac05efb164\") " pod="openshift-multus/multus-mf9rg" Jul 07 15:13:04.462217 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.461889 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3791d99d-ae07-48b8-9311-051b2f83e54a-socket-dir\") pod \"aws-ebs-csi-driver-node-6dr6g\" (UID: \"3791d99d-ae07-48b8-9311-051b2f83e54a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6dr6g" Jul 07 15:13:04.462217 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.461915 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ctlv\" (UniqueName: \"kubernetes.io/projected/da22234a-c3b5-450f-804c-6b6f6b238d3b-kube-api-access-9ctlv\") pod \"node-resolver-t8hcc\" (UID: \"da22234a-c3b5-450f-804c-6b6f6b238d3b\") " pod="openshift-dns/node-resolver-t8hcc" Jul 07 15:13:04.462217 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.461940 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/22eaf67c-4e87-4339-b266-e1f7330bf89c-host-cni-bin\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.462217 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.461987 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f9f4e27e-0a19-420b-977e-c2ac05efb164-host-run-k8s-cni-cncf-io\") pod \"multus-mf9rg\" (UID: \"f9f4e27e-0a19-420b-977e-c2ac05efb164\") " pod="openshift-multus/multus-mf9rg" Jul 07 15:13:04.462217 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.462060 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3791d99d-ae07-48b8-9311-051b2f83e54a-etc-selinux\") pod \"aws-ebs-csi-driver-node-6dr6g\" (UID: \"3791d99d-ae07-48b8-9311-051b2f83e54a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6dr6g" Jul 07 15:13:04.462217 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.462084 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/104cc0be-5dba-411e-be9b-6742c6898f6c-etc-systemd\") pod \"tuned-2x8hc\" (UID: \"104cc0be-5dba-411e-be9b-6742c6898f6c\") " pod="openshift-cluster-node-tuning-operator/tuned-2x8hc" Jul 07 15:13:04.462217 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.462123 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/104cc0be-5dba-411e-be9b-6742c6898f6c-run\") pod \"tuned-2x8hc\" (UID: \"104cc0be-5dba-411e-be9b-6742c6898f6c\") " pod="openshift-cluster-node-tuning-operator/tuned-2x8hc" Jul 07 15:13:04.462217 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.462129 2549 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 07 15:13:04.462598 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.462293 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/104cc0be-5dba-411e-be9b-6742c6898f6c-etc-modprobe-d\") pod \"tuned-2x8hc\" (UID: \"104cc0be-5dba-411e-be9b-6742c6898f6c\") " pod="openshift-cluster-node-tuning-operator/tuned-2x8hc" Jul 07 15:13:04.462598 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.462330 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/31fc8ff5-b255-4523-9758-bf21b0409efa-konnectivity-ca\") pod \"konnectivity-agent-dvrfw\" (UID: \"31fc8ff5-b255-4523-9758-bf21b0409efa\") " pod="kube-system/konnectivity-agent-dvrfw" Jul 07 15:13:04.462598 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.462152 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a5099dd3-70a7-4a61-a0da-4aaad1c064f4-host-slash\") pod \"iptables-alerter-sk2gc\" (UID: \"a5099dd3-70a7-4a61-a0da-4aaad1c064f4\") " pod="openshift-network-operator/iptables-alerter-sk2gc" Jul 07 15:13:04.462598 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.462422 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/22eaf67c-4e87-4339-b266-e1f7330bf89c-host-slash\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.462598 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.462431 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/22eaf67c-4e87-4339-b266-e1f7330bf89c-host-run-netns\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.462598 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.462476 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/22eaf67c-4e87-4339-b266-e1f7330bf89c-var-lib-openvswitch\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.462598 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.462482 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f9f4e27e-0a19-420b-977e-c2ac05efb164-host-run-k8s-cni-cncf-io\") pod \"multus-mf9rg\" (UID: \"f9f4e27e-0a19-420b-977e-c2ac05efb164\") " pod="openshift-multus/multus-mf9rg" Jul 07 15:13:04.462598 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.462386 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/104cc0be-5dba-411e-be9b-6742c6898f6c-lib-modules\") pod \"tuned-2x8hc\" (UID: \"104cc0be-5dba-411e-be9b-6742c6898f6c\") " pod="openshift-cluster-node-tuning-operator/tuned-2x8hc" Jul 07 15:13:04.462598 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.462477 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/22eaf67c-4e87-4339-b266-e1f7330bf89c-env-overrides\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.462598 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.462522 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/22eaf67c-4e87-4339-b266-e1f7330bf89c-ovnkube-script-lib\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.462598 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.462526 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/104cc0be-5dba-411e-be9b-6742c6898f6c-etc-systemd\") pod \"tuned-2x8hc\" (UID: \"104cc0be-5dba-411e-be9b-6742c6898f6c\") " pod="openshift-cluster-node-tuning-operator/tuned-2x8hc" Jul 07 15:13:04.462598 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.462526 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/22eaf67c-4e87-4339-b266-e1f7330bf89c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.462598 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.462548 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f9f4e27e-0a19-420b-977e-c2ac05efb164-hostroot\") pod \"multus-mf9rg\" (UID: \"f9f4e27e-0a19-420b-977e-c2ac05efb164\") " pod="openshift-multus/multus-mf9rg" Jul 07 15:13:04.463171 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.462596 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f9f4e27e-0a19-420b-977e-c2ac05efb164-multus-conf-dir\") pod \"multus-mf9rg\" (UID: \"f9f4e27e-0a19-420b-977e-c2ac05efb164\") " pod="openshift-multus/multus-mf9rg" Jul 07 15:13:04.463171 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.462650 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3791d99d-ae07-48b8-9311-051b2f83e54a-etc-selinux\") pod \"aws-ebs-csi-driver-node-6dr6g\" (UID: \"3791d99d-ae07-48b8-9311-051b2f83e54a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6dr6g" Jul 07 15:13:04.463171 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.462637 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c59d2067-5399-4a3d-9f34-be99e63f7d2b-cnibin\") pod \"multus-additional-cni-plugins-qkznn\" (UID: \"c59d2067-5399-4a3d-9f34-be99e63f7d2b\") " pod="openshift-multus/multus-additional-cni-plugins-qkznn" Jul 07 15:13:04.463171 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.462701 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgrk9\" (UniqueName: \"kubernetes.io/projected/95c962c7-1672-403f-9d88-acc1fc891e91-kube-api-access-rgrk9\") pod \"network-check-target-9dwk4\" (UID: \"95c962c7-1672-403f-9d88-acc1fc891e91\") " pod="openshift-network-diagnostics/network-check-target-9dwk4" Jul 07 15:13:04.463171 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.462725 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nk9jk\" (UniqueName: \"kubernetes.io/projected/22eaf67c-4e87-4339-b266-e1f7330bf89c-kube-api-access-nk9jk\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.463171 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.462764 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f9f4e27e-0a19-420b-977e-c2ac05efb164-cnibin\") pod \"multus-mf9rg\" (UID: \"f9f4e27e-0a19-420b-977e-c2ac05efb164\") " pod="openshift-multus/multus-mf9rg" Jul 07 15:13:04.463171 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.462788 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/104cc0be-5dba-411e-be9b-6742c6898f6c-etc-kubernetes\") pod \"tuned-2x8hc\" (UID: \"104cc0be-5dba-411e-be9b-6742c6898f6c\") " pod="openshift-cluster-node-tuning-operator/tuned-2x8hc" Jul 07 15:13:04.463171 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.462837 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/104cc0be-5dba-411e-be9b-6742c6898f6c-etc-sysctl-d\") pod \"tuned-2x8hc\" (UID: \"104cc0be-5dba-411e-be9b-6742c6898f6c\") " pod="openshift-cluster-node-tuning-operator/tuned-2x8hc" Jul 07 15:13:04.463171 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.462885 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/22eaf67c-4e87-4339-b266-e1f7330bf89c-env-overrides\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.463171 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.462935 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/22eaf67c-4e87-4339-b266-e1f7330bf89c-host-run-netns\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.463171 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.462979 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/22eaf67c-4e87-4339-b266-e1f7330bf89c-host-cni-bin\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.463171 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.463074 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f9f4e27e-0a19-420b-977e-c2ac05efb164-hostroot\") pod \"multus-mf9rg\" (UID: \"f9f4e27e-0a19-420b-977e-c2ac05efb164\") " pod="openshift-multus/multus-mf9rg" Jul 07 15:13:04.463171 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.463091 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f9f4e27e-0a19-420b-977e-c2ac05efb164-multus-daemon-config\") pod \"multus-mf9rg\" (UID: \"f9f4e27e-0a19-420b-977e-c2ac05efb164\") " pod="openshift-multus/multus-mf9rg" Jul 07 15:13:04.463171 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.463114 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f9f4e27e-0a19-420b-977e-c2ac05efb164-multus-conf-dir\") pod \"multus-mf9rg\" (UID: \"f9f4e27e-0a19-420b-977e-c2ac05efb164\") " pod="openshift-multus/multus-mf9rg" Jul 07 15:13:04.463171 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.463163 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/104cc0be-5dba-411e-be9b-6742c6898f6c-run\") pod \"tuned-2x8hc\" (UID: \"104cc0be-5dba-411e-be9b-6742c6898f6c\") " pod="openshift-cluster-node-tuning-operator/tuned-2x8hc" Jul 07 15:13:04.463171 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.462889 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/104cc0be-5dba-411e-be9b-6742c6898f6c-tmp\") pod \"tuned-2x8hc\" (UID: \"104cc0be-5dba-411e-be9b-6742c6898f6c\") " pod="openshift-cluster-node-tuning-operator/tuned-2x8hc" Jul 07 15:13:04.463896 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.463203 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjsz7\" (UniqueName: \"kubernetes.io/projected/a5099dd3-70a7-4a61-a0da-4aaad1c064f4-kube-api-access-bjsz7\") pod \"iptables-alerter-sk2gc\" (UID: \"a5099dd3-70a7-4a61-a0da-4aaad1c064f4\") " pod="openshift-network-operator/iptables-alerter-sk2gc" Jul 07 15:13:04.463896 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.463234 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/22eaf67c-4e87-4339-b266-e1f7330bf89c-etc-openvswitch\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.463896 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.463213 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3791d99d-ae07-48b8-9311-051b2f83e54a-socket-dir\") pod \"aws-ebs-csi-driver-node-6dr6g\" (UID: \"3791d99d-ae07-48b8-9311-051b2f83e54a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6dr6g" Jul 07 15:13:04.463896 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.463274 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/22eaf67c-4e87-4339-b266-e1f7330bf89c-run-ovn\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.463896 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.463314 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/80a72d31-7f06-45f3-8c76-9dd94fdb476a-host\") pod \"node-ca-zd2vk\" (UID: \"80a72d31-7f06-45f3-8c76-9dd94fdb476a\") " pod="openshift-image-registry/node-ca-zd2vk" Jul 07 15:13:04.463896 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.463350 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/104cc0be-5dba-411e-be9b-6742c6898f6c-etc-sysconfig\") pod \"tuned-2x8hc\" (UID: \"104cc0be-5dba-411e-be9b-6742c6898f6c\") " pod="openshift-cluster-node-tuning-operator/tuned-2x8hc" Jul 07 15:13:04.463896 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.463383 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/104cc0be-5dba-411e-be9b-6742c6898f6c-var-lib-kubelet\") pod \"tuned-2x8hc\" (UID: \"104cc0be-5dba-411e-be9b-6742c6898f6c\") " pod="openshift-cluster-node-tuning-operator/tuned-2x8hc" Jul 07 15:13:04.463896 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.463411 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/104cc0be-5dba-411e-be9b-6742c6898f6c-host\") pod \"tuned-2x8hc\" (UID: \"104cc0be-5dba-411e-be9b-6742c6898f6c\") " pod="openshift-cluster-node-tuning-operator/tuned-2x8hc" Jul 07 15:13:04.463896 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.463441 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2549m\" (UniqueName: \"kubernetes.io/projected/104cc0be-5dba-411e-be9b-6742c6898f6c-kube-api-access-2549m\") pod \"tuned-2x8hc\" (UID: \"104cc0be-5dba-411e-be9b-6742c6898f6c\") " pod="openshift-cluster-node-tuning-operator/tuned-2x8hc" Jul 07 15:13:04.463896 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.463482 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/22eaf67c-4e87-4339-b266-e1f7330bf89c-ovnkube-config\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.463896 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.463499 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f9f4e27e-0a19-420b-977e-c2ac05efb164-cnibin\") pod \"multus-mf9rg\" (UID: \"f9f4e27e-0a19-420b-977e-c2ac05efb164\") " pod="openshift-multus/multus-mf9rg" Jul 07 15:13:04.463896 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.463509 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f9f4e27e-0a19-420b-977e-c2ac05efb164-multus-cni-dir\") pod \"multus-mf9rg\" (UID: \"f9f4e27e-0a19-420b-977e-c2ac05efb164\") " pod="openshift-multus/multus-mf9rg" Jul 07 15:13:04.463896 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.463537 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/22eaf67c-4e87-4339-b266-e1f7330bf89c-run-ovn\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.463896 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.463551 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/104cc0be-5dba-411e-be9b-6742c6898f6c-etc-sysconfig\") pod \"tuned-2x8hc\" (UID: \"104cc0be-5dba-411e-be9b-6742c6898f6c\") " pod="openshift-cluster-node-tuning-operator/tuned-2x8hc" Jul 07 15:13:04.463896 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.463611 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/22eaf67c-4e87-4339-b266-e1f7330bf89c-etc-openvswitch\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.463896 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.463613 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/104cc0be-5dba-411e-be9b-6742c6898f6c-var-lib-kubelet\") pod \"tuned-2x8hc\" (UID: \"104cc0be-5dba-411e-be9b-6742c6898f6c\") " pod="openshift-cluster-node-tuning-operator/tuned-2x8hc" Jul 07 15:13:04.463896 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.463538 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f9f4e27e-0a19-420b-977e-c2ac05efb164-host-run-netns\") pod \"multus-mf9rg\" (UID: \"f9f4e27e-0a19-420b-977e-c2ac05efb164\") " pod="openshift-multus/multus-mf9rg" Jul 07 15:13:04.463896 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.463661 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f9f4e27e-0a19-420b-977e-c2ac05efb164-host-var-lib-kubelet\") pod \"multus-mf9rg\" (UID: \"f9f4e27e-0a19-420b-977e-c2ac05efb164\") " pod="openshift-multus/multus-mf9rg" Jul 07 15:13:04.464705 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.463666 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/104cc0be-5dba-411e-be9b-6742c6898f6c-etc-kubernetes\") pod \"tuned-2x8hc\" (UID: \"104cc0be-5dba-411e-be9b-6742c6898f6c\") " pod="openshift-cluster-node-tuning-operator/tuned-2x8hc" Jul 07 15:13:04.464705 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.463663 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/80a72d31-7f06-45f3-8c76-9dd94fdb476a-host\") pod \"node-ca-zd2vk\" (UID: \"80a72d31-7f06-45f3-8c76-9dd94fdb476a\") " pod="openshift-image-registry/node-ca-zd2vk" Jul 07 15:13:04.464705 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.463697 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/104cc0be-5dba-411e-be9b-6742c6898f6c-host\") pod \"tuned-2x8hc\" (UID: \"104cc0be-5dba-411e-be9b-6742c6898f6c\") " pod="openshift-cluster-node-tuning-operator/tuned-2x8hc" Jul 07 15:13:04.464705 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.463701 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/473572dc-5da2-44c1-918d-a409370c493f-original-pull-secret\") pod \"global-pull-secret-syncer-xt54w\" (UID: \"473572dc-5da2-44c1-918d-a409370c493f\") " pod="kube-system/global-pull-secret-syncer-xt54w" Jul 07 15:13:04.464705 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.463715 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f9f4e27e-0a19-420b-977e-c2ac05efb164-host-var-lib-kubelet\") pod \"multus-mf9rg\" (UID: \"f9f4e27e-0a19-420b-977e-c2ac05efb164\") " pod="openshift-multus/multus-mf9rg" Jul 07 15:13:04.464705 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.463790 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/104cc0be-5dba-411e-be9b-6742c6898f6c-etc-sysctl-d\") pod \"tuned-2x8hc\" (UID: \"104cc0be-5dba-411e-be9b-6742c6898f6c\") " pod="openshift-cluster-node-tuning-operator/tuned-2x8hc" Jul 07 15:13:04.464705 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.463815 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f9f4e27e-0a19-420b-977e-c2ac05efb164-multus-cni-dir\") pod \"multus-mf9rg\" (UID: \"f9f4e27e-0a19-420b-977e-c2ac05efb164\") " pod="openshift-multus/multus-mf9rg" Jul 07 15:13:04.464705 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.463606 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f9f4e27e-0a19-420b-977e-c2ac05efb164-host-run-netns\") pod \"multus-mf9rg\" (UID: \"f9f4e27e-0a19-420b-977e-c2ac05efb164\") " pod="openshift-multus/multus-mf9rg" Jul 07 15:13:04.464705 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.463916 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c59d2067-5399-4a3d-9f34-be99e63f7d2b-os-release\") pod \"multus-additional-cni-plugins-qkznn\" (UID: \"c59d2067-5399-4a3d-9f34-be99e63f7d2b\") " pod="openshift-multus/multus-additional-cni-plugins-qkznn" Jul 07 15:13:04.464705 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.463943 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fz8pn\" (UniqueName: \"kubernetes.io/projected/c59d2067-5399-4a3d-9f34-be99e63f7d2b-kube-api-access-fz8pn\") pod \"multus-additional-cni-plugins-qkznn\" (UID: \"c59d2067-5399-4a3d-9f34-be99e63f7d2b\") " pod="openshift-multus/multus-additional-cni-plugins-qkznn" Jul 07 15:13:04.464705 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.463966 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a5099dd3-70a7-4a61-a0da-4aaad1c064f4-iptables-alerter-script\") pod \"iptables-alerter-sk2gc\" (UID: \"a5099dd3-70a7-4a61-a0da-4aaad1c064f4\") " pod="openshift-network-operator/iptables-alerter-sk2gc" Jul 07 15:13:04.464705 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.463990 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/22eaf67c-4e87-4339-b266-e1f7330bf89c-systemd-units\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.464705 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.464012 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4p8jg\" (UniqueName: \"kubernetes.io/projected/80a72d31-7f06-45f3-8c76-9dd94fdb476a-kube-api-access-4p8jg\") pod \"node-ca-zd2vk\" (UID: \"80a72d31-7f06-45f3-8c76-9dd94fdb476a\") " pod="openshift-image-registry/node-ca-zd2vk" Jul 07 15:13:04.464705 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.464032 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/473572dc-5da2-44c1-918d-a409370c493f-kubelet-config\") pod \"global-pull-secret-syncer-xt54w\" (UID: \"473572dc-5da2-44c1-918d-a409370c493f\") " pod="kube-system/global-pull-secret-syncer-xt54w" Jul 07 15:13:04.464705 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.464054 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c59d2067-5399-4a3d-9f34-be99e63f7d2b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qkznn\" (UID: \"c59d2067-5399-4a3d-9f34-be99e63f7d2b\") " pod="openshift-multus/multus-additional-cni-plugins-qkznn" Jul 07 15:13:04.464705 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.464080 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da22234a-c3b5-450f-804c-6b6f6b238d3b-tmp-dir\") pod \"node-resolver-t8hcc\" (UID: \"da22234a-c3b5-450f-804c-6b6f6b238d3b\") " pod="openshift-dns/node-resolver-t8hcc" Jul 07 15:13:04.464705 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.464106 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/22eaf67c-4e87-4339-b266-e1f7330bf89c-ovn-node-metrics-cert\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.465514 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.464144 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnjjx\" (UniqueName: \"kubernetes.io/projected/f9f4e27e-0a19-420b-977e-c2ac05efb164-kube-api-access-vnjjx\") pod \"multus-mf9rg\" (UID: \"f9f4e27e-0a19-420b-977e-c2ac05efb164\") " pod="openshift-multus/multus-mf9rg" Jul 07 15:13:04.465514 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.464174 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/80a72d31-7f06-45f3-8c76-9dd94fdb476a-serviceca\") pod \"node-ca-zd2vk\" (UID: \"80a72d31-7f06-45f3-8c76-9dd94fdb476a\") " pod="openshift-image-registry/node-ca-zd2vk" Jul 07 15:13:04.465514 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.464196 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c59d2067-5399-4a3d-9f34-be99e63f7d2b-cni-binary-copy\") pod \"multus-additional-cni-plugins-qkznn\" (UID: \"c59d2067-5399-4a3d-9f34-be99e63f7d2b\") " pod="openshift-multus/multus-additional-cni-plugins-qkznn" Jul 07 15:13:04.465514 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.464226 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/22eaf67c-4e87-4339-b266-e1f7330bf89c-host-kubelet\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.465514 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.464246 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3791d99d-ae07-48b8-9311-051b2f83e54a-sys-fs\") pod \"aws-ebs-csi-driver-node-6dr6g\" (UID: \"3791d99d-ae07-48b8-9311-051b2f83e54a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6dr6g" Jul 07 15:13:04.465514 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.464268 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/104cc0be-5dba-411e-be9b-6742c6898f6c-etc-tuned\") pod \"tuned-2x8hc\" (UID: \"104cc0be-5dba-411e-be9b-6742c6898f6c\") " pod="openshift-cluster-node-tuning-operator/tuned-2x8hc" Jul 07 15:13:04.465514 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.464293 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4frnx\" (UniqueName: \"kubernetes.io/projected/3791d99d-ae07-48b8-9311-051b2f83e54a-kube-api-access-4frnx\") pod \"aws-ebs-csi-driver-node-6dr6g\" (UID: \"3791d99d-ae07-48b8-9311-051b2f83e54a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6dr6g" Jul 07 15:13:04.465514 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.464318 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/22eaf67c-4e87-4339-b266-e1f7330bf89c-run-openvswitch\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.465514 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.464354 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/22eaf67c-4e87-4339-b266-e1f7330bf89c-node-log\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.465514 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.464358 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/22eaf67c-4e87-4339-b266-e1f7330bf89c-ovnkube-config\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.465514 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.464379 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/22eaf67c-4e87-4339-b266-e1f7330bf89c-log-socket\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.465514 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.464360 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/22eaf67c-4e87-4339-b266-e1f7330bf89c-ovnkube-script-lib\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.465514 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.464416 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f9f4e27e-0a19-420b-977e-c2ac05efb164-host-run-multus-certs\") pod \"multus-mf9rg\" (UID: \"f9f4e27e-0a19-420b-977e-c2ac05efb164\") " pod="openshift-multus/multus-mf9rg" Jul 07 15:13:04.465514 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.464450 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/22eaf67c-4e87-4339-b266-e1f7330bf89c-systemd-units\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.465514 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.464448 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c59d2067-5399-4a3d-9f34-be99e63f7d2b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qkznn\" (UID: \"c59d2067-5399-4a3d-9f34-be99e63f7d2b\") " pod="openshift-multus/multus-additional-cni-plugins-qkznn" Jul 07 15:13:04.465514 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.464497 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xkz2\" (UniqueName: \"kubernetes.io/projected/274e9d34-35d9-452a-996f-1183641bf618-kube-api-access-6xkz2\") pod \"network-metrics-daemon-xqp6p\" (UID: \"274e9d34-35d9-452a-996f-1183641bf618\") " pod="openshift-multus/network-metrics-daemon-xqp6p" Jul 07 15:13:04.465514 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.464504 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/22eaf67c-4e87-4339-b266-e1f7330bf89c-log-socket\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.466374 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.464537 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f9f4e27e-0a19-420b-977e-c2ac05efb164-os-release\") pod \"multus-mf9rg\" (UID: \"f9f4e27e-0a19-420b-977e-c2ac05efb164\") " pod="openshift-multus/multus-mf9rg" Jul 07 15:13:04.466374 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.464599 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f9f4e27e-0a19-420b-977e-c2ac05efb164-cni-binary-copy\") pod \"multus-mf9rg\" (UID: \"f9f4e27e-0a19-420b-977e-c2ac05efb164\") " pod="openshift-multus/multus-mf9rg" Jul 07 15:13:04.466374 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.464641 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f9f4e27e-0a19-420b-977e-c2ac05efb164-multus-socket-dir-parent\") pod \"multus-mf9rg\" (UID: \"f9f4e27e-0a19-420b-977e-c2ac05efb164\") " pod="openshift-multus/multus-mf9rg" Jul 07 15:13:04.466374 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.464677 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/104cc0be-5dba-411e-be9b-6742c6898f6c-etc-sysctl-conf\") pod \"tuned-2x8hc\" (UID: \"104cc0be-5dba-411e-be9b-6742c6898f6c\") " pod="openshift-cluster-node-tuning-operator/tuned-2x8hc" Jul 07 15:13:04.466374 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.464760 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/22eaf67c-4e87-4339-b266-e1f7330bf89c-host-kubelet\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.466374 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.464781 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3791d99d-ae07-48b8-9311-051b2f83e54a-sys-fs\") pod \"aws-ebs-csi-driver-node-6dr6g\" (UID: \"3791d99d-ae07-48b8-9311-051b2f83e54a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6dr6g" Jul 07 15:13:04.466374 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.464798 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/22eaf67c-4e87-4339-b266-e1f7330bf89c-run-openvswitch\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.466374 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.464826 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/22eaf67c-4e87-4339-b266-e1f7330bf89c-node-log\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.466374 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.465085 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da22234a-c3b5-450f-804c-6b6f6b238d3b-tmp-dir\") pod \"node-resolver-t8hcc\" (UID: \"da22234a-c3b5-450f-804c-6b6f6b238d3b\") " pod="openshift-dns/node-resolver-t8hcc" Jul 07 15:13:04.466374 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.465137 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f9f4e27e-0a19-420b-977e-c2ac05efb164-cni-binary-copy\") pod \"multus-mf9rg\" (UID: \"f9f4e27e-0a19-420b-977e-c2ac05efb164\") " pod="openshift-multus/multus-mf9rg" Jul 07 15:13:04.466374 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.465156 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f9f4e27e-0a19-420b-977e-c2ac05efb164-host-run-multus-certs\") pod \"multus-mf9rg\" (UID: \"f9f4e27e-0a19-420b-977e-c2ac05efb164\") " pod="openshift-multus/multus-mf9rg" Jul 07 15:13:04.466374 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.465193 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/22eaf67c-4e87-4339-b266-e1f7330bf89c-host-run-ovn-kubernetes\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.466374 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.465233 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f9f4e27e-0a19-420b-977e-c2ac05efb164-multus-socket-dir-parent\") pod \"multus-mf9rg\" (UID: \"f9f4e27e-0a19-420b-977e-c2ac05efb164\") " pod="openshift-multus/multus-mf9rg" Jul 07 15:13:04.466374 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.465292 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c59d2067-5399-4a3d-9f34-be99e63f7d2b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qkznn\" (UID: \"c59d2067-5399-4a3d-9f34-be99e63f7d2b\") " pod="openshift-multus/multus-additional-cni-plugins-qkznn" Jul 07 15:13:04.466374 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.465329 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/da22234a-c3b5-450f-804c-6b6f6b238d3b-hosts-file\") pod \"node-resolver-t8hcc\" (UID: \"da22234a-c3b5-450f-804c-6b6f6b238d3b\") " pod="openshift-dns/node-resolver-t8hcc" Jul 07 15:13:04.466374 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.465371 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f9f4e27e-0a19-420b-977e-c2ac05efb164-host-var-lib-cni-multus\") pod \"multus-mf9rg\" (UID: \"f9f4e27e-0a19-420b-977e-c2ac05efb164\") " pod="openshift-multus/multus-mf9rg" Jul 07 15:13:04.466374 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.465398 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3791d99d-ae07-48b8-9311-051b2f83e54a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6dr6g\" (UID: \"3791d99d-ae07-48b8-9311-051b2f83e54a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6dr6g" Jul 07 15:13:04.466374 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.465421 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/104cc0be-5dba-411e-be9b-6742c6898f6c-sys\") pod \"tuned-2x8hc\" (UID: \"104cc0be-5dba-411e-be9b-6742c6898f6c\") " pod="openshift-cluster-node-tuning-operator/tuned-2x8hc" Jul 07 15:13:04.467239 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.465449 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c59d2067-5399-4a3d-9f34-be99e63f7d2b-system-cni-dir\") pod \"multus-additional-cni-plugins-qkznn\" (UID: \"c59d2067-5399-4a3d-9f34-be99e63f7d2b\") " pod="openshift-multus/multus-additional-cni-plugins-qkznn" Jul 07 15:13:04.467239 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.465547 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/da22234a-c3b5-450f-804c-6b6f6b238d3b-hosts-file\") pod \"node-resolver-t8hcc\" (UID: \"da22234a-c3b5-450f-804c-6b6f6b238d3b\") " pod="openshift-dns/node-resolver-t8hcc" Jul 07 15:13:04.467239 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.465709 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f9f4e27e-0a19-420b-977e-c2ac05efb164-os-release\") pod \"multus-mf9rg\" (UID: \"f9f4e27e-0a19-420b-977e-c2ac05efb164\") " pod="openshift-multus/multus-mf9rg" Jul 07 15:13:04.467239 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.465751 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/22eaf67c-4e87-4339-b266-e1f7330bf89c-host-run-ovn-kubernetes\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.467239 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.465774 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3791d99d-ae07-48b8-9311-051b2f83e54a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6dr6g\" (UID: \"3791d99d-ae07-48b8-9311-051b2f83e54a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6dr6g" Jul 07 15:13:04.467239 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.465822 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/104cc0be-5dba-411e-be9b-6742c6898f6c-sys\") pod \"tuned-2x8hc\" (UID: \"104cc0be-5dba-411e-be9b-6742c6898f6c\") " pod="openshift-cluster-node-tuning-operator/tuned-2x8hc" Jul 07 15:13:04.467239 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.465847 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/104cc0be-5dba-411e-be9b-6742c6898f6c-etc-sysctl-conf\") pod \"tuned-2x8hc\" (UID: \"104cc0be-5dba-411e-be9b-6742c6898f6c\") " pod="openshift-cluster-node-tuning-operator/tuned-2x8hc" Jul 07 15:13:04.467239 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.465869 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/22eaf67c-4e87-4339-b266-e1f7330bf89c-host-cni-netd\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.467239 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.465847 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/31fc8ff5-b255-4523-9758-bf21b0409efa-agent-certs\") pod \"konnectivity-agent-dvrfw\" (UID: \"31fc8ff5-b255-4523-9758-bf21b0409efa\") " pod="kube-system/konnectivity-agent-dvrfw" Jul 07 15:13:04.467239 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.465896 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f9f4e27e-0a19-420b-977e-c2ac05efb164-host-var-lib-cni-multus\") pod \"multus-mf9rg\" (UID: \"f9f4e27e-0a19-420b-977e-c2ac05efb164\") " pod="openshift-multus/multus-mf9rg" Jul 07 15:13:04.467239 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.465942 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/473572dc-5da2-44c1-918d-a409370c493f-dbus\") pod \"global-pull-secret-syncer-xt54w\" (UID: \"473572dc-5da2-44c1-918d-a409370c493f\") " pod="kube-system/global-pull-secret-syncer-xt54w" Jul 07 15:13:04.467239 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.465965 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/22eaf67c-4e87-4339-b266-e1f7330bf89c-host-cni-netd\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.467239 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.465970 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/80a72d31-7f06-45f3-8c76-9dd94fdb476a-serviceca\") pod \"node-ca-zd2vk\" (UID: \"80a72d31-7f06-45f3-8c76-9dd94fdb476a\") " pod="openshift-image-registry/node-ca-zd2vk" Jul 07 15:13:04.467239 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.466045 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/274e9d34-35d9-452a-996f-1183641bf618-metrics-certs\") pod \"network-metrics-daemon-xqp6p\" (UID: \"274e9d34-35d9-452a-996f-1183641bf618\") " pod="openshift-multus/network-metrics-daemon-xqp6p" Jul 07 15:13:04.467239 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.466133 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f9f4e27e-0a19-420b-977e-c2ac05efb164-system-cni-dir\") pod \"multus-mf9rg\" (UID: \"f9f4e27e-0a19-420b-977e-c2ac05efb164\") " pod="openshift-multus/multus-mf9rg" Jul 07 15:13:04.467239 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:04.466236 2549 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:13:04.467239 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.466262 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f9f4e27e-0a19-420b-977e-c2ac05efb164-host-var-lib-cni-bin\") pod \"multus-mf9rg\" (UID: \"f9f4e27e-0a19-420b-977e-c2ac05efb164\") " pod="openshift-multus/multus-mf9rg" Jul 07 15:13:04.467907 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:04.466312 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/274e9d34-35d9-452a-996f-1183641bf618-metrics-certs podName:274e9d34-35d9-452a-996f-1183641bf618 nodeName:}" failed. No retries permitted until 2026-07-07 15:13:04.966292808 +0000 UTC m=+2.973770252 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/274e9d34-35d9-452a-996f-1183641bf618-metrics-certs") pod "network-metrics-daemon-xqp6p" (UID: "274e9d34-35d9-452a-996f-1183641bf618") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:13:04.467907 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.466331 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f9f4e27e-0a19-420b-977e-c2ac05efb164-host-var-lib-cni-bin\") pod \"multus-mf9rg\" (UID: \"f9f4e27e-0a19-420b-977e-c2ac05efb164\") " pod="openshift-multus/multus-mf9rg" Jul 07 15:13:04.467907 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.466349 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/22eaf67c-4e87-4339-b266-e1f7330bf89c-run-systemd\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.467907 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.466383 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f9f4e27e-0a19-420b-977e-c2ac05efb164-system-cni-dir\") pod \"multus-mf9rg\" (UID: \"f9f4e27e-0a19-420b-977e-c2ac05efb164\") " pod="openshift-multus/multus-mf9rg" Jul 07 15:13:04.467907 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.466424 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f9f4e27e-0a19-420b-977e-c2ac05efb164-etc-kubernetes\") pod \"multus-mf9rg\" (UID: \"f9f4e27e-0a19-420b-977e-c2ac05efb164\") " pod="openshift-multus/multus-mf9rg" Jul 07 15:13:04.467907 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.466448 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3791d99d-ae07-48b8-9311-051b2f83e54a-registration-dir\") pod \"aws-ebs-csi-driver-node-6dr6g\" (UID: \"3791d99d-ae07-48b8-9311-051b2f83e54a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6dr6g" Jul 07 15:13:04.467907 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.466467 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/104cc0be-5dba-411e-be9b-6742c6898f6c-tmp\") pod \"tuned-2x8hc\" (UID: \"104cc0be-5dba-411e-be9b-6742c6898f6c\") " pod="openshift-cluster-node-tuning-operator/tuned-2x8hc" Jul 07 15:13:04.467907 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.466495 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3791d99d-ae07-48b8-9311-051b2f83e54a-device-dir\") pod \"aws-ebs-csi-driver-node-6dr6g\" (UID: \"3791d99d-ae07-48b8-9311-051b2f83e54a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6dr6g" Jul 07 15:13:04.467907 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.466557 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/22eaf67c-4e87-4339-b266-e1f7330bf89c-run-systemd\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.467907 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.466598 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3791d99d-ae07-48b8-9311-051b2f83e54a-registration-dir\") pod \"aws-ebs-csi-driver-node-6dr6g\" (UID: \"3791d99d-ae07-48b8-9311-051b2f83e54a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6dr6g" Jul 07 15:13:04.467907 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.466621 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f9f4e27e-0a19-420b-977e-c2ac05efb164-etc-kubernetes\") pod \"multus-mf9rg\" (UID: \"f9f4e27e-0a19-420b-977e-c2ac05efb164\") " pod="openshift-multus/multus-mf9rg" Jul 07 15:13:04.467907 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.466658 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3791d99d-ae07-48b8-9311-051b2f83e54a-device-dir\") pod \"aws-ebs-csi-driver-node-6dr6g\" (UID: \"3791d99d-ae07-48b8-9311-051b2f83e54a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6dr6g" Jul 07 15:13:04.468463 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.468351 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/22eaf67c-4e87-4339-b266-e1f7330bf89c-ovn-node-metrics-cert\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.468553 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.468528 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/104cc0be-5dba-411e-be9b-6742c6898f6c-etc-tuned\") pod \"tuned-2x8hc\" (UID: \"104cc0be-5dba-411e-be9b-6742c6898f6c\") " pod="openshift-cluster-node-tuning-operator/tuned-2x8hc" Jul 07 15:13:04.470350 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:04.470331 2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 15:13:04.470350 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:04.470351 2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 15:13:04.470543 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:04.470362 2549 projected.go:196] Error preparing data for projected volume kube-api-access-rgrk9 for pod openshift-network-diagnostics/network-check-target-9dwk4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:13:04.470543 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:04.470424 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/95c962c7-1672-403f-9d88-acc1fc891e91-kube-api-access-rgrk9 podName:95c962c7-1672-403f-9d88-acc1fc891e91 nodeName:}" failed. No retries permitted until 2026-07-07 15:13:04.970412425 +0000 UTC m=+2.977889854 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-rgrk9" (UniqueName: "kubernetes.io/projected/95c962c7-1672-403f-9d88-acc1fc891e91-kube-api-access-rgrk9") pod "network-check-target-9dwk4" (UID: "95c962c7-1672-403f-9d88-acc1fc891e91") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:13:04.472688 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.472628 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ctlv\" (UniqueName: \"kubernetes.io/projected/da22234a-c3b5-450f-804c-6b6f6b238d3b-kube-api-access-9ctlv\") pod \"node-resolver-t8hcc\" (UID: \"da22234a-c3b5-450f-804c-6b6f6b238d3b\") " pod="openshift-dns/node-resolver-t8hcc" Jul 07 15:13:04.473050 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.473028 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4p8jg\" (UniqueName: \"kubernetes.io/projected/80a72d31-7f06-45f3-8c76-9dd94fdb476a-kube-api-access-4p8jg\") pod \"node-ca-zd2vk\" (UID: \"80a72d31-7f06-45f3-8c76-9dd94fdb476a\") " pod="openshift-image-registry/node-ca-zd2vk" Jul 07 15:13:04.473387 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.473369 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nk9jk\" (UniqueName: \"kubernetes.io/projected/22eaf67c-4e87-4339-b266-e1f7330bf89c-kube-api-access-nk9jk\") pod \"ovnkube-node-9p7dn\" (UID: \"22eaf67c-4e87-4339-b266-e1f7330bf89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.473553 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.473540 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2549m\" (UniqueName: \"kubernetes.io/projected/104cc0be-5dba-411e-be9b-6742c6898f6c-kube-api-access-2549m\") pod \"tuned-2x8hc\" (UID: \"104cc0be-5dba-411e-be9b-6742c6898f6c\") " pod="openshift-cluster-node-tuning-operator/tuned-2x8hc" Jul 07 15:13:04.482934 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.482909 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xkz2\" (UniqueName: \"kubernetes.io/projected/274e9d34-35d9-452a-996f-1183641bf618-kube-api-access-6xkz2\") pod \"network-metrics-daemon-xqp6p\" (UID: \"274e9d34-35d9-452a-996f-1183641bf618\") " pod="openshift-multus/network-metrics-daemon-xqp6p" Jul 07 15:13:04.483115 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.483098 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnjjx\" (UniqueName: \"kubernetes.io/projected/f9f4e27e-0a19-420b-977e-c2ac05efb164-kube-api-access-vnjjx\") pod \"multus-mf9rg\" (UID: \"f9f4e27e-0a19-420b-977e-c2ac05efb164\") " pod="openshift-multus/multus-mf9rg" Jul 07 15:13:04.484609 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.484543 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4frnx\" (UniqueName: \"kubernetes.io/projected/3791d99d-ae07-48b8-9311-051b2f83e54a-kube-api-access-4frnx\") pod \"aws-ebs-csi-driver-node-6dr6g\" (UID: \"3791d99d-ae07-48b8-9311-051b2f83e54a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6dr6g" Jul 07 15:13:04.532785 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.532737 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-11.ec2.internal" event={"ID":"f3ea74c0531243ed68ab1cca77160244","Type":"ContainerStarted","Data":"a34601e919d73207ecaae1facf02231e72bc4b30ab3e8154916cff659fdedad6"} Jul 07 15:13:04.533862 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.533831 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-11.ec2.internal" event={"ID":"b12720c0f68435524972635f743dfbf4","Type":"ContainerStarted","Data":"7338f91c0b883cc9ecf51bd838ca3e87989f81b20d2524eb1097ed9d30aaa089"} Jul 07 15:13:04.566911 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.566867 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/473572dc-5da2-44c1-918d-a409370c493f-dbus\") pod \"global-pull-secret-syncer-xt54w\" (UID: \"473572dc-5da2-44c1-918d-a409370c493f\") " pod="kube-system/global-pull-secret-syncer-xt54w" Jul 07 15:13:04.567138 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.566959 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a5099dd3-70a7-4a61-a0da-4aaad1c064f4-host-slash\") pod \"iptables-alerter-sk2gc\" (UID: \"a5099dd3-70a7-4a61-a0da-4aaad1c064f4\") " pod="openshift-network-operator/iptables-alerter-sk2gc" Jul 07 15:13:04.567138 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.566983 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c59d2067-5399-4a3d-9f34-be99e63f7d2b-cnibin\") pod \"multus-additional-cni-plugins-qkznn\" (UID: \"c59d2067-5399-4a3d-9f34-be99e63f7d2b\") " pod="openshift-multus/multus-additional-cni-plugins-qkznn" Jul 07 15:13:04.567138 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.567018 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjsz7\" (UniqueName: \"kubernetes.io/projected/a5099dd3-70a7-4a61-a0da-4aaad1c064f4-kube-api-access-bjsz7\") pod \"iptables-alerter-sk2gc\" (UID: \"a5099dd3-70a7-4a61-a0da-4aaad1c064f4\") " pod="openshift-network-operator/iptables-alerter-sk2gc" Jul 07 15:13:04.567138 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.567053 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/473572dc-5da2-44c1-918d-a409370c493f-original-pull-secret\") pod \"global-pull-secret-syncer-xt54w\" (UID: \"473572dc-5da2-44c1-918d-a409370c493f\") " pod="kube-system/global-pull-secret-syncer-xt54w" Jul 07 15:13:04.567138 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.567078 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c59d2067-5399-4a3d-9f34-be99e63f7d2b-os-release\") pod \"multus-additional-cni-plugins-qkznn\" (UID: \"c59d2067-5399-4a3d-9f34-be99e63f7d2b\") " pod="openshift-multus/multus-additional-cni-plugins-qkznn" Jul 07 15:13:04.567138 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.567089 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c59d2067-5399-4a3d-9f34-be99e63f7d2b-cnibin\") pod \"multus-additional-cni-plugins-qkznn\" (UID: \"c59d2067-5399-4a3d-9f34-be99e63f7d2b\") " pod="openshift-multus/multus-additional-cni-plugins-qkznn" Jul 07 15:13:04.567138 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.567102 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fz8pn\" (UniqueName: \"kubernetes.io/projected/c59d2067-5399-4a3d-9f34-be99e63f7d2b-kube-api-access-fz8pn\") pod \"multus-additional-cni-plugins-qkznn\" (UID: \"c59d2067-5399-4a3d-9f34-be99e63f7d2b\") " pod="openshift-multus/multus-additional-cni-plugins-qkznn" Jul 07 15:13:04.567138 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.567090 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/473572dc-5da2-44c1-918d-a409370c493f-dbus\") pod \"global-pull-secret-syncer-xt54w\" (UID: \"473572dc-5da2-44c1-918d-a409370c493f\") " pod="kube-system/global-pull-secret-syncer-xt54w" Jul 07 15:13:04.567138 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.567065 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a5099dd3-70a7-4a61-a0da-4aaad1c064f4-host-slash\") pod \"iptables-alerter-sk2gc\" (UID: \"a5099dd3-70a7-4a61-a0da-4aaad1c064f4\") " pod="openshift-network-operator/iptables-alerter-sk2gc" Jul 07 15:13:04.567138 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.567131 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a5099dd3-70a7-4a61-a0da-4aaad1c064f4-iptables-alerter-script\") pod \"iptables-alerter-sk2gc\" (UID: \"a5099dd3-70a7-4a61-a0da-4aaad1c064f4\") " pod="openshift-network-operator/iptables-alerter-sk2gc" Jul 07 15:13:04.567620 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.567158 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c59d2067-5399-4a3d-9f34-be99e63f7d2b-os-release\") pod \"multus-additional-cni-plugins-qkznn\" (UID: \"c59d2067-5399-4a3d-9f34-be99e63f7d2b\") " pod="openshift-multus/multus-additional-cni-plugins-qkznn" Jul 07 15:13:04.567620 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:04.567173 2549 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 15:13:04.567620 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.567193 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/473572dc-5da2-44c1-918d-a409370c493f-kubelet-config\") pod \"global-pull-secret-syncer-xt54w\" (UID: \"473572dc-5da2-44c1-918d-a409370c493f\") " pod="kube-system/global-pull-secret-syncer-xt54w" Jul 07 15:13:04.567620 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.567230 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c59d2067-5399-4a3d-9f34-be99e63f7d2b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qkznn\" (UID: \"c59d2067-5399-4a3d-9f34-be99e63f7d2b\") " pod="openshift-multus/multus-additional-cni-plugins-qkznn" Jul 07 15:13:04.567620 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.567271 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c59d2067-5399-4a3d-9f34-be99e63f7d2b-cni-binary-copy\") pod \"multus-additional-cni-plugins-qkznn\" (UID: \"c59d2067-5399-4a3d-9f34-be99e63f7d2b\") " pod="openshift-multus/multus-additional-cni-plugins-qkznn" Jul 07 15:13:04.567620 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.567313 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c59d2067-5399-4a3d-9f34-be99e63f7d2b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qkznn\" (UID: \"c59d2067-5399-4a3d-9f34-be99e63f7d2b\") " pod="openshift-multus/multus-additional-cni-plugins-qkznn" Jul 07 15:13:04.567620 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.567313 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/473572dc-5da2-44c1-918d-a409370c493f-kubelet-config\") pod \"global-pull-secret-syncer-xt54w\" (UID: \"473572dc-5da2-44c1-918d-a409370c493f\") " pod="kube-system/global-pull-secret-syncer-xt54w" Jul 07 15:13:04.567620 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:04.567341 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/473572dc-5da2-44c1-918d-a409370c493f-original-pull-secret podName:473572dc-5da2-44c1-918d-a409370c493f nodeName:}" failed. No retries permitted until 2026-07-07 15:13:05.06731934 +0000 UTC m=+3.074796772 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/473572dc-5da2-44c1-918d-a409370c493f-original-pull-secret") pod "global-pull-secret-syncer-xt54w" (UID: "473572dc-5da2-44c1-918d-a409370c493f") : object "kube-system"/"original-pull-secret" not registered Jul 07 15:13:04.567620 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.567440 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c59d2067-5399-4a3d-9f34-be99e63f7d2b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qkznn\" (UID: \"c59d2067-5399-4a3d-9f34-be99e63f7d2b\") " pod="openshift-multus/multus-additional-cni-plugins-qkznn" Jul 07 15:13:04.567620 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.567478 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c59d2067-5399-4a3d-9f34-be99e63f7d2b-system-cni-dir\") pod \"multus-additional-cni-plugins-qkznn\" (UID: \"c59d2067-5399-4a3d-9f34-be99e63f7d2b\") " pod="openshift-multus/multus-additional-cni-plugins-qkznn" Jul 07 15:13:04.567620 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.567540 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c59d2067-5399-4a3d-9f34-be99e63f7d2b-system-cni-dir\") pod \"multus-additional-cni-plugins-qkznn\" (UID: \"c59d2067-5399-4a3d-9f34-be99e63f7d2b\") " pod="openshift-multus/multus-additional-cni-plugins-qkznn" Jul 07 15:13:04.568173 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.567680 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c59d2067-5399-4a3d-9f34-be99e63f7d2b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qkznn\" (UID: \"c59d2067-5399-4a3d-9f34-be99e63f7d2b\") " pod="openshift-multus/multus-additional-cni-plugins-qkznn" Jul 07 15:13:04.568173 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.567811 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a5099dd3-70a7-4a61-a0da-4aaad1c064f4-iptables-alerter-script\") pod \"iptables-alerter-sk2gc\" (UID: \"a5099dd3-70a7-4a61-a0da-4aaad1c064f4\") " pod="openshift-network-operator/iptables-alerter-sk2gc" Jul 07 15:13:04.568173 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.567988 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c59d2067-5399-4a3d-9f34-be99e63f7d2b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qkznn\" (UID: \"c59d2067-5399-4a3d-9f34-be99e63f7d2b\") " pod="openshift-multus/multus-additional-cni-plugins-qkznn" Jul 07 15:13:04.568173 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.567997 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c59d2067-5399-4a3d-9f34-be99e63f7d2b-cni-binary-copy\") pod \"multus-additional-cni-plugins-qkznn\" (UID: \"c59d2067-5399-4a3d-9f34-be99e63f7d2b\") " pod="openshift-multus/multus-additional-cni-plugins-qkznn" Jul 07 15:13:04.568173 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.568068 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c59d2067-5399-4a3d-9f34-be99e63f7d2b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qkznn\" (UID: \"c59d2067-5399-4a3d-9f34-be99e63f7d2b\") " pod="openshift-multus/multus-additional-cni-plugins-qkznn" Jul 07 15:13:04.578639 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.578597 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjsz7\" (UniqueName: \"kubernetes.io/projected/a5099dd3-70a7-4a61-a0da-4aaad1c064f4-kube-api-access-bjsz7\") pod \"iptables-alerter-sk2gc\" (UID: \"a5099dd3-70a7-4a61-a0da-4aaad1c064f4\") " pod="openshift-network-operator/iptables-alerter-sk2gc" Jul 07 15:13:04.578833 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.578756 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fz8pn\" (UniqueName: \"kubernetes.io/projected/c59d2067-5399-4a3d-9f34-be99e63f7d2b-kube-api-access-fz8pn\") pod \"multus-additional-cni-plugins-qkznn\" (UID: \"c59d2067-5399-4a3d-9f34-be99e63f7d2b\") " pod="openshift-multus/multus-additional-cni-plugins-qkznn" Jul 07 15:13:04.653829 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.653735 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6dr6g" Jul 07 15:13:04.662485 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.662460 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-t8hcc" Jul 07 15:13:04.673229 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.673209 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zd2vk" Jul 07 15:13:04.677729 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.677707 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:04.684333 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.684316 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dvrfw" Jul 07 15:13:04.692890 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.692872 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mf9rg" Jul 07 15:13:04.699895 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.699877 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2x8hc" Jul 07 15:13:04.706355 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.706336 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-sk2gc" Jul 07 15:13:04.712870 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.712851 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qkznn" Jul 07 15:13:04.755404 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.755367 2549 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 15:13:04.969133 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:04.969052 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/274e9d34-35d9-452a-996f-1183641bf618-metrics-certs\") pod \"network-metrics-daemon-xqp6p\" (UID: \"274e9d34-35d9-452a-996f-1183641bf618\") " pod="openshift-multus/network-metrics-daemon-xqp6p" Jul 07 15:13:04.969300 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:04.969199 2549 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:13:04.969300 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:04.969267 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/274e9d34-35d9-452a-996f-1183641bf618-metrics-certs podName:274e9d34-35d9-452a-996f-1183641bf618 nodeName:}" failed. No retries permitted until 2026-07-07 15:13:05.969250957 +0000 UTC m=+3.976728386 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/274e9d34-35d9-452a-996f-1183641bf618-metrics-certs") pod "network-metrics-daemon-xqp6p" (UID: "274e9d34-35d9-452a-996f-1183641bf618") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:13:05.070544 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:05.070509 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgrk9\" (UniqueName: \"kubernetes.io/projected/95c962c7-1672-403f-9d88-acc1fc891e91-kube-api-access-rgrk9\") pod \"network-check-target-9dwk4\" (UID: \"95c962c7-1672-403f-9d88-acc1fc891e91\") " pod="openshift-network-diagnostics/network-check-target-9dwk4" Jul 07 15:13:05.070712 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:05.070560 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/473572dc-5da2-44c1-918d-a409370c493f-original-pull-secret\") pod \"global-pull-secret-syncer-xt54w\" (UID: \"473572dc-5da2-44c1-918d-a409370c493f\") " pod="kube-system/global-pull-secret-syncer-xt54w" Jul 07 15:13:05.070712 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:05.070684 2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 15:13:05.070712 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:05.070712 2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 15:13:05.070858 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:05.070690 2549 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 15:13:05.070858 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:05.070726 2549 projected.go:196] Error preparing data for projected volume kube-api-access-rgrk9 for pod openshift-network-diagnostics/network-check-target-9dwk4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:13:05.070858 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:05.070788 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/473572dc-5da2-44c1-918d-a409370c493f-original-pull-secret podName:473572dc-5da2-44c1-918d-a409370c493f nodeName:}" failed. No retries permitted until 2026-07-07 15:13:06.070769868 +0000 UTC m=+4.078247300 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/473572dc-5da2-44c1-918d-a409370c493f-original-pull-secret") pod "global-pull-secret-syncer-xt54w" (UID: "473572dc-5da2-44c1-918d-a409370c493f") : object "kube-system"/"original-pull-secret" not registered Jul 07 15:13:05.070858 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:05.070807 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/95c962c7-1672-403f-9d88-acc1fc891e91-kube-api-access-rgrk9 podName:95c962c7-1672-403f-9d88-acc1fc891e91 nodeName:}" failed. No retries permitted until 2026-07-07 15:13:06.070799407 +0000 UTC m=+4.078276851 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-rgrk9" (UniqueName: "kubernetes.io/projected/95c962c7-1672-403f-9d88-acc1fc891e91-kube-api-access-rgrk9") pod "network-check-target-9dwk4" (UID: "95c962c7-1672-403f-9d88-acc1fc891e91") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:13:05.395621 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:05.395510 2549 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 15:08:04 +0000 UTC" deadline="2028-01-10 17:49:51.763864936 +0000 UTC" Jul 07 15:13:05.395621 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:05.395549 2549 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13250h36m46.368319991s" Jul 07 15:13:05.437427 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:05.437398 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9f4e27e_0a19_420b_977e_c2ac05efb164.slice/crio-cfedf2eecb0866ace9e263c4dae42eeb7c0c9ed9750bfdbd2a4dcecbfadb1c5c WatchSource:0}: Error finding container cfedf2eecb0866ace9e263c4dae42eeb7c0c9ed9750bfdbd2a4dcecbfadb1c5c: Status 404 returned error can't find the container with id cfedf2eecb0866ace9e263c4dae42eeb7c0c9ed9750bfdbd2a4dcecbfadb1c5c Jul 07 15:13:05.455346 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:05.455306 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod104cc0be_5dba_411e_be9b_6742c6898f6c.slice/crio-1b0563bf2512a43a3ee214f7c8ac4bf1899e52f63c5beefacbb826ee88128784 WatchSource:0}: Error finding container 1b0563bf2512a43a3ee214f7c8ac4bf1899e52f63c5beefacbb826ee88128784: Status 404 returned error can't find the container with id 1b0563bf2512a43a3ee214f7c8ac4bf1899e52f63c5beefacbb826ee88128784 Jul 07 15:13:05.456097 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:05.456074 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5099dd3_70a7_4a61_a0da_4aaad1c064f4.slice/crio-95c0bf3dc918e8a3413b6414269f9915da851ebe95b357bf74ec23d07933c965 WatchSource:0}: Error finding container 95c0bf3dc918e8a3413b6414269f9915da851ebe95b357bf74ec23d07933c965: Status 404 returned error can't find the container with id 95c0bf3dc918e8a3413b6414269f9915da851ebe95b357bf74ec23d07933c965 Jul 07 15:13:05.456957 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:05.456932 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22eaf67c_4e87_4339_b266_e1f7330bf89c.slice/crio-241db0dbd4b861e365a6478a407f6e16456d93e59d92d6d32a9a7e3c00a01d98 WatchSource:0}: Error finding container 241db0dbd4b861e365a6478a407f6e16456d93e59d92d6d32a9a7e3c00a01d98: Status 404 returned error can't find the container with id 241db0dbd4b861e365a6478a407f6e16456d93e59d92d6d32a9a7e3c00a01d98 Jul 07 15:13:05.458262 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:05.458143 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3791d99d_ae07_48b8_9311_051b2f83e54a.slice/crio-284f2597fb50970b5562b9d2e7153c34ff4cc7b1a377f55fa3636e0d1c693ccf WatchSource:0}: Error finding container 284f2597fb50970b5562b9d2e7153c34ff4cc7b1a377f55fa3636e0d1c693ccf: Status 404 returned error can't find the container with id 284f2597fb50970b5562b9d2e7153c34ff4cc7b1a377f55fa3636e0d1c693ccf Jul 07 15:13:05.459044 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:05.458938 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80a72d31_7f06_45f3_8c76_9dd94fdb476a.slice/crio-5ba0c1a9479afce1a78692d0762348b96fbb77ae96c706ed481755cfaee1ca91 WatchSource:0}: Error finding container 5ba0c1a9479afce1a78692d0762348b96fbb77ae96c706ed481755cfaee1ca91: Status 404 returned error can't find the container with id 5ba0c1a9479afce1a78692d0762348b96fbb77ae96c706ed481755cfaee1ca91 Jul 07 15:13:05.460495 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:05.460385 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda22234a_c3b5_450f_804c_6b6f6b238d3b.slice/crio-92214065d9e04a9975dee4551c72f4675c60f1ba0814f2b0d81e3215b120d906 WatchSource:0}: Error finding container 92214065d9e04a9975dee4551c72f4675c60f1ba0814f2b0d81e3215b120d906: Status 404 returned error can't find the container with id 92214065d9e04a9975dee4551c72f4675c60f1ba0814f2b0d81e3215b120d906 Jul 07 15:13:05.461846 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:05.461635 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc59d2067_5399_4a3d_9f34_be99e63f7d2b.slice/crio-f3d9a51d097a42b22f239fa7330bda74c82cf6c99f73d22af143efcd9cd324d3 WatchSource:0}: Error finding container f3d9a51d097a42b22f239fa7330bda74c82cf6c99f73d22af143efcd9cd324d3: Status 404 returned error can't find the container with id f3d9a51d097a42b22f239fa7330bda74c82cf6c99f73d22af143efcd9cd324d3 Jul 07 15:13:05.528335 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:05.528169 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqp6p" Jul 07 15:13:05.528421 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:05.528177 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9dwk4" Jul 07 15:13:05.528475 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:05.528450 2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xqp6p" podUID="274e9d34-35d9-452a-996f-1183641bf618" Jul 07 15:13:05.528511 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:05.528485 2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9dwk4" podUID="95c962c7-1672-403f-9d88-acc1fc891e91" Jul 07 15:13:05.536247 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:05.536212 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qkznn" event={"ID":"c59d2067-5399-4a3d-9f34-be99e63f7d2b","Type":"ContainerStarted","Data":"f3d9a51d097a42b22f239fa7330bda74c82cf6c99f73d22af143efcd9cd324d3"} Jul 07 15:13:05.537382 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:05.537360 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-t8hcc" event={"ID":"da22234a-c3b5-450f-804c-6b6f6b238d3b","Type":"ContainerStarted","Data":"92214065d9e04a9975dee4551c72f4675c60f1ba0814f2b0d81e3215b120d906"} Jul 07 15:13:05.538679 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:05.538620 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-sk2gc" event={"ID":"a5099dd3-70a7-4a61-a0da-4aaad1c064f4","Type":"ContainerStarted","Data":"95c0bf3dc918e8a3413b6414269f9915da851ebe95b357bf74ec23d07933c965"} Jul 07 15:13:05.539622 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:05.539603 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2x8hc" event={"ID":"104cc0be-5dba-411e-be9b-6742c6898f6c","Type":"ContainerStarted","Data":"1b0563bf2512a43a3ee214f7c8ac4bf1899e52f63c5beefacbb826ee88128784"} Jul 07 15:13:05.540799 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:05.540777 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mf9rg" event={"ID":"f9f4e27e-0a19-420b-977e-c2ac05efb164","Type":"ContainerStarted","Data":"cfedf2eecb0866ace9e263c4dae42eeb7c0c9ed9750bfdbd2a4dcecbfadb1c5c"} Jul 07 15:13:05.542612 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:05.542574 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-11.ec2.internal" event={"ID":"f3ea74c0531243ed68ab1cca77160244","Type":"ContainerStarted","Data":"456852242b5cdc5481e880d381ac0030b4d920c43e17f8c31d70ae56891869cf"} Jul 07 15:13:05.543837 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:05.543816 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zd2vk" event={"ID":"80a72d31-7f06-45f3-8c76-9dd94fdb476a","Type":"ContainerStarted","Data":"5ba0c1a9479afce1a78692d0762348b96fbb77ae96c706ed481755cfaee1ca91"} Jul 07 15:13:05.544859 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:05.544837 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6dr6g" event={"ID":"3791d99d-ae07-48b8-9311-051b2f83e54a","Type":"ContainerStarted","Data":"284f2597fb50970b5562b9d2e7153c34ff4cc7b1a377f55fa3636e0d1c693ccf"} Jul 07 15:13:05.546045 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:05.546025 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" event={"ID":"22eaf67c-4e87-4339-b266-e1f7330bf89c","Type":"ContainerStarted","Data":"241db0dbd4b861e365a6478a407f6e16456d93e59d92d6d32a9a7e3c00a01d98"} Jul 07 15:13:05.547169 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:05.547151 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dvrfw" event={"ID":"31fc8ff5-b255-4523-9758-bf21b0409efa","Type":"ContainerStarted","Data":"4431410be31db4c0caefa6a73f40958fe976bc0f722ab399d41e073de4419d4f"} Jul 07 15:13:05.558518 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:05.558477 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-11.ec2.internal" podStartSLOduration=2.558466579 podStartE2EDuration="2.558466579s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:13:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:13:05.558354419 +0000 UTC m=+3.565831869" watchObservedRunningTime="2026-07-07 15:13:05.558466579 +0000 UTC m=+3.565944029" Jul 07 15:13:05.979048 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:05.978964 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/274e9d34-35d9-452a-996f-1183641bf618-metrics-certs\") pod \"network-metrics-daemon-xqp6p\" (UID: \"274e9d34-35d9-452a-996f-1183641bf618\") " pod="openshift-multus/network-metrics-daemon-xqp6p" Jul 07 15:13:05.979224 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:05.979106 2549 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:13:05.979224 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:05.979155 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/274e9d34-35d9-452a-996f-1183641bf618-metrics-certs podName:274e9d34-35d9-452a-996f-1183641bf618 nodeName:}" failed. No retries permitted until 2026-07-07 15:13:07.979141139 +0000 UTC m=+5.986618569 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/274e9d34-35d9-452a-996f-1183641bf618-metrics-certs") pod "network-metrics-daemon-xqp6p" (UID: "274e9d34-35d9-452a-996f-1183641bf618") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:13:06.080177 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:06.080136 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgrk9\" (UniqueName: \"kubernetes.io/projected/95c962c7-1672-403f-9d88-acc1fc891e91-kube-api-access-rgrk9\") pod \"network-check-target-9dwk4\" (UID: \"95c962c7-1672-403f-9d88-acc1fc891e91\") " pod="openshift-network-diagnostics/network-check-target-9dwk4" Jul 07 15:13:06.080352 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:06.080201 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/473572dc-5da2-44c1-918d-a409370c493f-original-pull-secret\") pod \"global-pull-secret-syncer-xt54w\" (UID: \"473572dc-5da2-44c1-918d-a409370c493f\") " pod="kube-system/global-pull-secret-syncer-xt54w" Jul 07 15:13:06.080414 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:06.080359 2549 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 15:13:06.080470 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:06.080421 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/473572dc-5da2-44c1-918d-a409370c493f-original-pull-secret podName:473572dc-5da2-44c1-918d-a409370c493f nodeName:}" failed. No retries permitted until 2026-07-07 15:13:08.080403564 +0000 UTC m=+6.087881008 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/473572dc-5da2-44c1-918d-a409370c493f-original-pull-secret") pod "global-pull-secret-syncer-xt54w" (UID: "473572dc-5da2-44c1-918d-a409370c493f") : object "kube-system"/"original-pull-secret" not registered Jul 07 15:13:06.080899 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:06.080882 2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 15:13:06.080996 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:06.080906 2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 15:13:06.080996 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:06.080919 2549 projected.go:196] Error preparing data for projected volume kube-api-access-rgrk9 for pod openshift-network-diagnostics/network-check-target-9dwk4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:13:06.080996 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:06.080963 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/95c962c7-1672-403f-9d88-acc1fc891e91-kube-api-access-rgrk9 podName:95c962c7-1672-403f-9d88-acc1fc891e91 nodeName:}" failed. No retries permitted until 2026-07-07 15:13:08.080949397 +0000 UTC m=+6.088426829 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-rgrk9" (UniqueName: "kubernetes.io/projected/95c962c7-1672-403f-9d88-acc1fc891e91-kube-api-access-rgrk9") pod "network-check-target-9dwk4" (UID: "95c962c7-1672-403f-9d88-acc1fc891e91") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:13:06.396237 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:06.396138 2549 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 15:08:04 +0000 UTC" deadline="2028-03-21 06:30:00.487236344 +0000 UTC" Jul 07 15:13:06.396237 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:06.396201 2549 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14943h16m54.091039811s" Jul 07 15:13:06.530892 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:06.530861 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xt54w" Jul 07 15:13:06.531065 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:06.531003 2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xt54w" podUID="473572dc-5da2-44c1-918d-a409370c493f" Jul 07 15:13:06.565171 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:06.565101 2549 generic.go:356] "Generic (PLEG): container finished" podID="b12720c0f68435524972635f743dfbf4" containerID="0a0a242893ef5d48b75d83560b1192e9a6d4ec146e90ae485ded54b6ae5366d6" exitCode=0 Jul 07 15:13:06.565334 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:06.565278 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-11.ec2.internal" event={"ID":"b12720c0f68435524972635f743dfbf4","Type":"ContainerDied","Data":"0a0a242893ef5d48b75d83560b1192e9a6d4ec146e90ae485ded54b6ae5366d6"} Jul 07 15:13:07.528627 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:07.528592 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9dwk4" Jul 07 15:13:07.529105 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:07.528725 2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9dwk4" podUID="95c962c7-1672-403f-9d88-acc1fc891e91" Jul 07 15:13:07.529173 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:07.529123 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqp6p" Jul 07 15:13:07.529471 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:07.529228 2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xqp6p" podUID="274e9d34-35d9-452a-996f-1183641bf618" Jul 07 15:13:07.587825 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:07.587779 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-11.ec2.internal" event={"ID":"b12720c0f68435524972635f743dfbf4","Type":"ContainerStarted","Data":"0627f177ccd4832403de9d3381adef058fa7910a4d625c52b26ae272566a8f14"} Jul 07 15:13:07.996189 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:07.996103 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/274e9d34-35d9-452a-996f-1183641bf618-metrics-certs\") pod \"network-metrics-daemon-xqp6p\" (UID: \"274e9d34-35d9-452a-996f-1183641bf618\") " pod="openshift-multus/network-metrics-daemon-xqp6p" Jul 07 15:13:07.996365 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:07.996264 2549 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:13:07.996365 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:07.996328 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/274e9d34-35d9-452a-996f-1183641bf618-metrics-certs podName:274e9d34-35d9-452a-996f-1183641bf618 nodeName:}" failed. No retries permitted until 2026-07-07 15:13:11.99631073 +0000 UTC m=+10.003788166 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/274e9d34-35d9-452a-996f-1183641bf618-metrics-certs") pod "network-metrics-daemon-xqp6p" (UID: "274e9d34-35d9-452a-996f-1183641bf618") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:13:08.097502 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:08.097457 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/473572dc-5da2-44c1-918d-a409370c493f-original-pull-secret\") pod \"global-pull-secret-syncer-xt54w\" (UID: \"473572dc-5da2-44c1-918d-a409370c493f\") " pod="kube-system/global-pull-secret-syncer-xt54w" Jul 07 15:13:08.097689 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:08.097596 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgrk9\" (UniqueName: \"kubernetes.io/projected/95c962c7-1672-403f-9d88-acc1fc891e91-kube-api-access-rgrk9\") pod \"network-check-target-9dwk4\" (UID: \"95c962c7-1672-403f-9d88-acc1fc891e91\") " pod="openshift-network-diagnostics/network-check-target-9dwk4" Jul 07 15:13:08.097844 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:08.097825 2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 15:13:08.097907 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:08.097853 2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 15:13:08.097907 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:08.097866 2549 projected.go:196] Error preparing data for projected volume kube-api-access-rgrk9 for pod openshift-network-diagnostics/network-check-target-9dwk4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:13:08.098117 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:08.097949 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/95c962c7-1672-403f-9d88-acc1fc891e91-kube-api-access-rgrk9 podName:95c962c7-1672-403f-9d88-acc1fc891e91 nodeName:}" failed. No retries permitted until 2026-07-07 15:13:12.097907524 +0000 UTC m=+10.105384959 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-rgrk9" (UniqueName: "kubernetes.io/projected/95c962c7-1672-403f-9d88-acc1fc891e91-kube-api-access-rgrk9") pod "network-check-target-9dwk4" (UID: "95c962c7-1672-403f-9d88-acc1fc891e91") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:13:08.098397 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:08.098376 2549 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 15:13:08.098520 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:08.098431 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/473572dc-5da2-44c1-918d-a409370c493f-original-pull-secret podName:473572dc-5da2-44c1-918d-a409370c493f nodeName:}" failed. No retries permitted until 2026-07-07 15:13:12.098417849 +0000 UTC m=+10.105895283 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/473572dc-5da2-44c1-918d-a409370c493f-original-pull-secret") pod "global-pull-secret-syncer-xt54w" (UID: "473572dc-5da2-44c1-918d-a409370c493f") : object "kube-system"/"original-pull-secret" not registered Jul 07 15:13:08.529964 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:08.529728 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xt54w" Jul 07 15:13:08.529964 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:08.529890 2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xt54w" podUID="473572dc-5da2-44c1-918d-a409370c493f" Jul 07 15:13:09.528901 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:09.528865 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9dwk4" Jul 07 15:13:09.529107 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:09.529005 2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9dwk4" podUID="95c962c7-1672-403f-9d88-acc1fc891e91" Jul 07 15:13:09.529258 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:09.529236 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqp6p" Jul 07 15:13:09.529365 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:09.529345 2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xqp6p" podUID="274e9d34-35d9-452a-996f-1183641bf618" Jul 07 15:13:10.528486 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:10.528401 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xt54w" Jul 07 15:13:10.528923 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:10.528536 2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xt54w" podUID="473572dc-5da2-44c1-918d-a409370c493f" Jul 07 15:13:11.528794 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:11.528758 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqp6p" Jul 07 15:13:11.529270 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:11.528758 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9dwk4" Jul 07 15:13:11.529270 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:11.528909 2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xqp6p" podUID="274e9d34-35d9-452a-996f-1183641bf618" Jul 07 15:13:11.529270 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:11.529032 2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9dwk4" podUID="95c962c7-1672-403f-9d88-acc1fc891e91" Jul 07 15:13:12.030669 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:12.030628 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/274e9d34-35d9-452a-996f-1183641bf618-metrics-certs\") pod \"network-metrics-daemon-xqp6p\" (UID: \"274e9d34-35d9-452a-996f-1183641bf618\") " pod="openshift-multus/network-metrics-daemon-xqp6p" Jul 07 15:13:12.031279 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:12.030851 2549 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:13:12.031279 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:12.030932 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/274e9d34-35d9-452a-996f-1183641bf618-metrics-certs podName:274e9d34-35d9-452a-996f-1183641bf618 nodeName:}" failed. No retries permitted until 2026-07-07 15:13:20.030905002 +0000 UTC m=+18.038382455 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/274e9d34-35d9-452a-996f-1183641bf618-metrics-certs") pod "network-metrics-daemon-xqp6p" (UID: "274e9d34-35d9-452a-996f-1183641bf618") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:13:12.132260 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:12.131672 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgrk9\" (UniqueName: \"kubernetes.io/projected/95c962c7-1672-403f-9d88-acc1fc891e91-kube-api-access-rgrk9\") pod \"network-check-target-9dwk4\" (UID: \"95c962c7-1672-403f-9d88-acc1fc891e91\") " pod="openshift-network-diagnostics/network-check-target-9dwk4" Jul 07 15:13:12.132260 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:12.131728 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/473572dc-5da2-44c1-918d-a409370c493f-original-pull-secret\") pod \"global-pull-secret-syncer-xt54w\" (UID: \"473572dc-5da2-44c1-918d-a409370c493f\") " pod="kube-system/global-pull-secret-syncer-xt54w" Jul 07 15:13:12.132260 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:12.131862 2549 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 15:13:12.132260 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:12.131934 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/473572dc-5da2-44c1-918d-a409370c493f-original-pull-secret podName:473572dc-5da2-44c1-918d-a409370c493f nodeName:}" failed. No retries permitted until 2026-07-07 15:13:20.131914107 +0000 UTC m=+18.139391556 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/473572dc-5da2-44c1-918d-a409370c493f-original-pull-secret") pod "global-pull-secret-syncer-xt54w" (UID: "473572dc-5da2-44c1-918d-a409370c493f") : object "kube-system"/"original-pull-secret" not registered Jul 07 15:13:12.132260 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:12.131954 2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 15:13:12.132260 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:12.131982 2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 15:13:12.132260 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:12.131996 2549 projected.go:196] Error preparing data for projected volume kube-api-access-rgrk9 for pod openshift-network-diagnostics/network-check-target-9dwk4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:13:12.132260 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:12.132047 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/95c962c7-1672-403f-9d88-acc1fc891e91-kube-api-access-rgrk9 podName:95c962c7-1672-403f-9d88-acc1fc891e91 nodeName:}" failed. No retries permitted until 2026-07-07 15:13:20.13203129 +0000 UTC m=+18.139508734 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-rgrk9" (UniqueName: "kubernetes.io/projected/95c962c7-1672-403f-9d88-acc1fc891e91-kube-api-access-rgrk9") pod "network-check-target-9dwk4" (UID: "95c962c7-1672-403f-9d88-acc1fc891e91") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:13:12.530170 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:12.530088 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xt54w" Jul 07 15:13:12.530703 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:12.530206 2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xt54w" podUID="473572dc-5da2-44c1-918d-a409370c493f" Jul 07 15:13:13.528641 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:13.528520 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqp6p" Jul 07 15:13:13.528641 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:13.528535 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9dwk4" Jul 07 15:13:13.528866 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:13.528671 2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xqp6p" podUID="274e9d34-35d9-452a-996f-1183641bf618" Jul 07 15:13:13.528866 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:13.528794 2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9dwk4" podUID="95c962c7-1672-403f-9d88-acc1fc891e91" Jul 07 15:13:14.531512 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:14.531483 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xt54w" Jul 07 15:13:14.531997 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:14.531632 2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xt54w" podUID="473572dc-5da2-44c1-918d-a409370c493f" Jul 07 15:13:15.528114 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:15.528078 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9dwk4" Jul 07 15:13:15.528299 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:15.528097 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqp6p" Jul 07 15:13:15.528299 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:15.528223 2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9dwk4" podUID="95c962c7-1672-403f-9d88-acc1fc891e91" Jul 07 15:13:15.528414 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:15.528322 2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xqp6p" podUID="274e9d34-35d9-452a-996f-1183641bf618" Jul 07 15:13:16.528361 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:16.528262 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xt54w" Jul 07 15:13:16.528810 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:16.528424 2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xt54w" podUID="473572dc-5da2-44c1-918d-a409370c493f" Jul 07 15:13:16.610548 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:16.610321 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zd2vk" event={"ID":"80a72d31-7f06-45f3-8c76-9dd94fdb476a","Type":"ContainerStarted","Data":"f0a738600ab3f447593575a38731927b2b053ebc9c2305a49770aa2f585b55cf"} Jul 07 15:13:16.611927 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:16.611897 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dvrfw" event={"ID":"31fc8ff5-b255-4523-9758-bf21b0409efa","Type":"ContainerStarted","Data":"0456cdb62752873596c8dd1dc78ba83f834eadb767aa5f266ee111dd8908f568"} Jul 07 15:13:16.625363 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:16.625308 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-zd2vk" podStartSLOduration=4.21099694 podStartE2EDuration="14.625294566s" totalImagesPullingTime="10.414297626s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:13:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:13:16.625135074 +0000 UTC m=+14.632612526" watchObservedRunningTime="2026-07-07 15:13:16.625294566 +0000 UTC m=+14.632772016" Jul 07 15:13:16.625546 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:16.625407 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-11.ec2.internal" podStartSLOduration=13.625399767 podStartE2EDuration="13.625399767s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:13:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:13:07.608512852 +0000 UTC m=+5.615990301" watchObservedRunningTime="2026-07-07 15:13:16.625399767 +0000 UTC m=+14.632877223" Jul 07 15:13:16.641623 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:16.641557 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-dvrfw" podStartSLOduration=4.232138188 podStartE2EDuration="14.641539212s" totalImagesPullingTime="10.409401024s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:13:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:13:16.641337067 +0000 UTC m=+14.648814527" watchObservedRunningTime="2026-07-07 15:13:16.641539212 +0000 UTC m=+14.649016664" Jul 07 15:13:17.528429 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:17.528393 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqp6p" Jul 07 15:13:17.528947 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:17.528445 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9dwk4" Jul 07 15:13:17.528947 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:17.528532 2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xqp6p" podUID="274e9d34-35d9-452a-996f-1183641bf618" Jul 07 15:13:17.528947 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:17.528673 2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9dwk4" podUID="95c962c7-1672-403f-9d88-acc1fc891e91" Jul 07 15:13:18.328394 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:18.328337 2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-dvrfw" Jul 07 15:13:18.329057 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:18.329033 2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-dvrfw" Jul 07 15:13:18.528592 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:18.528545 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xt54w" Jul 07 15:13:18.529070 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:18.528705 2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xt54w" podUID="473572dc-5da2-44c1-918d-a409370c493f" Jul 07 15:13:19.528255 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:19.528219 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9dwk4" Jul 07 15:13:19.528546 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:19.528218 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqp6p" Jul 07 15:13:19.528546 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:19.528356 2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9dwk4" podUID="95c962c7-1672-403f-9d88-acc1fc891e91" Jul 07 15:13:19.528546 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:19.528443 2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xqp6p" podUID="274e9d34-35d9-452a-996f-1183641bf618" Jul 07 15:13:19.617549 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:19.617517 2549 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 07 15:13:20.095214 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:20.095180 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/274e9d34-35d9-452a-996f-1183641bf618-metrics-certs\") pod \"network-metrics-daemon-xqp6p\" (UID: \"274e9d34-35d9-452a-996f-1183641bf618\") " pod="openshift-multus/network-metrics-daemon-xqp6p" Jul 07 15:13:20.095400 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:20.095347 2549 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:13:20.095456 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:20.095415 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/274e9d34-35d9-452a-996f-1183641bf618-metrics-certs podName:274e9d34-35d9-452a-996f-1183641bf618 nodeName:}" failed. No retries permitted until 2026-07-07 15:13:36.095398976 +0000 UTC m=+34.102876405 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/274e9d34-35d9-452a-996f-1183641bf618-metrics-certs") pod "network-metrics-daemon-xqp6p" (UID: "274e9d34-35d9-452a-996f-1183641bf618") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:13:20.195965 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:20.195875 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/473572dc-5da2-44c1-918d-a409370c493f-original-pull-secret\") pod \"global-pull-secret-syncer-xt54w\" (UID: \"473572dc-5da2-44c1-918d-a409370c493f\") " pod="kube-system/global-pull-secret-syncer-xt54w" Jul 07 15:13:20.195965 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:20.195954 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgrk9\" (UniqueName: \"kubernetes.io/projected/95c962c7-1672-403f-9d88-acc1fc891e91-kube-api-access-rgrk9\") pod \"network-check-target-9dwk4\" (UID: \"95c962c7-1672-403f-9d88-acc1fc891e91\") " pod="openshift-network-diagnostics/network-check-target-9dwk4" Jul 07 15:13:20.196219 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:20.196070 2549 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 15:13:20.196219 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:20.196153 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/473572dc-5da2-44c1-918d-a409370c493f-original-pull-secret podName:473572dc-5da2-44c1-918d-a409370c493f nodeName:}" failed. No retries permitted until 2026-07-07 15:13:36.196132274 +0000 UTC m=+34.203609720 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/473572dc-5da2-44c1-918d-a409370c493f-original-pull-secret") pod "global-pull-secret-syncer-xt54w" (UID: "473572dc-5da2-44c1-918d-a409370c493f") : object "kube-system"/"original-pull-secret" not registered Jul 07 15:13:20.196219 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:20.196079 2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 15:13:20.196219 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:20.196208 2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 15:13:20.196219 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:20.196224 2549 projected.go:196] Error preparing data for projected volume kube-api-access-rgrk9 for pod openshift-network-diagnostics/network-check-target-9dwk4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:13:20.196461 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:20.196283 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/95c962c7-1672-403f-9d88-acc1fc891e91-kube-api-access-rgrk9 podName:95c962c7-1672-403f-9d88-acc1fc891e91 nodeName:}" failed. No retries permitted until 2026-07-07 15:13:36.196266086 +0000 UTC m=+34.203743533 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-rgrk9" (UniqueName: "kubernetes.io/projected/95c962c7-1672-403f-9d88-acc1fc891e91-kube-api-access-rgrk9") pod "network-check-target-9dwk4" (UID: "95c962c7-1672-403f-9d88-acc1fc891e91") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:13:20.528798 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:20.528713 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xt54w" Jul 07 15:13:20.529191 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:20.528863 2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xt54w" podUID="473572dc-5da2-44c1-918d-a409370c493f" Jul 07 15:13:21.352234 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:21.352198 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-dvrfw" Jul 07 15:13:21.352436 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:21.352333 2549 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 07 15:13:21.352851 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:21.352824 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-dvrfw" Jul 07 15:13:21.528964 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:21.528929 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9dwk4" Jul 07 15:13:21.529416 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:21.528928 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqp6p" Jul 07 15:13:21.529416 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:21.529079 2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9dwk4" podUID="95c962c7-1672-403f-9d88-acc1fc891e91" Jul 07 15:13:21.529416 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:21.529140 2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xqp6p" podUID="274e9d34-35d9-452a-996f-1183641bf618" Jul 07 15:13:22.530033 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:22.529916 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xt54w" Jul 07 15:13:22.530033 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:22.530030 2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xt54w" podUID="473572dc-5da2-44c1-918d-a409370c493f" Jul 07 15:13:23.529135 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:23.529097 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9dwk4" Jul 07 15:13:23.529324 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:23.529097 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqp6p" Jul 07 15:13:23.529324 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:23.529244 2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9dwk4" podUID="95c962c7-1672-403f-9d88-acc1fc891e91" Jul 07 15:13:23.529419 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:23.529318 2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xqp6p" podUID="274e9d34-35d9-452a-996f-1183641bf618" Jul 07 15:13:24.528451 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:24.528152 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xt54w" Jul 07 15:13:24.529022 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:24.528601 2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xt54w" podUID="473572dc-5da2-44c1-918d-a409370c493f" Jul 07 15:13:24.629078 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:24.629049 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2x8hc" event={"ID":"104cc0be-5dba-411e-be9b-6742c6898f6c","Type":"ContainerStarted","Data":"133264088c13c55e92fc3f12bef422eea3b4ea52172caf97fa78838d11c58f42"} Jul 07 15:13:24.630564 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:24.630541 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mf9rg" event={"ID":"f9f4e27e-0a19-420b-977e-c2ac05efb164","Type":"ContainerStarted","Data":"69469edead57ba5e277df517604bf3a479430577b437df32cd045ab9a7650647"} Jul 07 15:13:24.631991 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:24.631959 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6dr6g" event={"ID":"3791d99d-ae07-48b8-9311-051b2f83e54a","Type":"ContainerStarted","Data":"fd387fbed20c40107fcb97cf54909218e79d3f18dd0430ba2959dc99c6359620"} Jul 07 15:13:24.634292 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:24.634268 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" event={"ID":"22eaf67c-4e87-4339-b266-e1f7330bf89c","Type":"ContainerStarted","Data":"2b840868f9a5ae49029ba5f157623a5034e9184a28e233d83a93818f6aa5ff99"} Jul 07 15:13:24.634396 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:24.634344 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" event={"ID":"22eaf67c-4e87-4339-b266-e1f7330bf89c","Type":"ContainerStarted","Data":"30ebfec4b0dade4853a14c8da931301cd722e9156f8ec50da8003347250ae897"} Jul 07 15:13:24.634396 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:24.634391 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" event={"ID":"22eaf67c-4e87-4339-b266-e1f7330bf89c","Type":"ContainerStarted","Data":"7bef7920b202203754058913de08c236dce83ba41698250890f0fc50d149d7dc"} Jul 07 15:13:24.634498 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:24.634440 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" event={"ID":"22eaf67c-4e87-4339-b266-e1f7330bf89c","Type":"ContainerStarted","Data":"dcbce464dcd6254e7d5cc5dcacfa13f9ec7ad0f72f5f23879c2270e64d808bda"} Jul 07 15:13:24.635695 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:24.635671 2549 generic.go:356] "Generic (PLEG): container finished" podID="c59d2067-5399-4a3d-9f34-be99e63f7d2b" containerID="0b8d56e0738054ea37152e1d3a5e8c41a10c352a2b6bd9dd6509dc10cedf4a1a" exitCode=0 Jul 07 15:13:24.635791 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:24.635751 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qkznn" event={"ID":"c59d2067-5399-4a3d-9f34-be99e63f7d2b","Type":"ContainerDied","Data":"0b8d56e0738054ea37152e1d3a5e8c41a10c352a2b6bd9dd6509dc10cedf4a1a"} Jul 07 15:13:24.636980 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:24.636959 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-t8hcc" event={"ID":"da22234a-c3b5-450f-804c-6b6f6b238d3b","Type":"ContainerStarted","Data":"19eae05733974e81b430c6b7741acbd0ac84793bc30b525e089a05c840eefc29"} Jul 07 15:13:24.646289 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:24.646254 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-2x8hc" podStartSLOduration=4.043496059 podStartE2EDuration="22.646244085s" totalImagesPullingTime="18.602748026s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:13:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:13:24.645554355 +0000 UTC m=+22.653031827" watchObservedRunningTime="2026-07-07 15:13:24.646244085 +0000 UTC m=+22.653721537" Jul 07 15:13:24.662886 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:24.662847 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-t8hcc" podStartSLOduration=12.229840332 podStartE2EDuration="22.662834573s" totalImagesPullingTime="10.432994241s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:13:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:13:24.662310289 +0000 UTC m=+22.669787741" watchObservedRunningTime="2026-07-07 15:13:24.662834573 +0000 UTC m=+22.670312024" Jul 07 15:13:24.682313 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:24.682247 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-mf9rg" podStartSLOduration=4.014075829 podStartE2EDuration="22.682236166s" totalImagesPullingTime="18.668160337s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:13:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:13:24.681242963 +0000 UTC m=+22.688720416" watchObservedRunningTime="2026-07-07 15:13:24.682236166 +0000 UTC m=+22.689713616" Jul 07 15:13:25.528819 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:25.528788 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqp6p" Jul 07 15:13:25.529208 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:25.528804 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9dwk4" Jul 07 15:13:25.529208 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:25.528919 2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xqp6p" podUID="274e9d34-35d9-452a-996f-1183641bf618" Jul 07 15:13:25.529208 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:25.528948 2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9dwk4" podUID="95c962c7-1672-403f-9d88-acc1fc891e91" Jul 07 15:13:25.640655 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:25.640599 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-sk2gc" event={"ID":"a5099dd3-70a7-4a61-a0da-4aaad1c064f4","Type":"ContainerStarted","Data":"4ebe0233eece2bde3b19102b36692f07b16972b29fa1801c51dd18fe8f18d75a"} Jul 07 15:13:25.643673 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:25.643649 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" event={"ID":"22eaf67c-4e87-4339-b266-e1f7330bf89c","Type":"ContainerStarted","Data":"20ce0a64206661c18bdc67c9850df98640459033ce4c354545f62678c9f8fd4b"} Jul 07 15:13:25.643780 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:25.643724 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" event={"ID":"22eaf67c-4e87-4339-b266-e1f7330bf89c","Type":"ContainerStarted","Data":"09b03a1ba5dcf6df6398439a48d47406d6ba68142af5a6d5d809407f2e542bcd"} Jul 07 15:13:25.662259 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:25.662204 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-sk2gc" podStartSLOduration=13.21777059 podStartE2EDuration="23.662186208s" totalImagesPullingTime="10.444415618s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:13:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:13:25.661520865 +0000 UTC m=+23.668998318" watchObservedRunningTime="2026-07-07 15:13:25.662186208 +0000 UTC m=+23.669663659" Jul 07 15:13:25.773212 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:25.773175 2549 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 07 15:13:26.410229 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:26.410099 2549 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T15:13:25.773193334Z","UUID":"98478406-2b75-4396-89f8-0ab4995fe445","Handler":null,"Name":"","Endpoint":""} Jul 07 15:13:26.411886 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:26.411860 2549 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 07 15:13:26.412006 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:26.411907 2549 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 07 15:13:26.528887 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:26.528854 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xt54w" Jul 07 15:13:26.529319 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:26.528991 2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xt54w" podUID="473572dc-5da2-44c1-918d-a409370c493f" Jul 07 15:13:26.647840 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:26.647792 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6dr6g" event={"ID":"3791d99d-ae07-48b8-9311-051b2f83e54a","Type":"ContainerStarted","Data":"cda98851b526b31255fb995c27d805f1769a93a35a9517a2c6e815494f7df8cf"} Jul 07 15:13:27.528886 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:27.528855 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9dwk4" Jul 07 15:13:27.529080 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:27.528861 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqp6p" Jul 07 15:13:27.529080 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:27.528969 2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9dwk4" podUID="95c962c7-1672-403f-9d88-acc1fc891e91" Jul 07 15:13:27.529435 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:27.529076 2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xqp6p" podUID="274e9d34-35d9-452a-996f-1183641bf618" Jul 07 15:13:27.652653 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:27.652615 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6dr6g" event={"ID":"3791d99d-ae07-48b8-9311-051b2f83e54a","Type":"ContainerStarted","Data":"3065fa163d8820514916216f4d4079394ae69e8bfa2e9aeb7c47dff6b0d4d985"} Jul 07 15:13:27.656273 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:27.656244 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" event={"ID":"22eaf67c-4e87-4339-b266-e1f7330bf89c","Type":"ContainerStarted","Data":"f7d327a9a2bf2021ff0017693d0ad48515bd266ad485fe10e9e2c7f9aa0756c3"} Jul 07 15:13:27.672375 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:27.672325 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6dr6g" podStartSLOduration=12.702453637 podStartE2EDuration="25.672306665s" totalImagesPullingTime="12.969853028s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:13:02 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:13:27.67078368 +0000 UTC m=+25.678261133" watchObservedRunningTime="2026-07-07 15:13:27.672306665 +0000 UTC m=+25.679784116" Jul 07 15:13:28.528501 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:28.528323 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xt54w" Jul 07 15:13:28.528711 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:28.528623 2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xt54w" podUID="473572dc-5da2-44c1-918d-a409370c493f" Jul 07 15:13:29.528720 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:29.528629 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9dwk4" Jul 07 15:13:29.529145 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:29.528638 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqp6p" Jul 07 15:13:29.529145 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:29.528738 2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9dwk4" podUID="95c962c7-1672-403f-9d88-acc1fc891e91" Jul 07 15:13:29.529145 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:29.528820 2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xqp6p" podUID="274e9d34-35d9-452a-996f-1183641bf618" Jul 07 15:13:30.528931 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:30.528713 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xt54w" Jul 07 15:13:30.529517 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:30.528962 2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xt54w" podUID="473572dc-5da2-44c1-918d-a409370c493f" Jul 07 15:13:30.666462 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:30.666425 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" event={"ID":"22eaf67c-4e87-4339-b266-e1f7330bf89c","Type":"ContainerStarted","Data":"e60f9317d73e6231c0325c5bb6feddbc3f71fcb7c7e1bc6e9a621eed64463260"} Jul 07 15:13:30.666747 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:30.666722 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:30.666884 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:30.666798 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:30.666884 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:30.666849 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:30.668418 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:30.668391 2549 generic.go:356] "Generic (PLEG): container finished" podID="c59d2067-5399-4a3d-9f34-be99e63f7d2b" containerID="7c908c33d421b5d2a1a9393e62129682b276fc3086a3f0703b9b1b64a0d69890" exitCode=0 Jul 07 15:13:30.668493 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:30.668450 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qkznn" event={"ID":"c59d2067-5399-4a3d-9f34-be99e63f7d2b","Type":"ContainerDied","Data":"7c908c33d421b5d2a1a9393e62129682b276fc3086a3f0703b9b1b64a0d69890"} Jul 07 15:13:30.681922 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:30.681895 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:30.682025 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:30.682005 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:13:30.698919 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:30.698518 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" podStartSLOduration=10.028832043 podStartE2EDuration="28.698498407s" totalImagesPullingTime="18.669666364s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:13:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:13:30.69702989 +0000 UTC m=+28.704507343" watchObservedRunningTime="2026-07-07 15:13:30.698498407 +0000 UTC m=+28.705975873" Jul 07 15:13:31.501645 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:31.501614 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9dwk4"] Jul 07 15:13:31.501841 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:31.501778 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9dwk4" Jul 07 15:13:31.501894 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:31.501872 2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9dwk4" podUID="95c962c7-1672-403f-9d88-acc1fc891e91" Jul 07 15:13:31.504953 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:31.504931 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xt54w"] Jul 07 15:13:31.505076 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:31.505020 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xt54w" Jul 07 15:13:31.505126 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:31.505103 2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xt54w" podUID="473572dc-5da2-44c1-918d-a409370c493f" Jul 07 15:13:31.505654 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:31.505626 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xqp6p"] Jul 07 15:13:31.505734 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:31.505722 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqp6p" Jul 07 15:13:31.505864 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:31.505830 2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xqp6p" podUID="274e9d34-35d9-452a-996f-1183641bf618" Jul 07 15:13:32.529233 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:32.529197 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xt54w" Jul 07 15:13:32.529804 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:32.529700 2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xt54w" podUID="473572dc-5da2-44c1-918d-a409370c493f" Jul 07 15:13:32.675303 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:32.675263 2549 generic.go:356] "Generic (PLEG): container finished" podID="c59d2067-5399-4a3d-9f34-be99e63f7d2b" containerID="2806d2ebf777280a18bd0866f9d108d74546247bbe958c8a5848ea401fd921d7" exitCode=0 Jul 07 15:13:32.675490 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:32.675350 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qkznn" event={"ID":"c59d2067-5399-4a3d-9f34-be99e63f7d2b","Type":"ContainerDied","Data":"2806d2ebf777280a18bd0866f9d108d74546247bbe958c8a5848ea401fd921d7"} Jul 07 15:13:33.528766 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:33.528688 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqp6p" Jul 07 15:13:33.528766 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:33.528711 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9dwk4" Jul 07 15:13:33.528945 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:33.528809 2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xqp6p" podUID="274e9d34-35d9-452a-996f-1183641bf618" Jul 07 15:13:33.528945 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:33.528909 2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9dwk4" podUID="95c962c7-1672-403f-9d88-acc1fc891e91" Jul 07 15:13:34.528969 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:34.528932 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xt54w" Jul 07 15:13:34.529493 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:34.529044 2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xt54w" podUID="473572dc-5da2-44c1-918d-a409370c493f" Jul 07 15:13:34.682133 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:34.682100 2549 generic.go:356] "Generic (PLEG): container finished" podID="c59d2067-5399-4a3d-9f34-be99e63f7d2b" containerID="6594736ec9fc724b04b97026e2cf354fb011566d860036aab61b81bc27c64c78" exitCode=0 Jul 07 15:13:34.682304 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:34.682151 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qkznn" event={"ID":"c59d2067-5399-4a3d-9f34-be99e63f7d2b","Type":"ContainerDied","Data":"6594736ec9fc724b04b97026e2cf354fb011566d860036aab61b81bc27c64c78"} Jul 07 15:13:35.528924 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:35.528889 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqp6p" Jul 07 15:13:35.529139 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:35.529025 2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xqp6p" podUID="274e9d34-35d9-452a-996f-1183641bf618" Jul 07 15:13:35.529139 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:35.529076 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9dwk4" Jul 07 15:13:35.529680 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:35.529168 2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9dwk4" podUID="95c962c7-1672-403f-9d88-acc1fc891e91" Jul 07 15:13:36.114125 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.114094 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/274e9d34-35d9-452a-996f-1183641bf618-metrics-certs\") pod \"network-metrics-daemon-xqp6p\" (UID: \"274e9d34-35d9-452a-996f-1183641bf618\") " pod="openshift-multus/network-metrics-daemon-xqp6p" Jul 07 15:13:36.114297 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:36.114255 2549 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:13:36.114346 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:36.114319 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/274e9d34-35d9-452a-996f-1183641bf618-metrics-certs podName:274e9d34-35d9-452a-996f-1183641bf618 nodeName:}" failed. No retries permitted until 2026-07-07 15:14:08.114301149 +0000 UTC m=+66.121778587 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/274e9d34-35d9-452a-996f-1183641bf618-metrics-certs") pod "network-metrics-daemon-xqp6p" (UID: "274e9d34-35d9-452a-996f-1183641bf618") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:13:36.215181 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.215148 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgrk9\" (UniqueName: \"kubernetes.io/projected/95c962c7-1672-403f-9d88-acc1fc891e91-kube-api-access-rgrk9\") pod \"network-check-target-9dwk4\" (UID: \"95c962c7-1672-403f-9d88-acc1fc891e91\") " pod="openshift-network-diagnostics/network-check-target-9dwk4" Jul 07 15:13:36.215181 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.215187 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/473572dc-5da2-44c1-918d-a409370c493f-original-pull-secret\") pod \"global-pull-secret-syncer-xt54w\" (UID: \"473572dc-5da2-44c1-918d-a409370c493f\") " pod="kube-system/global-pull-secret-syncer-xt54w" Jul 07 15:13:36.215437 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:36.215354 2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 15:13:36.215437 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:36.215372 2549 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 15:13:36.215437 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:36.215380 2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 15:13:36.215437 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:36.215394 2549 projected.go:196] Error preparing data for projected volume kube-api-access-rgrk9 for pod openshift-network-diagnostics/network-check-target-9dwk4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:13:36.215437 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:36.215426 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/473572dc-5da2-44c1-918d-a409370c493f-original-pull-secret podName:473572dc-5da2-44c1-918d-a409370c493f nodeName:}" failed. No retries permitted until 2026-07-07 15:14:08.215409402 +0000 UTC m=+66.222886851 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/473572dc-5da2-44c1-918d-a409370c493f-original-pull-secret") pod "global-pull-secret-syncer-xt54w" (UID: "473572dc-5da2-44c1-918d-a409370c493f") : object "kube-system"/"original-pull-secret" not registered Jul 07 15:13:36.215437 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:36.215440 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/95c962c7-1672-403f-9d88-acc1fc891e91-kube-api-access-rgrk9 podName:95c962c7-1672-403f-9d88-acc1fc891e91 nodeName:}" failed. No retries permitted until 2026-07-07 15:14:08.215434674 +0000 UTC m=+66.222912104 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-rgrk9" (UniqueName: "kubernetes.io/projected/95c962c7-1672-403f-9d88-acc1fc891e91-kube-api-access-rgrk9") pod "network-check-target-9dwk4" (UID: "95c962c7-1672-403f-9d88-acc1fc891e91") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:13:36.528429 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.528398 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xt54w" Jul 07 15:13:36.528598 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:36.528537 2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xt54w" podUID="473572dc-5da2-44c1-918d-a409370c493f" Jul 07 15:13:36.696224 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.696191 2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-11.ec2.internal" event="NodeReady" Jul 07 15:13:36.696652 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.696353 2549 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 07 15:13:36.734391 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.734270 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-689c879f5d-clmr6"] Jul 07 15:13:36.736333 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.736309 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-689c879f5d-clmr6" Jul 07 15:13:36.737301 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.737277 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-wdtl4"] Jul 07 15:13:36.739074 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.738866 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 07 15:13:36.739284 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.739268 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-wdtl4" Jul 07 15:13:36.739543 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.739509 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 07 15:13:36.739688 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.739659 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-st7wq\"" Jul 07 15:13:36.739756 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.739703 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 07 15:13:36.742053 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.742032 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 07 15:13:36.742149 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.742114 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 07 15:13:36.742248 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.742231 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 07 15:13:36.742317 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.742271 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 07 15:13:36.742373 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.742338 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-rdqtr\"" Jul 07 15:13:36.747562 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.747539 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 07 15:13:36.751892 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.751301 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-dgd58"] Jul 07 15:13:36.759968 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.757440 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-88b42"] Jul 07 15:13:36.759968 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.757567 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dgd58" Jul 07 15:13:36.761661 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.761223 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-pb8qd\"" Jul 07 15:13:36.761661 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.761455 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 07 15:13:36.761819 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.761699 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 07 15:13:36.762119 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.761977 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-ntm4v"] Jul 07 15:13:36.762207 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.762145 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-88b42" Jul 07 15:13:36.763661 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.763635 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-64478b6d7c-vxvp5"] Jul 07 15:13:36.763845 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.763826 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-ntm4v" Jul 07 15:13:36.764470 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.764452 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-8bj6f\"" Jul 07 15:13:36.765229 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.765187 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 07 15:13:36.765326 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.765202 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 07 15:13:36.765395 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.765353 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 07 15:13:36.765395 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.765352 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 07 15:13:36.765841 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.765821 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-9f95m"] Jul 07 15:13:36.765928 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.765908 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-64478b6d7c-vxvp5" Jul 07 15:13:36.766862 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.766840 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 07 15:13:36.766967 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.766940 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 07 15:13:36.770727 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.767647 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ddv8m"] Jul 07 15:13:36.770727 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.768298 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-9f95m" Jul 07 15:13:36.770727 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.769724 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 07 15:13:36.770727 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.769864 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 07 15:13:36.770727 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.769897 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-kspmr\"" Jul 07 15:13:36.770727 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.770086 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 07 15:13:36.770727 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.770161 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 07 15:13:36.770727 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.770168 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-66pbb"] Jul 07 15:13:36.770727 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.770461 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 07 15:13:36.770727 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.770474 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 07 15:13:36.770727 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.770685 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ddv8m" Jul 07 15:13:36.773166 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.770734 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 07 15:13:36.773166 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.771390 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 07 15:13:36.773166 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.771492 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-rf9jr\"" Jul 07 15:13:36.773166 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.771780 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 07 15:13:36.773166 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.772011 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-f9dsj\"" Jul 07 15:13:36.773166 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.772457 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-kb4ml"] Jul 07 15:13:36.773166 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.772828 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-66pbb" Jul 07 15:13:36.776247 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.776228 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 07 15:13:36.776646 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.776611 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 07 15:13:36.776834 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.776813 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 07 15:13:36.776981 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.776936 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 07 15:13:36.777054 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.777027 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-lb5l7"] Jul 07 15:13:36.777146 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.777110 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 07 15:13:36.777208 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.777130 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-4wpvl\"" Jul 07 15:13:36.777422 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.777324 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-cxv4p\"" Jul 07 15:13:36.777645 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.777626 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-kb4ml" Jul 07 15:13:36.779237 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.777633 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 07 15:13:36.780011 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.779374 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 07 15:13:36.780081 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.780010 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 07 15:13:36.780515 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.780492 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 07 15:13:36.780825 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.780809 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-zh2mx"] Jul 07 15:13:36.781607 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.781012 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-lb5l7" Jul 07 15:13:36.782954 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.782345 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-slrtq\"" Jul 07 15:13:36.782954 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.782558 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 07 15:13:36.782954 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.782839 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 07 15:13:36.782954 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.782864 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 07 15:13:36.783796 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.783297 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 07 15:13:36.783921 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.783900 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 15:13:36.784991 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.784490 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-knv9f\"" Jul 07 15:13:36.784991 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.784683 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 15:13:36.785656 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.785639 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-wdtl4"] Jul 07 15:13:36.785820 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.785805 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-689c879f5d-clmr6"] Jul 07 15:13:36.785938 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.785925 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-7smq7"] Jul 07 15:13:36.786295 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.785930 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-zh2mx" Jul 07 15:13:36.791515 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.791483 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ddv8m"] Jul 07 15:13:36.791639 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.791532 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-88b42"] Jul 07 15:13:36.791639 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.791556 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-ntm4v"] Jul 07 15:13:36.791758 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.791646 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-66pbb"] Jul 07 15:13:36.791758 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.791711 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-zh2mx"] Jul 07 15:13:36.791758 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.791737 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-kb4ml"] Jul 07 15:13:36.791985 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.791970 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7smq7"] Jul 07 15:13:36.792048 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.792001 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dgd58"] Jul 07 15:13:36.792048 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.792031 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-64478b6d7c-vxvp5"] Jul 07 15:13:36.792152 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.792140 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7smq7" Jul 07 15:13:36.795063 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.795036 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 07 15:13:36.796033 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.796009 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-c5fvs\"" Jul 07 15:13:36.796302 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.796283 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 07 15:13:36.796762 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.796735 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-9f95m"] Jul 07 15:13:36.796851 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.796806 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 07 15:13:36.800521 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.800503 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 07 15:13:36.802532 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.802381 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-lb5l7"] Jul 07 15:13:36.802656 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.802639 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 07 15:13:36.802760 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.802729 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 07 15:13:36.802856 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.802808 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-24msw\"" Jul 07 15:13:36.819847 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.819818 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/22d3c83b-a693-4f0a-8ae2-07d83467c620-image-registry-private-configuration\") pod \"image-registry-689c879f5d-clmr6\" (UID: \"22d3c83b-a693-4f0a-8ae2-07d83467c620\") " pod="openshift-image-registry/image-registry-689c879f5d-clmr6" Jul 07 15:13:36.819948 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.819864 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/22d3c83b-a693-4f0a-8ae2-07d83467c620-registry-tls\") pod \"image-registry-689c879f5d-clmr6\" (UID: \"22d3c83b-a693-4f0a-8ae2-07d83467c620\") " pod="openshift-image-registry/image-registry-689c879f5d-clmr6" Jul 07 15:13:36.819948 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.819892 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8056aeaf-ce8a-4d1d-a6b6-c0df02057284-config\") pod \"service-ca-operator-5d46d9ffd4-wdtl4\" (UID: \"8056aeaf-ce8a-4d1d-a6b6-c0df02057284\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-wdtl4" Jul 07 15:13:36.819948 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.819917 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/22d3c83b-a693-4f0a-8ae2-07d83467c620-registry-certificates\") pod \"image-registry-689c879f5d-clmr6\" (UID: \"22d3c83b-a693-4f0a-8ae2-07d83467c620\") " pod="openshift-image-registry/image-registry-689c879f5d-clmr6" Jul 07 15:13:36.820113 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.819967 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/22d3c83b-a693-4f0a-8ae2-07d83467c620-ca-trust-extracted\") pod \"image-registry-689c879f5d-clmr6\" (UID: \"22d3c83b-a693-4f0a-8ae2-07d83467c620\") " pod="openshift-image-registry/image-registry-689c879f5d-clmr6" Jul 07 15:13:36.820113 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.819989 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/22d3c83b-a693-4f0a-8ae2-07d83467c620-trusted-ca\") pod \"image-registry-689c879f5d-clmr6\" (UID: \"22d3c83b-a693-4f0a-8ae2-07d83467c620\") " pod="openshift-image-registry/image-registry-689c879f5d-clmr6" Jul 07 15:13:36.820113 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.820013 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8056aeaf-ce8a-4d1d-a6b6-c0df02057284-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-wdtl4\" (UID: \"8056aeaf-ce8a-4d1d-a6b6-c0df02057284\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-wdtl4" Jul 07 15:13:36.820113 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.820040 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zflwn\" (UniqueName: \"kubernetes.io/projected/22d3c83b-a693-4f0a-8ae2-07d83467c620-kube-api-access-zflwn\") pod \"image-registry-689c879f5d-clmr6\" (UID: \"22d3c83b-a693-4f0a-8ae2-07d83467c620\") " pod="openshift-image-registry/image-registry-689c879f5d-clmr6" Jul 07 15:13:36.820113 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.820069 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfgl7\" (UniqueName: \"kubernetes.io/projected/8056aeaf-ce8a-4d1d-a6b6-c0df02057284-kube-api-access-sfgl7\") pod \"service-ca-operator-5d46d9ffd4-wdtl4\" (UID: \"8056aeaf-ce8a-4d1d-a6b6-c0df02057284\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-wdtl4" Jul 07 15:13:36.820352 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.820110 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/22d3c83b-a693-4f0a-8ae2-07d83467c620-installation-pull-secrets\") pod \"image-registry-689c879f5d-clmr6\" (UID: \"22d3c83b-a693-4f0a-8ae2-07d83467c620\") " pod="openshift-image-registry/image-registry-689c879f5d-clmr6" Jul 07 15:13:36.820352 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.820178 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/22d3c83b-a693-4f0a-8ae2-07d83467c620-bound-sa-token\") pod \"image-registry-689c879f5d-clmr6\" (UID: \"22d3c83b-a693-4f0a-8ae2-07d83467c620\") " pod="openshift-image-registry/image-registry-689c879f5d-clmr6" Jul 07 15:13:36.921208 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.921174 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/22d3c83b-a693-4f0a-8ae2-07d83467c620-trusted-ca\") pod \"image-registry-689c879f5d-clmr6\" (UID: \"22d3c83b-a693-4f0a-8ae2-07d83467c620\") " pod="openshift-image-registry/image-registry-689c879f5d-clmr6" Jul 07 15:13:36.921380 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.921218 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8056aeaf-ce8a-4d1d-a6b6-c0df02057284-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-wdtl4\" (UID: \"8056aeaf-ce8a-4d1d-a6b6-c0df02057284\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-wdtl4" Jul 07 15:13:36.921380 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.921250 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2a8ea65f-139e-4b35-8a9f-1471da0e27ed-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-66pbb\" (UID: \"2a8ea65f-139e-4b35-8a9f-1471da0e27ed\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-66pbb" Jul 07 15:13:36.921380 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.921280 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77ec27ef-75d0-454a-9cd5-803a776c4985-metrics-certs\") pod \"router-default-64478b6d7c-vxvp5\" (UID: \"77ec27ef-75d0-454a-9cd5-803a776c4985\") " pod="openshift-ingress/router-default-64478b6d7c-vxvp5" Jul 07 15:13:36.921380 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.921305 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zmzw\" (UniqueName: \"kubernetes.io/projected/42fc06ec-88cb-4308-9070-c4defaf37385-kube-api-access-6zmzw\") pod \"dns-default-dgd58\" (UID: \"42fc06ec-88cb-4308-9070-c4defaf37385\") " pod="openshift-dns/dns-default-dgd58" Jul 07 15:13:36.921380 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.921333 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/22d3c83b-a693-4f0a-8ae2-07d83467c620-installation-pull-secrets\") pod \"image-registry-689c879f5d-clmr6\" (UID: \"22d3c83b-a693-4f0a-8ae2-07d83467c620\") " pod="openshift-image-registry/image-registry-689c879f5d-clmr6" Jul 07 15:13:36.921380 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.921360 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8cb2a4ff-480a-47b8-b781-f695be5d74a1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-ddv8m\" (UID: \"8cb2a4ff-480a-47b8-b781-f695be5d74a1\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ddv8m" Jul 07 15:13:36.921684 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.921394 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/22d3c83b-a693-4f0a-8ae2-07d83467c620-bound-sa-token\") pod \"image-registry-689c879f5d-clmr6\" (UID: \"22d3c83b-a693-4f0a-8ae2-07d83467c620\") " pod="openshift-image-registry/image-registry-689c879f5d-clmr6" Jul 07 15:13:36.921684 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.921423 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3301f6ef-6671-4755-9047-16959836232c-serving-cert\") pod \"console-operator-68b757fb67-kb4ml\" (UID: \"3301f6ef-6671-4755-9047-16959836232c\") " pod="openshift-console-operator/console-operator-68b757fb67-kb4ml" Jul 07 15:13:36.921684 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.921454 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4f1ac273-713a-4bb6-94d4-4ec09090a194-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-zh2mx\" (UID: \"4f1ac273-713a-4bb6-94d4-4ec09090a194\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-zh2mx" Jul 07 15:13:36.921684 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.921478 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/665ed2cb-7aaf-42ab-8731-03e7cea27fc4-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-88b42\" (UID: \"665ed2cb-7aaf-42ab-8731-03e7cea27fc4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-88b42" Jul 07 15:13:36.921684 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.921540 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/77ec27ef-75d0-454a-9cd5-803a776c4985-default-certificate\") pod \"router-default-64478b6d7c-vxvp5\" (UID: \"77ec27ef-75d0-454a-9cd5-803a776c4985\") " pod="openshift-ingress/router-default-64478b6d7c-vxvp5" Jul 07 15:13:36.921684 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.921557 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6bd34c29-c215-4974-98fd-7759cb30f6b8-serving-cert\") pod \"insights-operator-5c847dcfc4-ntm4v\" (UID: \"6bd34c29-c215-4974-98fd-7759cb30f6b8\") " pod="openshift-insights/insights-operator-5c847dcfc4-ntm4v" Jul 07 15:13:36.921684 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.921576 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77ec27ef-75d0-454a-9cd5-803a776c4985-service-ca-bundle\") pod \"router-default-64478b6d7c-vxvp5\" (UID: \"77ec27ef-75d0-454a-9cd5-803a776c4985\") " pod="openshift-ingress/router-default-64478b6d7c-vxvp5" Jul 07 15:13:36.921684 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.921644 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/42fc06ec-88cb-4308-9070-c4defaf37385-config-volume\") pod \"dns-default-dgd58\" (UID: \"42fc06ec-88cb-4308-9070-c4defaf37385\") " pod="openshift-dns/dns-default-dgd58" Jul 07 15:13:36.922038 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.921712 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8056aeaf-ce8a-4d1d-a6b6-c0df02057284-config\") pod \"service-ca-operator-5d46d9ffd4-wdtl4\" (UID: \"8056aeaf-ce8a-4d1d-a6b6-c0df02057284\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-wdtl4" Jul 07 15:13:36.922038 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.921798 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/8cb2a4ff-480a-47b8-b781-f695be5d74a1-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-ddv8m\" (UID: \"8cb2a4ff-480a-47b8-b781-f695be5d74a1\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ddv8m" Jul 07 15:13:36.922038 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.921838 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/22d3c83b-a693-4f0a-8ae2-07d83467c620-ca-trust-extracted\") pod \"image-registry-689c879f5d-clmr6\" (UID: \"22d3c83b-a693-4f0a-8ae2-07d83467c620\") " pod="openshift-image-registry/image-registry-689c879f5d-clmr6" Jul 07 15:13:36.922038 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.921867 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfgl7\" (UniqueName: \"kubernetes.io/projected/8056aeaf-ce8a-4d1d-a6b6-c0df02057284-kube-api-access-sfgl7\") pod \"service-ca-operator-5d46d9ffd4-wdtl4\" (UID: \"8056aeaf-ce8a-4d1d-a6b6-c0df02057284\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-wdtl4" Jul 07 15:13:36.922038 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.921893 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6bd34c29-c215-4974-98fd-7759cb30f6b8-tmp\") pod \"insights-operator-5c847dcfc4-ntm4v\" (UID: \"6bd34c29-c215-4974-98fd-7759cb30f6b8\") " pod="openshift-insights/insights-operator-5c847dcfc4-ntm4v" Jul 07 15:13:36.922038 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.921923 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zflwn\" (UniqueName: \"kubernetes.io/projected/22d3c83b-a693-4f0a-8ae2-07d83467c620-kube-api-access-zflwn\") pod \"image-registry-689c879f5d-clmr6\" (UID: \"22d3c83b-a693-4f0a-8ae2-07d83467c620\") " pod="openshift-image-registry/image-registry-689c879f5d-clmr6" Jul 07 15:13:36.922038 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.921950 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmhfq\" (UniqueName: \"kubernetes.io/projected/77ec27ef-75d0-454a-9cd5-803a776c4985-kube-api-access-fmhfq\") pod \"router-default-64478b6d7c-vxvp5\" (UID: \"77ec27ef-75d0-454a-9cd5-803a776c4985\") " pod="openshift-ingress/router-default-64478b6d7c-vxvp5" Jul 07 15:13:36.922038 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.921974 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/42fc06ec-88cb-4308-9070-c4defaf37385-metrics-tls\") pod \"dns-default-dgd58\" (UID: \"42fc06ec-88cb-4308-9070-c4defaf37385\") " pod="openshift-dns/dns-default-dgd58" Jul 07 15:13:36.922038 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.922013 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5svb\" (UniqueName: \"kubernetes.io/projected/2a8ea65f-139e-4b35-8a9f-1471da0e27ed-kube-api-access-b5svb\") pod \"cluster-samples-operator-56fcf86bf-66pbb\" (UID: \"2a8ea65f-139e-4b35-8a9f-1471da0e27ed\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-66pbb" Jul 07 15:13:36.922038 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.922036 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3301f6ef-6671-4755-9047-16959836232c-config\") pod \"console-operator-68b757fb67-kb4ml\" (UID: \"3301f6ef-6671-4755-9047-16959836232c\") " pod="openshift-console-operator/console-operator-68b757fb67-kb4ml" Jul 07 15:13:36.922416 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.922062 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6bd34c29-c215-4974-98fd-7759cb30f6b8-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-ntm4v\" (UID: \"6bd34c29-c215-4974-98fd-7759cb30f6b8\") " pod="openshift-insights/insights-operator-5c847dcfc4-ntm4v" Jul 07 15:13:36.922416 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.922091 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k47sh\" (UniqueName: \"kubernetes.io/projected/3301f6ef-6671-4755-9047-16959836232c-kube-api-access-k47sh\") pod \"console-operator-68b757fb67-kb4ml\" (UID: \"3301f6ef-6671-4755-9047-16959836232c\") " pod="openshift-console-operator/console-operator-68b757fb67-kb4ml" Jul 07 15:13:36.922416 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.922129 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vnbg\" (UniqueName: \"kubernetes.io/projected/7ca69696-483c-4a89-8761-ad8245ff5129-kube-api-access-2vnbg\") pod \"network-check-source-989f8b8dc-lb5l7\" (UID: \"7ca69696-483c-4a89-8761-ad8245ff5129\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-lb5l7" Jul 07 15:13:36.922416 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.922156 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4f1ac273-713a-4bb6-94d4-4ec09090a194-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-zh2mx\" (UID: \"4f1ac273-713a-4bb6-94d4-4ec09090a194\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-zh2mx" Jul 07 15:13:36.922416 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.922150 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/22d3c83b-a693-4f0a-8ae2-07d83467c620-ca-trust-extracted\") pod \"image-registry-689c879f5d-clmr6\" (UID: \"22d3c83b-a693-4f0a-8ae2-07d83467c620\") " pod="openshift-image-registry/image-registry-689c879f5d-clmr6" Jul 07 15:13:36.922416 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.922185 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/665ed2cb-7aaf-42ab-8731-03e7cea27fc4-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-88b42\" (UID: \"665ed2cb-7aaf-42ab-8731-03e7cea27fc4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-88b42" Jul 07 15:13:36.922416 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.922225 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnrvg\" (UniqueName: \"kubernetes.io/projected/665ed2cb-7aaf-42ab-8731-03e7cea27fc4-kube-api-access-qnrvg\") pod \"kube-storage-version-migrator-operator-846bf668f7-88b42\" (UID: \"665ed2cb-7aaf-42ab-8731-03e7cea27fc4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-88b42" Jul 07 15:13:36.922416 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.922247 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/22d3c83b-a693-4f0a-8ae2-07d83467c620-trusted-ca\") pod \"image-registry-689c879f5d-clmr6\" (UID: \"22d3c83b-a693-4f0a-8ae2-07d83467c620\") " pod="openshift-image-registry/image-registry-689c879f5d-clmr6" Jul 07 15:13:36.922416 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.922287 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8056aeaf-ce8a-4d1d-a6b6-c0df02057284-config\") pod \"service-ca-operator-5d46d9ffd4-wdtl4\" (UID: \"8056aeaf-ce8a-4d1d-a6b6-c0df02057284\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-wdtl4" Jul 07 15:13:36.922416 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.922347 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bvg5\" (UniqueName: \"kubernetes.io/projected/f18cae1a-0479-4be9-a321-e00e056a4786-kube-api-access-4bvg5\") pod \"volume-data-source-validator-5d456fdbcd-9f95m\" (UID: \"f18cae1a-0479-4be9-a321-e00e056a4786\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-9f95m" Jul 07 15:13:36.922416 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.922389 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/22d3c83b-a693-4f0a-8ae2-07d83467c620-image-registry-private-configuration\") pod \"image-registry-689c879f5d-clmr6\" (UID: \"22d3c83b-a693-4f0a-8ae2-07d83467c620\") " pod="openshift-image-registry/image-registry-689c879f5d-clmr6" Jul 07 15:13:36.922416 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.922418 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/6bd34c29-c215-4974-98fd-7759cb30f6b8-snapshots\") pod \"insights-operator-5c847dcfc4-ntm4v\" (UID: \"6bd34c29-c215-4974-98fd-7759cb30f6b8\") " pod="openshift-insights/insights-operator-5c847dcfc4-ntm4v" Jul 07 15:13:36.922910 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.922443 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frgtz\" (UniqueName: \"kubernetes.io/projected/6bd34c29-c215-4974-98fd-7759cb30f6b8-kube-api-access-frgtz\") pod \"insights-operator-5c847dcfc4-ntm4v\" (UID: \"6bd34c29-c215-4974-98fd-7759cb30f6b8\") " pod="openshift-insights/insights-operator-5c847dcfc4-ntm4v" Jul 07 15:13:36.922910 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.922468 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2ggg\" (UniqueName: \"kubernetes.io/projected/4d2b93c0-1bf2-4ca8-913f-6c1e3d2e60ac-kube-api-access-z2ggg\") pod \"ingress-canary-7smq7\" (UID: \"4d2b93c0-1bf2-4ca8-913f-6c1e3d2e60ac\") " pod="openshift-ingress-canary/ingress-canary-7smq7" Jul 07 15:13:36.922910 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.922492 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42fc06ec-88cb-4308-9070-c4defaf37385-tmp-dir\") pod \"dns-default-dgd58\" (UID: \"42fc06ec-88cb-4308-9070-c4defaf37385\") " pod="openshift-dns/dns-default-dgd58" Jul 07 15:13:36.922910 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.922513 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6bd34c29-c215-4974-98fd-7759cb30f6b8-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-ntm4v\" (UID: \"6bd34c29-c215-4974-98fd-7759cb30f6b8\") " pod="openshift-insights/insights-operator-5c847dcfc4-ntm4v" Jul 07 15:13:36.922910 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.922537 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/22d3c83b-a693-4f0a-8ae2-07d83467c620-registry-tls\") pod \"image-registry-689c879f5d-clmr6\" (UID: \"22d3c83b-a693-4f0a-8ae2-07d83467c620\") " pod="openshift-image-registry/image-registry-689c879f5d-clmr6" Jul 07 15:13:36.922910 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.922559 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/22d3c83b-a693-4f0a-8ae2-07d83467c620-registry-certificates\") pod \"image-registry-689c879f5d-clmr6\" (UID: \"22d3c83b-a693-4f0a-8ae2-07d83467c620\") " pod="openshift-image-registry/image-registry-689c879f5d-clmr6" Jul 07 15:13:36.922910 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:36.922640 2549 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 15:13:36.922910 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:36.922658 2549 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-689c879f5d-clmr6: secret "image-registry-tls" not found Jul 07 15:13:36.922910 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.922676 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4d2b93c0-1bf2-4ca8-913f-6c1e3d2e60ac-cert\") pod \"ingress-canary-7smq7\" (UID: \"4d2b93c0-1bf2-4ca8-913f-6c1e3d2e60ac\") " pod="openshift-ingress-canary/ingress-canary-7smq7" Jul 07 15:13:36.922910 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.922707 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqzbc\" (UniqueName: \"kubernetes.io/projected/8cb2a4ff-480a-47b8-b781-f695be5d74a1-kube-api-access-gqzbc\") pod \"cluster-monitoring-operator-64786d9cbb-ddv8m\" (UID: \"8cb2a4ff-480a-47b8-b781-f695be5d74a1\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ddv8m" Jul 07 15:13:36.922910 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.922781 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/77ec27ef-75d0-454a-9cd5-803a776c4985-stats-auth\") pod \"router-default-64478b6d7c-vxvp5\" (UID: \"77ec27ef-75d0-454a-9cd5-803a776c4985\") " pod="openshift-ingress/router-default-64478b6d7c-vxvp5" Jul 07 15:13:36.922910 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:36.922816 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/22d3c83b-a693-4f0a-8ae2-07d83467c620-registry-tls podName:22d3c83b-a693-4f0a-8ae2-07d83467c620 nodeName:}" failed. No retries permitted until 2026-07-07 15:13:37.422792587 +0000 UTC m=+35.430270033 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/22d3c83b-a693-4f0a-8ae2-07d83467c620-registry-tls") pod "image-registry-689c879f5d-clmr6" (UID: "22d3c83b-a693-4f0a-8ae2-07d83467c620") : secret "image-registry-tls" not found Jul 07 15:13:36.922910 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.922849 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3301f6ef-6671-4755-9047-16959836232c-trusted-ca\") pod \"console-operator-68b757fb67-kb4ml\" (UID: \"3301f6ef-6671-4755-9047-16959836232c\") " pod="openshift-console-operator/console-operator-68b757fb67-kb4ml" Jul 07 15:13:36.923500 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.923017 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/22d3c83b-a693-4f0a-8ae2-07d83467c620-registry-certificates\") pod \"image-registry-689c879f5d-clmr6\" (UID: \"22d3c83b-a693-4f0a-8ae2-07d83467c620\") " pod="openshift-image-registry/image-registry-689c879f5d-clmr6" Jul 07 15:13:36.925769 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.925737 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8056aeaf-ce8a-4d1d-a6b6-c0df02057284-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-wdtl4\" (UID: \"8056aeaf-ce8a-4d1d-a6b6-c0df02057284\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-wdtl4" Jul 07 15:13:36.925984 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.925963 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/22d3c83b-a693-4f0a-8ae2-07d83467c620-installation-pull-secrets\") pod \"image-registry-689c879f5d-clmr6\" (UID: \"22d3c83b-a693-4f0a-8ae2-07d83467c620\") " pod="openshift-image-registry/image-registry-689c879f5d-clmr6" Jul 07 15:13:36.926169 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.926000 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/22d3c83b-a693-4f0a-8ae2-07d83467c620-image-registry-private-configuration\") pod \"image-registry-689c879f5d-clmr6\" (UID: \"22d3c83b-a693-4f0a-8ae2-07d83467c620\") " pod="openshift-image-registry/image-registry-689c879f5d-clmr6" Jul 07 15:13:36.929544 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.929515 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/22d3c83b-a693-4f0a-8ae2-07d83467c620-bound-sa-token\") pod \"image-registry-689c879f5d-clmr6\" (UID: \"22d3c83b-a693-4f0a-8ae2-07d83467c620\") " pod="openshift-image-registry/image-registry-689c879f5d-clmr6" Jul 07 15:13:36.930048 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.930024 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zflwn\" (UniqueName: \"kubernetes.io/projected/22d3c83b-a693-4f0a-8ae2-07d83467c620-kube-api-access-zflwn\") pod \"image-registry-689c879f5d-clmr6\" (UID: \"22d3c83b-a693-4f0a-8ae2-07d83467c620\") " pod="openshift-image-registry/image-registry-689c879f5d-clmr6" Jul 07 15:13:36.930448 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:36.930422 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfgl7\" (UniqueName: \"kubernetes.io/projected/8056aeaf-ce8a-4d1d-a6b6-c0df02057284-kube-api-access-sfgl7\") pod \"service-ca-operator-5d46d9ffd4-wdtl4\" (UID: \"8056aeaf-ce8a-4d1d-a6b6-c0df02057284\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-wdtl4" Jul 07 15:13:37.023332 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.023255 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/8cb2a4ff-480a-47b8-b781-f695be5d74a1-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-ddv8m\" (UID: \"8cb2a4ff-480a-47b8-b781-f695be5d74a1\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ddv8m" Jul 07 15:13:37.023332 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.023306 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6bd34c29-c215-4974-98fd-7759cb30f6b8-tmp\") pod \"insights-operator-5c847dcfc4-ntm4v\" (UID: \"6bd34c29-c215-4974-98fd-7759cb30f6b8\") " pod="openshift-insights/insights-operator-5c847dcfc4-ntm4v" Jul 07 15:13:37.023563 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.023339 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmhfq\" (UniqueName: \"kubernetes.io/projected/77ec27ef-75d0-454a-9cd5-803a776c4985-kube-api-access-fmhfq\") pod \"router-default-64478b6d7c-vxvp5\" (UID: \"77ec27ef-75d0-454a-9cd5-803a776c4985\") " pod="openshift-ingress/router-default-64478b6d7c-vxvp5" Jul 07 15:13:37.023563 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.023363 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/42fc06ec-88cb-4308-9070-c4defaf37385-metrics-tls\") pod \"dns-default-dgd58\" (UID: \"42fc06ec-88cb-4308-9070-c4defaf37385\") " pod="openshift-dns/dns-default-dgd58" Jul 07 15:13:37.023563 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:37.023467 2549 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 15:13:37.023563 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.023496 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5svb\" (UniqueName: \"kubernetes.io/projected/2a8ea65f-139e-4b35-8a9f-1471da0e27ed-kube-api-access-b5svb\") pod \"cluster-samples-operator-56fcf86bf-66pbb\" (UID: \"2a8ea65f-139e-4b35-8a9f-1471da0e27ed\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-66pbb" Jul 07 15:13:37.023563 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:37.023523 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42fc06ec-88cb-4308-9070-c4defaf37385-metrics-tls podName:42fc06ec-88cb-4308-9070-c4defaf37385 nodeName:}" failed. No retries permitted until 2026-07-07 15:13:37.523506342 +0000 UTC m=+35.530983798 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/42fc06ec-88cb-4308-9070-c4defaf37385-metrics-tls") pod "dns-default-dgd58" (UID: "42fc06ec-88cb-4308-9070-c4defaf37385") : secret "dns-default-metrics-tls" not found Jul 07 15:13:37.023563 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.023548 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3301f6ef-6671-4755-9047-16959836232c-config\") pod \"console-operator-68b757fb67-kb4ml\" (UID: \"3301f6ef-6671-4755-9047-16959836232c\") " pod="openshift-console-operator/console-operator-68b757fb67-kb4ml" Jul 07 15:13:37.023873 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.023573 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6bd34c29-c215-4974-98fd-7759cb30f6b8-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-ntm4v\" (UID: \"6bd34c29-c215-4974-98fd-7759cb30f6b8\") " pod="openshift-insights/insights-operator-5c847dcfc4-ntm4v" Jul 07 15:13:37.023873 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.023618 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k47sh\" (UniqueName: \"kubernetes.io/projected/3301f6ef-6671-4755-9047-16959836232c-kube-api-access-k47sh\") pod \"console-operator-68b757fb67-kb4ml\" (UID: \"3301f6ef-6671-4755-9047-16959836232c\") " pod="openshift-console-operator/console-operator-68b757fb67-kb4ml" Jul 07 15:13:37.023873 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.023674 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vnbg\" (UniqueName: \"kubernetes.io/projected/7ca69696-483c-4a89-8761-ad8245ff5129-kube-api-access-2vnbg\") pod \"network-check-source-989f8b8dc-lb5l7\" (UID: \"7ca69696-483c-4a89-8761-ad8245ff5129\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-lb5l7" Jul 07 15:13:37.023873 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.023715 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4f1ac273-713a-4bb6-94d4-4ec09090a194-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-zh2mx\" (UID: \"4f1ac273-713a-4bb6-94d4-4ec09090a194\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-zh2mx" Jul 07 15:13:37.023873 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.023742 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/665ed2cb-7aaf-42ab-8731-03e7cea27fc4-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-88b42\" (UID: \"665ed2cb-7aaf-42ab-8731-03e7cea27fc4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-88b42" Jul 07 15:13:37.023873 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.023770 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnrvg\" (UniqueName: \"kubernetes.io/projected/665ed2cb-7aaf-42ab-8731-03e7cea27fc4-kube-api-access-qnrvg\") pod \"kube-storage-version-migrator-operator-846bf668f7-88b42\" (UID: \"665ed2cb-7aaf-42ab-8731-03e7cea27fc4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-88b42" Jul 07 15:13:37.023873 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.023777 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6bd34c29-c215-4974-98fd-7759cb30f6b8-tmp\") pod \"insights-operator-5c847dcfc4-ntm4v\" (UID: \"6bd34c29-c215-4974-98fd-7759cb30f6b8\") " pod="openshift-insights/insights-operator-5c847dcfc4-ntm4v" Jul 07 15:13:37.023873 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.023795 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bvg5\" (UniqueName: \"kubernetes.io/projected/f18cae1a-0479-4be9-a321-e00e056a4786-kube-api-access-4bvg5\") pod \"volume-data-source-validator-5d456fdbcd-9f95m\" (UID: \"f18cae1a-0479-4be9-a321-e00e056a4786\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-9f95m" Jul 07 15:13:37.023873 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.023835 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/6bd34c29-c215-4974-98fd-7759cb30f6b8-snapshots\") pod \"insights-operator-5c847dcfc4-ntm4v\" (UID: \"6bd34c29-c215-4974-98fd-7759cb30f6b8\") " pod="openshift-insights/insights-operator-5c847dcfc4-ntm4v" Jul 07 15:13:37.023873 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.023872 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frgtz\" (UniqueName: \"kubernetes.io/projected/6bd34c29-c215-4974-98fd-7759cb30f6b8-kube-api-access-frgtz\") pod \"insights-operator-5c847dcfc4-ntm4v\" (UID: \"6bd34c29-c215-4974-98fd-7759cb30f6b8\") " pod="openshift-insights/insights-operator-5c847dcfc4-ntm4v" Jul 07 15:13:37.024307 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.023902 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2ggg\" (UniqueName: \"kubernetes.io/projected/4d2b93c0-1bf2-4ca8-913f-6c1e3d2e60ac-kube-api-access-z2ggg\") pod \"ingress-canary-7smq7\" (UID: \"4d2b93c0-1bf2-4ca8-913f-6c1e3d2e60ac\") " pod="openshift-ingress-canary/ingress-canary-7smq7" Jul 07 15:13:37.024307 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.023926 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42fc06ec-88cb-4308-9070-c4defaf37385-tmp-dir\") pod \"dns-default-dgd58\" (UID: \"42fc06ec-88cb-4308-9070-c4defaf37385\") " pod="openshift-dns/dns-default-dgd58" Jul 07 15:13:37.024307 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.023951 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6bd34c29-c215-4974-98fd-7759cb30f6b8-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-ntm4v\" (UID: \"6bd34c29-c215-4974-98fd-7759cb30f6b8\") " pod="openshift-insights/insights-operator-5c847dcfc4-ntm4v" Jul 07 15:13:37.024307 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.023988 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4d2b93c0-1bf2-4ca8-913f-6c1e3d2e60ac-cert\") pod \"ingress-canary-7smq7\" (UID: \"4d2b93c0-1bf2-4ca8-913f-6c1e3d2e60ac\") " pod="openshift-ingress-canary/ingress-canary-7smq7" Jul 07 15:13:37.024307 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.024012 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqzbc\" (UniqueName: \"kubernetes.io/projected/8cb2a4ff-480a-47b8-b781-f695be5d74a1-kube-api-access-gqzbc\") pod \"cluster-monitoring-operator-64786d9cbb-ddv8m\" (UID: \"8cb2a4ff-480a-47b8-b781-f695be5d74a1\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ddv8m" Jul 07 15:13:37.024307 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.024057 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/77ec27ef-75d0-454a-9cd5-803a776c4985-stats-auth\") pod \"router-default-64478b6d7c-vxvp5\" (UID: \"77ec27ef-75d0-454a-9cd5-803a776c4985\") " pod="openshift-ingress/router-default-64478b6d7c-vxvp5" Jul 07 15:13:37.024307 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.024082 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3301f6ef-6671-4755-9047-16959836232c-trusted-ca\") pod \"console-operator-68b757fb67-kb4ml\" (UID: \"3301f6ef-6671-4755-9047-16959836232c\") " pod="openshift-console-operator/console-operator-68b757fb67-kb4ml" Jul 07 15:13:37.024307 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.024118 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2a8ea65f-139e-4b35-8a9f-1471da0e27ed-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-66pbb\" (UID: \"2a8ea65f-139e-4b35-8a9f-1471da0e27ed\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-66pbb" Jul 07 15:13:37.024307 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.024149 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/8cb2a4ff-480a-47b8-b781-f695be5d74a1-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-ddv8m\" (UID: \"8cb2a4ff-480a-47b8-b781-f695be5d74a1\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ddv8m" Jul 07 15:13:37.024307 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.024161 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77ec27ef-75d0-454a-9cd5-803a776c4985-metrics-certs\") pod \"router-default-64478b6d7c-vxvp5\" (UID: \"77ec27ef-75d0-454a-9cd5-803a776c4985\") " pod="openshift-ingress/router-default-64478b6d7c-vxvp5" Jul 07 15:13:37.024307 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.024188 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zmzw\" (UniqueName: \"kubernetes.io/projected/42fc06ec-88cb-4308-9070-c4defaf37385-kube-api-access-6zmzw\") pod \"dns-default-dgd58\" (UID: \"42fc06ec-88cb-4308-9070-c4defaf37385\") " pod="openshift-dns/dns-default-dgd58" Jul 07 15:13:37.024307 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.024228 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8cb2a4ff-480a-47b8-b781-f695be5d74a1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-ddv8m\" (UID: \"8cb2a4ff-480a-47b8-b781-f695be5d74a1\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ddv8m" Jul 07 15:13:37.024307 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.024266 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3301f6ef-6671-4755-9047-16959836232c-serving-cert\") pod \"console-operator-68b757fb67-kb4ml\" (UID: \"3301f6ef-6671-4755-9047-16959836232c\") " pod="openshift-console-operator/console-operator-68b757fb67-kb4ml" Jul 07 15:13:37.024307 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.024296 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4f1ac273-713a-4bb6-94d4-4ec09090a194-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-zh2mx\" (UID: \"4f1ac273-713a-4bb6-94d4-4ec09090a194\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-zh2mx" Jul 07 15:13:37.024933 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.024322 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/665ed2cb-7aaf-42ab-8731-03e7cea27fc4-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-88b42\" (UID: \"665ed2cb-7aaf-42ab-8731-03e7cea27fc4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-88b42" Jul 07 15:13:37.024933 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.024359 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/77ec27ef-75d0-454a-9cd5-803a776c4985-default-certificate\") pod \"router-default-64478b6d7c-vxvp5\" (UID: \"77ec27ef-75d0-454a-9cd5-803a776c4985\") " pod="openshift-ingress/router-default-64478b6d7c-vxvp5" Jul 07 15:13:37.024933 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.024384 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6bd34c29-c215-4974-98fd-7759cb30f6b8-serving-cert\") pod \"insights-operator-5c847dcfc4-ntm4v\" (UID: \"6bd34c29-c215-4974-98fd-7759cb30f6b8\") " pod="openshift-insights/insights-operator-5c847dcfc4-ntm4v" Jul 07 15:13:37.024933 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.024401 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3301f6ef-6671-4755-9047-16959836232c-config\") pod \"console-operator-68b757fb67-kb4ml\" (UID: \"3301f6ef-6671-4755-9047-16959836232c\") " pod="openshift-console-operator/console-operator-68b757fb67-kb4ml" Jul 07 15:13:37.024933 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.024412 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77ec27ef-75d0-454a-9cd5-803a776c4985-service-ca-bundle\") pod \"router-default-64478b6d7c-vxvp5\" (UID: \"77ec27ef-75d0-454a-9cd5-803a776c4985\") " pod="openshift-ingress/router-default-64478b6d7c-vxvp5" Jul 07 15:13:37.024933 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.024440 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/42fc06ec-88cb-4308-9070-c4defaf37385-config-volume\") pod \"dns-default-dgd58\" (UID: \"42fc06ec-88cb-4308-9070-c4defaf37385\") " pod="openshift-dns/dns-default-dgd58" Jul 07 15:13:37.024933 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.024493 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/6bd34c29-c215-4974-98fd-7759cb30f6b8-snapshots\") pod \"insights-operator-5c847dcfc4-ntm4v\" (UID: \"6bd34c29-c215-4974-98fd-7759cb30f6b8\") " pod="openshift-insights/insights-operator-5c847dcfc4-ntm4v" Jul 07 15:13:37.024933 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:37.024563 2549 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 07 15:13:37.024933 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:37.024637 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f1ac273-713a-4bb6-94d4-4ec09090a194-networking-console-plugin-cert podName:4f1ac273-713a-4bb6-94d4-4ec09090a194 nodeName:}" failed. No retries permitted until 2026-07-07 15:13:37.524622642 +0000 UTC m=+35.532100076 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4f1ac273-713a-4bb6-94d4-4ec09090a194-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-zh2mx" (UID: "4f1ac273-713a-4bb6-94d4-4ec09090a194") : secret "networking-console-plugin-cert" not found Jul 07 15:13:37.024933 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.024825 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6bd34c29-c215-4974-98fd-7759cb30f6b8-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-ntm4v\" (UID: \"6bd34c29-c215-4974-98fd-7759cb30f6b8\") " pod="openshift-insights/insights-operator-5c847dcfc4-ntm4v" Jul 07 15:13:37.029455 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.025750 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42fc06ec-88cb-4308-9070-c4defaf37385-tmp-dir\") pod \"dns-default-dgd58\" (UID: \"42fc06ec-88cb-4308-9070-c4defaf37385\") " pod="openshift-dns/dns-default-dgd58" Jul 07 15:13:37.029455 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.025930 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6bd34c29-c215-4974-98fd-7759cb30f6b8-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-ntm4v\" (UID: \"6bd34c29-c215-4974-98fd-7759cb30f6b8\") " pod="openshift-insights/insights-operator-5c847dcfc4-ntm4v" Jul 07 15:13:37.029455 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:37.026217 2549 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 15:13:37.029455 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:37.026292 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cb2a4ff-480a-47b8-b781-f695be5d74a1-cluster-monitoring-operator-tls podName:8cb2a4ff-480a-47b8-b781-f695be5d74a1 nodeName:}" failed. No retries permitted until 2026-07-07 15:13:37.526275031 +0000 UTC m=+35.533752478 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/8cb2a4ff-480a-47b8-b781-f695be5d74a1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-ddv8m" (UID: "8cb2a4ff-480a-47b8-b781-f695be5d74a1") : secret "cluster-monitoring-operator-tls" not found Jul 07 15:13:37.029455 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.026971 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4f1ac273-713a-4bb6-94d4-4ec09090a194-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-zh2mx\" (UID: \"4f1ac273-713a-4bb6-94d4-4ec09090a194\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-zh2mx" Jul 07 15:13:37.029455 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.027539 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/42fc06ec-88cb-4308-9070-c4defaf37385-config-volume\") pod \"dns-default-dgd58\" (UID: \"42fc06ec-88cb-4308-9070-c4defaf37385\") " pod="openshift-dns/dns-default-dgd58" Jul 07 15:13:37.029455 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:37.027777 2549 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 15:13:37.029455 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:37.027859 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/77ec27ef-75d0-454a-9cd5-803a776c4985-service-ca-bundle podName:77ec27ef-75d0-454a-9cd5-803a776c4985 nodeName:}" failed. No retries permitted until 2026-07-07 15:13:37.527841341 +0000 UTC m=+35.535318787 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/77ec27ef-75d0-454a-9cd5-803a776c4985-service-ca-bundle") pod "router-default-64478b6d7c-vxvp5" (UID: "77ec27ef-75d0-454a-9cd5-803a776c4985") : configmap references non-existent config key: service-ca.crt Jul 07 15:13:37.029455 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:37.027890 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77ec27ef-75d0-454a-9cd5-803a776c4985-metrics-certs podName:77ec27ef-75d0-454a-9cd5-803a776c4985 nodeName:}" failed. No retries permitted until 2026-07-07 15:13:37.527870833 +0000 UTC m=+35.535348262 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/77ec27ef-75d0-454a-9cd5-803a776c4985-metrics-certs") pod "router-default-64478b6d7c-vxvp5" (UID: "77ec27ef-75d0-454a-9cd5-803a776c4985") : secret "router-metrics-certs-default" not found Jul 07 15:13:37.029455 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.028146 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/665ed2cb-7aaf-42ab-8731-03e7cea27fc4-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-88b42\" (UID: \"665ed2cb-7aaf-42ab-8731-03e7cea27fc4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-88b42" Jul 07 15:13:37.029455 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:37.028271 2549 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 15:13:37.029455 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:37.028349 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a8ea65f-139e-4b35-8a9f-1471da0e27ed-samples-operator-tls podName:2a8ea65f-139e-4b35-8a9f-1471da0e27ed nodeName:}" failed. No retries permitted until 2026-07-07 15:13:37.528316164 +0000 UTC m=+35.535793594 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2a8ea65f-139e-4b35-8a9f-1471da0e27ed-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-66pbb" (UID: "2a8ea65f-139e-4b35-8a9f-1471da0e27ed") : secret "samples-operator-tls" not found Jul 07 15:13:37.029455 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.029081 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/665ed2cb-7aaf-42ab-8731-03e7cea27fc4-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-88b42\" (UID: \"665ed2cb-7aaf-42ab-8731-03e7cea27fc4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-88b42" Jul 07 15:13:37.029455 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.029392 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6bd34c29-c215-4974-98fd-7759cb30f6b8-serving-cert\") pod \"insights-operator-5c847dcfc4-ntm4v\" (UID: \"6bd34c29-c215-4974-98fd-7759cb30f6b8\") " pod="openshift-insights/insights-operator-5c847dcfc4-ntm4v" Jul 07 15:13:37.030241 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.029670 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4d2b93c0-1bf2-4ca8-913f-6c1e3d2e60ac-cert\") pod \"ingress-canary-7smq7\" (UID: \"4d2b93c0-1bf2-4ca8-913f-6c1e3d2e60ac\") " pod="openshift-ingress-canary/ingress-canary-7smq7" Jul 07 15:13:37.033520 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.030346 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3301f6ef-6671-4755-9047-16959836232c-trusted-ca\") pod \"console-operator-68b757fb67-kb4ml\" (UID: \"3301f6ef-6671-4755-9047-16959836232c\") " pod="openshift-console-operator/console-operator-68b757fb67-kb4ml" Jul 07 15:13:37.033520 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.030367 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/77ec27ef-75d0-454a-9cd5-803a776c4985-stats-auth\") pod \"router-default-64478b6d7c-vxvp5\" (UID: \"77ec27ef-75d0-454a-9cd5-803a776c4985\") " pod="openshift-ingress/router-default-64478b6d7c-vxvp5" Jul 07 15:13:37.033520 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.033173 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/77ec27ef-75d0-454a-9cd5-803a776c4985-default-certificate\") pod \"router-default-64478b6d7c-vxvp5\" (UID: \"77ec27ef-75d0-454a-9cd5-803a776c4985\") " pod="openshift-ingress/router-default-64478b6d7c-vxvp5" Jul 07 15:13:37.037005 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.036943 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmhfq\" (UniqueName: \"kubernetes.io/projected/77ec27ef-75d0-454a-9cd5-803a776c4985-kube-api-access-fmhfq\") pod \"router-default-64478b6d7c-vxvp5\" (UID: \"77ec27ef-75d0-454a-9cd5-803a776c4985\") " pod="openshift-ingress/router-default-64478b6d7c-vxvp5" Jul 07 15:13:37.038179 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.037455 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5svb\" (UniqueName: \"kubernetes.io/projected/2a8ea65f-139e-4b35-8a9f-1471da0e27ed-kube-api-access-b5svb\") pod \"cluster-samples-operator-56fcf86bf-66pbb\" (UID: \"2a8ea65f-139e-4b35-8a9f-1471da0e27ed\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-66pbb" Jul 07 15:13:37.038179 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.037929 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3301f6ef-6671-4755-9047-16959836232c-serving-cert\") pod \"console-operator-68b757fb67-kb4ml\" (UID: \"3301f6ef-6671-4755-9047-16959836232c\") " pod="openshift-console-operator/console-operator-68b757fb67-kb4ml" Jul 07 15:13:37.038989 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.038799 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zmzw\" (UniqueName: \"kubernetes.io/projected/42fc06ec-88cb-4308-9070-c4defaf37385-kube-api-access-6zmzw\") pod \"dns-default-dgd58\" (UID: \"42fc06ec-88cb-4308-9070-c4defaf37385\") " pod="openshift-dns/dns-default-dgd58" Jul 07 15:13:37.038989 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.038949 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2ggg\" (UniqueName: \"kubernetes.io/projected/4d2b93c0-1bf2-4ca8-913f-6c1e3d2e60ac-kube-api-access-z2ggg\") pod \"ingress-canary-7smq7\" (UID: \"4d2b93c0-1bf2-4ca8-913f-6c1e3d2e60ac\") " pod="openshift-ingress-canary/ingress-canary-7smq7" Jul 07 15:13:37.039950 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.039909 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vnbg\" (UniqueName: \"kubernetes.io/projected/7ca69696-483c-4a89-8761-ad8245ff5129-kube-api-access-2vnbg\") pod \"network-check-source-989f8b8dc-lb5l7\" (UID: \"7ca69696-483c-4a89-8761-ad8245ff5129\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-lb5l7" Jul 07 15:13:37.040991 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.040267 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k47sh\" (UniqueName: \"kubernetes.io/projected/3301f6ef-6671-4755-9047-16959836232c-kube-api-access-k47sh\") pod \"console-operator-68b757fb67-kb4ml\" (UID: \"3301f6ef-6671-4755-9047-16959836232c\") " pod="openshift-console-operator/console-operator-68b757fb67-kb4ml" Jul 07 15:13:37.040991 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.040918 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqzbc\" (UniqueName: \"kubernetes.io/projected/8cb2a4ff-480a-47b8-b781-f695be5d74a1-kube-api-access-gqzbc\") pod \"cluster-monitoring-operator-64786d9cbb-ddv8m\" (UID: \"8cb2a4ff-480a-47b8-b781-f695be5d74a1\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ddv8m" Jul 07 15:13:37.040991 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.040981 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnrvg\" (UniqueName: \"kubernetes.io/projected/665ed2cb-7aaf-42ab-8731-03e7cea27fc4-kube-api-access-qnrvg\") pod \"kube-storage-version-migrator-operator-846bf668f7-88b42\" (UID: \"665ed2cb-7aaf-42ab-8731-03e7cea27fc4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-88b42" Jul 07 15:13:37.041197 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.041034 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bvg5\" (UniqueName: \"kubernetes.io/projected/f18cae1a-0479-4be9-a321-e00e056a4786-kube-api-access-4bvg5\") pod \"volume-data-source-validator-5d456fdbcd-9f95m\" (UID: \"f18cae1a-0479-4be9-a321-e00e056a4786\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-9f95m" Jul 07 15:13:37.041252 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.041228 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-frgtz\" (UniqueName: \"kubernetes.io/projected/6bd34c29-c215-4974-98fd-7759cb30f6b8-kube-api-access-frgtz\") pod \"insights-operator-5c847dcfc4-ntm4v\" (UID: \"6bd34c29-c215-4974-98fd-7759cb30f6b8\") " pod="openshift-insights/insights-operator-5c847dcfc4-ntm4v" Jul 07 15:13:37.058694 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.058667 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-wdtl4" Jul 07 15:13:37.092672 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.092621 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-88b42" Jul 07 15:13:37.099947 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.099756 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-ntm4v" Jul 07 15:13:37.117101 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.116896 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-9f95m" Jul 07 15:13:37.137878 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.137849 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-kb4ml" Jul 07 15:13:37.146544 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.146334 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-lb5l7" Jul 07 15:13:37.160023 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.159798 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7smq7" Jul 07 15:13:37.248701 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.248550 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-wdtl4"] Jul 07 15:13:37.312202 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.309693 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-ntm4v"] Jul 07 15:13:37.320093 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:37.320053 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bd34c29_c215_4974_98fd_7759cb30f6b8.slice/crio-830e146ff6ee74ccf349a105b52a07e1a9ba99c3faa08b449f797172752109ed WatchSource:0}: Error finding container 830e146ff6ee74ccf349a105b52a07e1a9ba99c3faa08b449f797172752109ed: Status 404 returned error can't find the container with id 830e146ff6ee74ccf349a105b52a07e1a9ba99c3faa08b449f797172752109ed Jul 07 15:13:37.329187 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.329161 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-88b42"] Jul 07 15:13:37.335454 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:37.335419 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod665ed2cb_7aaf_42ab_8731_03e7cea27fc4.slice/crio-2134e058483408146306be0cbb49b0b97129e568ee14bccbe89577e8163ef9e5 WatchSource:0}: Error finding container 2134e058483408146306be0cbb49b0b97129e568ee14bccbe89577e8163ef9e5: Status 404 returned error can't find the container with id 2134e058483408146306be0cbb49b0b97129e568ee14bccbe89577e8163ef9e5 Jul 07 15:13:37.352123 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.351701 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-9f95m"] Jul 07 15:13:37.407321 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.407253 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7smq7"] Jul 07 15:13:37.408552 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.408528 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-kb4ml"] Jul 07 15:13:37.409643 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.409576 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-lb5l7"] Jul 07 15:13:37.410272 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:37.410250 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d2b93c0_1bf2_4ca8_913f_6c1e3d2e60ac.slice/crio-4f37da29709977ef6d7a70c5703bb0cadc3c1798b7f3b849969ad2eec2dbb2a2 WatchSource:0}: Error finding container 4f37da29709977ef6d7a70c5703bb0cadc3c1798b7f3b849969ad2eec2dbb2a2: Status 404 returned error can't find the container with id 4f37da29709977ef6d7a70c5703bb0cadc3c1798b7f3b849969ad2eec2dbb2a2 Jul 07 15:13:37.410796 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:37.410777 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3301f6ef_6671_4755_9047_16959836232c.slice/crio-558d0d4e244f161fc08084152e6878242209edeb7a52ee6169620465182f1ed6 WatchSource:0}: Error finding container 558d0d4e244f161fc08084152e6878242209edeb7a52ee6169620465182f1ed6: Status 404 returned error can't find the container with id 558d0d4e244f161fc08084152e6878242209edeb7a52ee6169620465182f1ed6 Jul 07 15:13:37.411498 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:13:37.411482 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ca69696_483c_4a89_8761_ad8245ff5129.slice/crio-c91dd877a70aead92471a5c518fea579807702f0a2faac3c0aed70a43ff9f3a3 WatchSource:0}: Error finding container c91dd877a70aead92471a5c518fea579807702f0a2faac3c0aed70a43ff9f3a3: Status 404 returned error can't find the container with id c91dd877a70aead92471a5c518fea579807702f0a2faac3c0aed70a43ff9f3a3 Jul 07 15:13:37.428447 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.428421 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/22d3c83b-a693-4f0a-8ae2-07d83467c620-registry-tls\") pod \"image-registry-689c879f5d-clmr6\" (UID: \"22d3c83b-a693-4f0a-8ae2-07d83467c620\") " pod="openshift-image-registry/image-registry-689c879f5d-clmr6" Jul 07 15:13:37.428569 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:37.428553 2549 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 15:13:37.428639 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:37.428572 2549 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-689c879f5d-clmr6: secret "image-registry-tls" not found Jul 07 15:13:37.428639 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:37.428637 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/22d3c83b-a693-4f0a-8ae2-07d83467c620-registry-tls podName:22d3c83b-a693-4f0a-8ae2-07d83467c620 nodeName:}" failed. No retries permitted until 2026-07-07 15:13:38.428621866 +0000 UTC m=+36.436099296 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/22d3c83b-a693-4f0a-8ae2-07d83467c620-registry-tls") pod "image-registry-689c879f5d-clmr6" (UID: "22d3c83b-a693-4f0a-8ae2-07d83467c620") : secret "image-registry-tls" not found Jul 07 15:13:37.528522 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.528428 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqp6p" Jul 07 15:13:37.528522 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.528480 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9dwk4" Jul 07 15:13:37.529010 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.528978 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4f1ac273-713a-4bb6-94d4-4ec09090a194-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-zh2mx\" (UID: \"4f1ac273-713a-4bb6-94d4-4ec09090a194\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-zh2mx" Jul 07 15:13:37.529128 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.529071 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2a8ea65f-139e-4b35-8a9f-1471da0e27ed-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-66pbb\" (UID: \"2a8ea65f-139e-4b35-8a9f-1471da0e27ed\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-66pbb" Jul 07 15:13:37.529128 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.529101 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77ec27ef-75d0-454a-9cd5-803a776c4985-metrics-certs\") pod \"router-default-64478b6d7c-vxvp5\" (UID: \"77ec27ef-75d0-454a-9cd5-803a776c4985\") " pod="openshift-ingress/router-default-64478b6d7c-vxvp5" Jul 07 15:13:37.529128 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:37.529119 2549 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 07 15:13:37.529250 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.529134 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8cb2a4ff-480a-47b8-b781-f695be5d74a1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-ddv8m\" (UID: \"8cb2a4ff-480a-47b8-b781-f695be5d74a1\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ddv8m" Jul 07 15:13:37.529250 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:37.529183 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f1ac273-713a-4bb6-94d4-4ec09090a194-networking-console-plugin-cert podName:4f1ac273-713a-4bb6-94d4-4ec09090a194 nodeName:}" failed. No retries permitted until 2026-07-07 15:13:38.529164617 +0000 UTC m=+36.536642048 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4f1ac273-713a-4bb6-94d4-4ec09090a194-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-zh2mx" (UID: "4f1ac273-713a-4bb6-94d4-4ec09090a194") : secret "networking-console-plugin-cert" not found Jul 07 15:13:37.529250 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:37.529202 2549 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 15:13:37.529250 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:37.529235 2549 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 15:13:37.529250 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:37.529250 2549 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 15:13:37.529450 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:37.529254 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a8ea65f-139e-4b35-8a9f-1471da0e27ed-samples-operator-tls podName:2a8ea65f-139e-4b35-8a9f-1471da0e27ed nodeName:}" failed. No retries permitted until 2026-07-07 15:13:38.529242433 +0000 UTC m=+36.536719877 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2a8ea65f-139e-4b35-8a9f-1471da0e27ed-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-66pbb" (UID: "2a8ea65f-139e-4b35-8a9f-1471da0e27ed") : secret "samples-operator-tls" not found Jul 07 15:13:37.529450 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:37.529304 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77ec27ef-75d0-454a-9cd5-803a776c4985-metrics-certs podName:77ec27ef-75d0-454a-9cd5-803a776c4985 nodeName:}" failed. No retries permitted until 2026-07-07 15:13:38.529289578 +0000 UTC m=+36.536767008 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/77ec27ef-75d0-454a-9cd5-803a776c4985-metrics-certs") pod "router-default-64478b6d7c-vxvp5" (UID: "77ec27ef-75d0-454a-9cd5-803a776c4985") : secret "router-metrics-certs-default" not found Jul 07 15:13:37.529450 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:37.529317 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cb2a4ff-480a-47b8-b781-f695be5d74a1-cluster-monitoring-operator-tls podName:8cb2a4ff-480a-47b8-b781-f695be5d74a1 nodeName:}" failed. No retries permitted until 2026-07-07 15:13:38.529309102 +0000 UTC m=+36.536786555 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/8cb2a4ff-480a-47b8-b781-f695be5d74a1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-ddv8m" (UID: "8cb2a4ff-480a-47b8-b781-f695be5d74a1") : secret "cluster-monitoring-operator-tls" not found Jul 07 15:13:37.529450 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.529336 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77ec27ef-75d0-454a-9cd5-803a776c4985-service-ca-bundle\") pod \"router-default-64478b6d7c-vxvp5\" (UID: \"77ec27ef-75d0-454a-9cd5-803a776c4985\") " pod="openshift-ingress/router-default-64478b6d7c-vxvp5" Jul 07 15:13:37.529450 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.529396 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/42fc06ec-88cb-4308-9070-c4defaf37385-metrics-tls\") pod \"dns-default-dgd58\" (UID: \"42fc06ec-88cb-4308-9070-c4defaf37385\") " pod="openshift-dns/dns-default-dgd58" Jul 07 15:13:37.529738 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:37.529464 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/77ec27ef-75d0-454a-9cd5-803a776c4985-service-ca-bundle podName:77ec27ef-75d0-454a-9cd5-803a776c4985 nodeName:}" failed. No retries permitted until 2026-07-07 15:13:38.529447444 +0000 UTC m=+36.536924893 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/77ec27ef-75d0-454a-9cd5-803a776c4985-service-ca-bundle") pod "router-default-64478b6d7c-vxvp5" (UID: "77ec27ef-75d0-454a-9cd5-803a776c4985") : configmap references non-existent config key: service-ca.crt Jul 07 15:13:37.529738 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:37.529531 2549 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 15:13:37.529738 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:37.529564 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42fc06ec-88cb-4308-9070-c4defaf37385-metrics-tls podName:42fc06ec-88cb-4308-9070-c4defaf37385 nodeName:}" failed. No retries permitted until 2026-07-07 15:13:38.529554682 +0000 UTC m=+36.537032136 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/42fc06ec-88cb-4308-9070-c4defaf37385-metrics-tls") pod "dns-default-dgd58" (UID: "42fc06ec-88cb-4308-9070-c4defaf37385") : secret "dns-default-metrics-tls" not found Jul 07 15:13:37.530964 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.530830 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 15:13:37.530964 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.530854 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-lhgw4\"" Jul 07 15:13:37.530964 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.530879 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-5bbhh\"" Jul 07 15:13:37.691497 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.691458 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7smq7" event={"ID":"4d2b93c0-1bf2-4ca8-913f-6c1e3d2e60ac","Type":"ContainerStarted","Data":"4f37da29709977ef6d7a70c5703bb0cadc3c1798b7f3b849969ad2eec2dbb2a2"} Jul 07 15:13:37.692710 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.692684 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-ntm4v" event={"ID":"6bd34c29-c215-4974-98fd-7759cb30f6b8","Type":"ContainerStarted","Data":"830e146ff6ee74ccf349a105b52a07e1a9ba99c3faa08b449f797172752109ed"} Jul 07 15:13:37.693907 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.693875 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-88b42" event={"ID":"665ed2cb-7aaf-42ab-8731-03e7cea27fc4","Type":"ContainerStarted","Data":"2134e058483408146306be0cbb49b0b97129e568ee14bccbe89577e8163ef9e5"} Jul 07 15:13:37.695055 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.695022 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-lb5l7" event={"ID":"7ca69696-483c-4a89-8761-ad8245ff5129","Type":"ContainerStarted","Data":"c91dd877a70aead92471a5c518fea579807702f0a2faac3c0aed70a43ff9f3a3"} Jul 07 15:13:37.696326 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.696298 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-kb4ml" event={"ID":"3301f6ef-6671-4755-9047-16959836232c","Type":"ContainerStarted","Data":"558d0d4e244f161fc08084152e6878242209edeb7a52ee6169620465182f1ed6"} Jul 07 15:13:37.697598 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.697564 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-9f95m" event={"ID":"f18cae1a-0479-4be9-a321-e00e056a4786","Type":"ContainerStarted","Data":"23b5ebe1b4e5d0cfb7c2b5fe64aa0966e47450166d49080d35f4c673a74680be"} Jul 07 15:13:37.698667 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:37.698643 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-wdtl4" event={"ID":"8056aeaf-ce8a-4d1d-a6b6-c0df02057284","Type":"ContainerStarted","Data":"4f49aff37df4294156f3d2df9962522b9998f122e0adb5a4fe402f4c0b4ea115"} Jul 07 15:13:38.439732 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:38.439691 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/22d3c83b-a693-4f0a-8ae2-07d83467c620-registry-tls\") pod \"image-registry-689c879f5d-clmr6\" (UID: \"22d3c83b-a693-4f0a-8ae2-07d83467c620\") " pod="openshift-image-registry/image-registry-689c879f5d-clmr6" Jul 07 15:13:38.439930 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:38.439855 2549 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 15:13:38.439930 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:38.439879 2549 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-689c879f5d-clmr6: secret "image-registry-tls" not found Jul 07 15:13:38.440034 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:38.439939 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/22d3c83b-a693-4f0a-8ae2-07d83467c620-registry-tls podName:22d3c83b-a693-4f0a-8ae2-07d83467c620 nodeName:}" failed. No retries permitted until 2026-07-07 15:13:40.439923394 +0000 UTC m=+38.447400826 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/22d3c83b-a693-4f0a-8ae2-07d83467c620-registry-tls") pod "image-registry-689c879f5d-clmr6" (UID: "22d3c83b-a693-4f0a-8ae2-07d83467c620") : secret "image-registry-tls" not found Jul 07 15:13:38.532422 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:38.532392 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xt54w" Jul 07 15:13:38.534911 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:38.534881 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 15:13:38.540305 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:38.540283 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/42fc06ec-88cb-4308-9070-c4defaf37385-metrics-tls\") pod \"dns-default-dgd58\" (UID: \"42fc06ec-88cb-4308-9070-c4defaf37385\") " pod="openshift-dns/dns-default-dgd58" Jul 07 15:13:38.540405 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:38.540362 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4f1ac273-713a-4bb6-94d4-4ec09090a194-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-zh2mx\" (UID: \"4f1ac273-713a-4bb6-94d4-4ec09090a194\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-zh2mx" Jul 07 15:13:38.540461 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:38.540417 2549 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 15:13:38.540461 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:38.540427 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2a8ea65f-139e-4b35-8a9f-1471da0e27ed-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-66pbb\" (UID: \"2a8ea65f-139e-4b35-8a9f-1471da0e27ed\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-66pbb" Jul 07 15:13:38.540573 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:38.540456 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77ec27ef-75d0-454a-9cd5-803a776c4985-metrics-certs\") pod \"router-default-64478b6d7c-vxvp5\" (UID: \"77ec27ef-75d0-454a-9cd5-803a776c4985\") " pod="openshift-ingress/router-default-64478b6d7c-vxvp5" Jul 07 15:13:38.540573 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:38.540481 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42fc06ec-88cb-4308-9070-c4defaf37385-metrics-tls podName:42fc06ec-88cb-4308-9070-c4defaf37385 nodeName:}" failed. No retries permitted until 2026-07-07 15:13:40.540461202 +0000 UTC m=+38.547938648 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/42fc06ec-88cb-4308-9070-c4defaf37385-metrics-tls") pod "dns-default-dgd58" (UID: "42fc06ec-88cb-4308-9070-c4defaf37385") : secret "dns-default-metrics-tls" not found Jul 07 15:13:38.540573 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:38.540564 2549 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 15:13:38.540737 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:38.540601 2549 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 07 15:13:38.540737 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:38.540622 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a8ea65f-139e-4b35-8a9f-1471da0e27ed-samples-operator-tls podName:2a8ea65f-139e-4b35-8a9f-1471da0e27ed nodeName:}" failed. No retries permitted until 2026-07-07 15:13:40.540609178 +0000 UTC m=+38.548086610 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2a8ea65f-139e-4b35-8a9f-1471da0e27ed-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-66pbb" (UID: "2a8ea65f-139e-4b35-8a9f-1471da0e27ed") : secret "samples-operator-tls" not found Jul 07 15:13:38.540737 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:38.540640 2549 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 15:13:38.540737 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:38.540655 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f1ac273-713a-4bb6-94d4-4ec09090a194-networking-console-plugin-cert podName:4f1ac273-713a-4bb6-94d4-4ec09090a194 nodeName:}" failed. No retries permitted until 2026-07-07 15:13:40.540639851 +0000 UTC m=+38.548117294 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4f1ac273-713a-4bb6-94d4-4ec09090a194-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-zh2mx" (UID: "4f1ac273-713a-4bb6-94d4-4ec09090a194") : secret "networking-console-plugin-cert" not found Jul 07 15:13:38.540737 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:38.540683 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77ec27ef-75d0-454a-9cd5-803a776c4985-metrics-certs podName:77ec27ef-75d0-454a-9cd5-803a776c4985 nodeName:}" failed. No retries permitted until 2026-07-07 15:13:40.540669211 +0000 UTC m=+38.548146651 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/77ec27ef-75d0-454a-9cd5-803a776c4985-metrics-certs") pod "router-default-64478b6d7c-vxvp5" (UID: "77ec27ef-75d0-454a-9cd5-803a776c4985") : secret "router-metrics-certs-default" not found Jul 07 15:13:38.540737 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:38.540706 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8cb2a4ff-480a-47b8-b781-f695be5d74a1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-ddv8m\" (UID: \"8cb2a4ff-480a-47b8-b781-f695be5d74a1\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ddv8m" Jul 07 15:13:38.541064 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:38.540764 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77ec27ef-75d0-454a-9cd5-803a776c4985-service-ca-bundle\") pod \"router-default-64478b6d7c-vxvp5\" (UID: \"77ec27ef-75d0-454a-9cd5-803a776c4985\") " pod="openshift-ingress/router-default-64478b6d7c-vxvp5" Jul 07 15:13:38.541064 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:38.540831 2549 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 15:13:38.541064 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:38.540881 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cb2a4ff-480a-47b8-b781-f695be5d74a1-cluster-monitoring-operator-tls podName:8cb2a4ff-480a-47b8-b781-f695be5d74a1 nodeName:}" failed. No retries permitted until 2026-07-07 15:13:40.540864662 +0000 UTC m=+38.548342098 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/8cb2a4ff-480a-47b8-b781-f695be5d74a1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-ddv8m" (UID: "8cb2a4ff-480a-47b8-b781-f695be5d74a1") : secret "cluster-monitoring-operator-tls" not found Jul 07 15:13:38.541064 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:38.540895 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/77ec27ef-75d0-454a-9cd5-803a776c4985-service-ca-bundle podName:77ec27ef-75d0-454a-9cd5-803a776c4985 nodeName:}" failed. No retries permitted until 2026-07-07 15:13:40.540886938 +0000 UTC m=+38.548364369 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/77ec27ef-75d0-454a-9cd5-803a776c4985-service-ca-bundle") pod "router-default-64478b6d7c-vxvp5" (UID: "77ec27ef-75d0-454a-9cd5-803a776c4985") : configmap references non-existent config key: service-ca.crt Jul 07 15:13:39.711525 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:39.711479 2549 generic.go:356] "Generic (PLEG): container finished" podID="c59d2067-5399-4a3d-9f34-be99e63f7d2b" containerID="11f046872d75be259894766a545c9148454bdad68b37b581f99962480d0f4837" exitCode=0 Jul 07 15:13:39.711996 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:39.711542 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qkznn" event={"ID":"c59d2067-5399-4a3d-9f34-be99e63f7d2b","Type":"ContainerDied","Data":"11f046872d75be259894766a545c9148454bdad68b37b581f99962480d0f4837"} Jul 07 15:13:40.464502 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:40.463662 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/22d3c83b-a693-4f0a-8ae2-07d83467c620-registry-tls\") pod \"image-registry-689c879f5d-clmr6\" (UID: \"22d3c83b-a693-4f0a-8ae2-07d83467c620\") " pod="openshift-image-registry/image-registry-689c879f5d-clmr6" Jul 07 15:13:40.464502 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:40.463814 2549 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 15:13:40.464502 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:40.464071 2549 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-689c879f5d-clmr6: secret "image-registry-tls" not found Jul 07 15:13:40.464502 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:40.464136 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/22d3c83b-a693-4f0a-8ae2-07d83467c620-registry-tls podName:22d3c83b-a693-4f0a-8ae2-07d83467c620 nodeName:}" failed. No retries permitted until 2026-07-07 15:13:44.464116856 +0000 UTC m=+42.471594298 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/22d3c83b-a693-4f0a-8ae2-07d83467c620-registry-tls") pod "image-registry-689c879f5d-clmr6" (UID: "22d3c83b-a693-4f0a-8ae2-07d83467c620") : secret "image-registry-tls" not found Jul 07 15:13:40.565681 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:40.564977 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/42fc06ec-88cb-4308-9070-c4defaf37385-metrics-tls\") pod \"dns-default-dgd58\" (UID: \"42fc06ec-88cb-4308-9070-c4defaf37385\") " pod="openshift-dns/dns-default-dgd58" Jul 07 15:13:40.565681 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:40.565052 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4f1ac273-713a-4bb6-94d4-4ec09090a194-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-zh2mx\" (UID: \"4f1ac273-713a-4bb6-94d4-4ec09090a194\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-zh2mx" Jul 07 15:13:40.565681 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:40.565124 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2a8ea65f-139e-4b35-8a9f-1471da0e27ed-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-66pbb\" (UID: \"2a8ea65f-139e-4b35-8a9f-1471da0e27ed\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-66pbb" Jul 07 15:13:40.565681 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:40.565149 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77ec27ef-75d0-454a-9cd5-803a776c4985-metrics-certs\") pod \"router-default-64478b6d7c-vxvp5\" (UID: \"77ec27ef-75d0-454a-9cd5-803a776c4985\") " pod="openshift-ingress/router-default-64478b6d7c-vxvp5" Jul 07 15:13:40.565681 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:40.565176 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8cb2a4ff-480a-47b8-b781-f695be5d74a1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-ddv8m\" (UID: \"8cb2a4ff-480a-47b8-b781-f695be5d74a1\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ddv8m" Jul 07 15:13:40.565681 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:40.565215 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77ec27ef-75d0-454a-9cd5-803a776c4985-service-ca-bundle\") pod \"router-default-64478b6d7c-vxvp5\" (UID: \"77ec27ef-75d0-454a-9cd5-803a776c4985\") " pod="openshift-ingress/router-default-64478b6d7c-vxvp5" Jul 07 15:13:40.565681 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:40.565348 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/77ec27ef-75d0-454a-9cd5-803a776c4985-service-ca-bundle podName:77ec27ef-75d0-454a-9cd5-803a776c4985 nodeName:}" failed. No retries permitted until 2026-07-07 15:13:44.56533066 +0000 UTC m=+42.572808089 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/77ec27ef-75d0-454a-9cd5-803a776c4985-service-ca-bundle") pod "router-default-64478b6d7c-vxvp5" (UID: "77ec27ef-75d0-454a-9cd5-803a776c4985") : configmap references non-existent config key: service-ca.crt Jul 07 15:13:40.565681 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:40.565397 2549 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 15:13:40.565681 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:40.565407 2549 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 15:13:40.565681 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:40.565461 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a8ea65f-139e-4b35-8a9f-1471da0e27ed-samples-operator-tls podName:2a8ea65f-139e-4b35-8a9f-1471da0e27ed nodeName:}" failed. No retries permitted until 2026-07-07 15:13:44.565443908 +0000 UTC m=+42.572921340 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2a8ea65f-139e-4b35-8a9f-1471da0e27ed-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-66pbb" (UID: "2a8ea65f-139e-4b35-8a9f-1471da0e27ed") : secret "samples-operator-tls" not found Jul 07 15:13:40.565681 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:40.565481 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42fc06ec-88cb-4308-9070-c4defaf37385-metrics-tls podName:42fc06ec-88cb-4308-9070-c4defaf37385 nodeName:}" failed. No retries permitted until 2026-07-07 15:13:44.565471416 +0000 UTC m=+42.572948850 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/42fc06ec-88cb-4308-9070-c4defaf37385-metrics-tls") pod "dns-default-dgd58" (UID: "42fc06ec-88cb-4308-9070-c4defaf37385") : secret "dns-default-metrics-tls" not found Jul 07 15:13:40.565681 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:40.565506 2549 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 07 15:13:40.565681 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:40.565545 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f1ac273-713a-4bb6-94d4-4ec09090a194-networking-console-plugin-cert podName:4f1ac273-713a-4bb6-94d4-4ec09090a194 nodeName:}" failed. No retries permitted until 2026-07-07 15:13:44.56553048 +0000 UTC m=+42.573007943 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4f1ac273-713a-4bb6-94d4-4ec09090a194-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-zh2mx" (UID: "4f1ac273-713a-4bb6-94d4-4ec09090a194") : secret "networking-console-plugin-cert" not found Jul 07 15:13:40.565681 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:40.565548 2549 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 15:13:40.566482 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:40.565603 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cb2a4ff-480a-47b8-b781-f695be5d74a1-cluster-monitoring-operator-tls podName:8cb2a4ff-480a-47b8-b781-f695be5d74a1 nodeName:}" failed. No retries permitted until 2026-07-07 15:13:44.565574 +0000 UTC m=+42.573051453 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/8cb2a4ff-480a-47b8-b781-f695be5d74a1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-ddv8m" (UID: "8cb2a4ff-480a-47b8-b781-f695be5d74a1") : secret "cluster-monitoring-operator-tls" not found Jul 07 15:13:40.566482 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:40.565642 2549 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 15:13:40.566482 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:40.565679 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77ec27ef-75d0-454a-9cd5-803a776c4985-metrics-certs podName:77ec27ef-75d0-454a-9cd5-803a776c4985 nodeName:}" failed. No retries permitted until 2026-07-07 15:13:44.565667012 +0000 UTC m=+42.573144443 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/77ec27ef-75d0-454a-9cd5-803a776c4985-metrics-certs") pod "router-default-64478b6d7c-vxvp5" (UID: "77ec27ef-75d0-454a-9cd5-803a776c4985") : secret "router-metrics-certs-default" not found Jul 07 15:13:40.719437 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:40.719239 2549 generic.go:356] "Generic (PLEG): container finished" podID="c59d2067-5399-4a3d-9f34-be99e63f7d2b" containerID="46e175542c6103faf59797dcb6b921d739ec72c66c363ed1fde0b2515f0eaf6a" exitCode=0 Jul 07 15:13:40.719437 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:40.719325 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qkznn" event={"ID":"c59d2067-5399-4a3d-9f34-be99e63f7d2b","Type":"ContainerDied","Data":"46e175542c6103faf59797dcb6b921d739ec72c66c363ed1fde0b2515f0eaf6a"} Jul 07 15:13:41.723993 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:41.723954 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-9f95m" event={"ID":"f18cae1a-0479-4be9-a321-e00e056a4786","Type":"ContainerStarted","Data":"ca9182a0f2989a12e46eada95c182fdb5f0012f1f29fc94e2a08f494c1438901"} Jul 07 15:13:41.727908 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:41.727885 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qkznn" event={"ID":"c59d2067-5399-4a3d-9f34-be99e63f7d2b","Type":"ContainerStarted","Data":"87667b6f26b383a7d5a4d44ae9dffdd31a146c910d4f56f6e5d6e867247842f3"} Jul 07 15:13:41.745085 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:41.745036 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-9f95m" podStartSLOduration=34.296349803 podStartE2EDuration="37.745020821s" totalImagesPullingTime="3.448671018s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:13:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:13:41.744928877 +0000 UTC m=+39.752406328" watchObservedRunningTime="2026-07-07 15:13:41.745020821 +0000 UTC m=+39.752498272" Jul 07 15:13:41.785002 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:41.784946 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-qkznn" podStartSLOduration=17.907490845 podStartE2EDuration="39.78493093s" totalImagesPullingTime="21.877440085s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:13:02 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:13:41.78365388 +0000 UTC m=+39.791131333" watchObservedRunningTime="2026-07-07 15:13:41.78493093 +0000 UTC m=+39.792408381" Jul 07 15:13:44.502895 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:44.502847 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/22d3c83b-a693-4f0a-8ae2-07d83467c620-registry-tls\") pod \"image-registry-689c879f5d-clmr6\" (UID: \"22d3c83b-a693-4f0a-8ae2-07d83467c620\") " pod="openshift-image-registry/image-registry-689c879f5d-clmr6" Jul 07 15:13:44.503469 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:44.503029 2549 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 15:13:44.503469 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:44.503052 2549 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-689c879f5d-clmr6: secret "image-registry-tls" not found Jul 07 15:13:44.503469 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:44.503125 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/22d3c83b-a693-4f0a-8ae2-07d83467c620-registry-tls podName:22d3c83b-a693-4f0a-8ae2-07d83467c620 nodeName:}" failed. No retries permitted until 2026-07-07 15:13:52.503102858 +0000 UTC m=+50.510580291 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/22d3c83b-a693-4f0a-8ae2-07d83467c620-registry-tls") pod "image-registry-689c879f5d-clmr6" (UID: "22d3c83b-a693-4f0a-8ae2-07d83467c620") : secret "image-registry-tls" not found Jul 07 15:13:44.603870 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:44.603836 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/42fc06ec-88cb-4308-9070-c4defaf37385-metrics-tls\") pod \"dns-default-dgd58\" (UID: \"42fc06ec-88cb-4308-9070-c4defaf37385\") " pod="openshift-dns/dns-default-dgd58" Jul 07 15:13:44.604030 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:44.603891 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4f1ac273-713a-4bb6-94d4-4ec09090a194-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-zh2mx\" (UID: \"4f1ac273-713a-4bb6-94d4-4ec09090a194\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-zh2mx" Jul 07 15:13:44.604030 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:44.603938 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2a8ea65f-139e-4b35-8a9f-1471da0e27ed-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-66pbb\" (UID: \"2a8ea65f-139e-4b35-8a9f-1471da0e27ed\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-66pbb" Jul 07 15:13:44.604030 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:44.603955 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77ec27ef-75d0-454a-9cd5-803a776c4985-metrics-certs\") pod \"router-default-64478b6d7c-vxvp5\" (UID: \"77ec27ef-75d0-454a-9cd5-803a776c4985\") " pod="openshift-ingress/router-default-64478b6d7c-vxvp5" Jul 07 15:13:44.604030 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:44.603976 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8cb2a4ff-480a-47b8-b781-f695be5d74a1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-ddv8m\" (UID: \"8cb2a4ff-480a-47b8-b781-f695be5d74a1\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ddv8m" Jul 07 15:13:44.604030 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:44.604010 2549 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 15:13:44.604242 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:44.604050 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77ec27ef-75d0-454a-9cd5-803a776c4985-service-ca-bundle\") pod \"router-default-64478b6d7c-vxvp5\" (UID: \"77ec27ef-75d0-454a-9cd5-803a776c4985\") " pod="openshift-ingress/router-default-64478b6d7c-vxvp5" Jul 07 15:13:44.604242 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:44.604064 2549 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 15:13:44.604242 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:44.604079 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42fc06ec-88cb-4308-9070-c4defaf37385-metrics-tls podName:42fc06ec-88cb-4308-9070-c4defaf37385 nodeName:}" failed. No retries permitted until 2026-07-07 15:13:52.604059031 +0000 UTC m=+50.611536479 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/42fc06ec-88cb-4308-9070-c4defaf37385-metrics-tls") pod "dns-default-dgd58" (UID: "42fc06ec-88cb-4308-9070-c4defaf37385") : secret "dns-default-metrics-tls" not found Jul 07 15:13:44.604242 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:44.604087 2549 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 07 15:13:44.604242 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:44.604095 2549 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 15:13:44.604242 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:44.604115 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a8ea65f-139e-4b35-8a9f-1471da0e27ed-samples-operator-tls podName:2a8ea65f-139e-4b35-8a9f-1471da0e27ed nodeName:}" failed. No retries permitted until 2026-07-07 15:13:52.604100762 +0000 UTC m=+50.611578191 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2a8ea65f-139e-4b35-8a9f-1471da0e27ed-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-66pbb" (UID: "2a8ea65f-139e-4b35-8a9f-1471da0e27ed") : secret "samples-operator-tls" not found Jul 07 15:13:44.604242 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:44.604134 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/77ec27ef-75d0-454a-9cd5-803a776c4985-service-ca-bundle podName:77ec27ef-75d0-454a-9cd5-803a776c4985 nodeName:}" failed. No retries permitted until 2026-07-07 15:13:52.604125137 +0000 UTC m=+50.611602570 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/77ec27ef-75d0-454a-9cd5-803a776c4985-service-ca-bundle") pod "router-default-64478b6d7c-vxvp5" (UID: "77ec27ef-75d0-454a-9cd5-803a776c4985") : configmap references non-existent config key: service-ca.crt Jul 07 15:13:44.604242 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:44.604149 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cb2a4ff-480a-47b8-b781-f695be5d74a1-cluster-monitoring-operator-tls podName:8cb2a4ff-480a-47b8-b781-f695be5d74a1 nodeName:}" failed. No retries permitted until 2026-07-07 15:13:52.604142443 +0000 UTC m=+50.611619872 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/8cb2a4ff-480a-47b8-b781-f695be5d74a1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-ddv8m" (UID: "8cb2a4ff-480a-47b8-b781-f695be5d74a1") : secret "cluster-monitoring-operator-tls" not found Jul 07 15:13:44.604242 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:44.604159 2549 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 15:13:44.604242 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:44.604165 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f1ac273-713a-4bb6-94d4-4ec09090a194-networking-console-plugin-cert podName:4f1ac273-713a-4bb6-94d4-4ec09090a194 nodeName:}" failed. No retries permitted until 2026-07-07 15:13:52.604155047 +0000 UTC m=+50.611632479 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4f1ac273-713a-4bb6-94d4-4ec09090a194-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-zh2mx" (UID: "4f1ac273-713a-4bb6-94d4-4ec09090a194") : secret "networking-console-plugin-cert" not found Jul 07 15:13:44.604242 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:44.604191 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77ec27ef-75d0-454a-9cd5-803a776c4985-metrics-certs podName:77ec27ef-75d0-454a-9cd5-803a776c4985 nodeName:}" failed. No retries permitted until 2026-07-07 15:13:52.604177359 +0000 UTC m=+50.611654799 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/77ec27ef-75d0-454a-9cd5-803a776c4985-metrics-certs") pod "router-default-64478b6d7c-vxvp5" (UID: "77ec27ef-75d0-454a-9cd5-803a776c4985") : secret "router-metrics-certs-default" not found Jul 07 15:13:46.741889 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:46.741845 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-wdtl4" event={"ID":"8056aeaf-ce8a-4d1d-a6b6-c0df02057284","Type":"ContainerStarted","Data":"0554bd916eeb0b28bc46c9f67b259f43d5011908edc77bbec5efbe1a422e64a4"} Jul 07 15:13:46.744235 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:46.744182 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7smq7" event={"ID":"4d2b93c0-1bf2-4ca8-913f-6c1e3d2e60ac","Type":"ContainerStarted","Data":"aa161adf95644453c0a465bfd4c5ecce8c91748c2c04204aa4cbfa1b32897bc3"} Jul 07 15:13:46.745887 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:46.745846 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-ntm4v" event={"ID":"6bd34c29-c215-4974-98fd-7759cb30f6b8","Type":"ContainerStarted","Data":"dd229155a484ee0e23f23e0cd5197e41645ad3b3b67114907ab8e81eb262fb77"} Jul 07 15:13:46.747520 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:46.747476 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-88b42" event={"ID":"665ed2cb-7aaf-42ab-8731-03e7cea27fc4","Type":"ContainerStarted","Data":"abcd8f8a9c9509e5bd3b61083fe484c238ad10705875605d8975e1830db2e160"} Jul 07 15:13:46.749103 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:46.749061 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-lb5l7" event={"ID":"7ca69696-483c-4a89-8761-ad8245ff5129","Type":"ContainerStarted","Data":"ff61210ee952ef1d0e951232970c8daca59b2f965be42b3accf8b120c84d6b7f"} Jul 07 15:13:46.750776 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:46.750759 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kb4ml_3301f6ef-6671-4755-9047-16959836232c/console-operator/0.log" Jul 07 15:13:46.750856 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:46.750795 2549 generic.go:356] "Generic (PLEG): container finished" podID="3301f6ef-6671-4755-9047-16959836232c" containerID="8ae7d7e01eea76f3b80f6f9e88e0ee0675a2d35c1c6f6f967cbee570e36b4ba5" exitCode=255 Jul 07 15:13:46.750856 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:46.750824 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-kb4ml" event={"ID":"3301f6ef-6671-4755-9047-16959836232c","Type":"ContainerDied","Data":"8ae7d7e01eea76f3b80f6f9e88e0ee0675a2d35c1c6f6f967cbee570e36b4ba5"} Jul 07 15:13:46.751030 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:46.751016 2549 scope.go:117] "RemoveContainer" containerID="8ae7d7e01eea76f3b80f6f9e88e0ee0675a2d35c1c6f6f967cbee570e36b4ba5" Jul 07 15:13:46.759391 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:46.759347 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-wdtl4" podStartSLOduration=35.744851244 podStartE2EDuration="44.759334341s" totalImagesPullingTime="9.014483097s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:13:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:13:46.758776507 +0000 UTC m=+44.766253960" watchObservedRunningTime="2026-07-07 15:13:46.759334341 +0000 UTC m=+44.766811783" Jul 07 15:13:46.798984 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:46.798775 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-88b42" podStartSLOduration=35.84866727 podStartE2EDuration="44.79875355s" totalImagesPullingTime="8.95008628s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:13:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:13:46.780825211 +0000 UTC m=+44.788302663" watchObservedRunningTime="2026-07-07 15:13:46.79875355 +0000 UTC m=+44.806231002" Jul 07 15:13:46.799398 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:46.799350 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-ntm4v" podStartSLOduration=35.838756454 podStartE2EDuration="44.799341039s" totalImagesPullingTime="8.960584585s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:13:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:13:46.798480824 +0000 UTC m=+44.805958276" watchObservedRunningTime="2026-07-07 15:13:46.799341039 +0000 UTC m=+44.806818491" Jul 07 15:13:46.831251 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:46.831194 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-lb5l7" podStartSLOduration=27.945073216 podStartE2EDuration="36.831171989s" totalImagesPullingTime="8.886098773s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:13:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:13:46.816914743 +0000 UTC m=+44.824392195" watchObservedRunningTime="2026-07-07 15:13:46.831171989 +0000 UTC m=+44.838649441" Jul 07 15:13:46.875109 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:46.875054 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-7smq7" podStartSLOduration=1.998541739 podStartE2EDuration="10.875032896s" totalImagesPullingTime="8.876491157s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:13:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:13:46.873493738 +0000 UTC m=+44.880971190" watchObservedRunningTime="2026-07-07 15:13:46.875032896 +0000 UTC m=+44.882510348" Jul 07 15:13:47.138678 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:47.138597 2549 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-kb4ml" Jul 07 15:13:47.138678 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:47.138677 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-kb4ml" Jul 07 15:13:47.759324 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:47.759298 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kb4ml_3301f6ef-6671-4755-9047-16959836232c/console-operator/1.log" Jul 07 15:13:47.759749 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:47.759713 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kb4ml_3301f6ef-6671-4755-9047-16959836232c/console-operator/0.log" Jul 07 15:13:47.759792 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:47.759743 2549 generic.go:356] "Generic (PLEG): container finished" podID="3301f6ef-6671-4755-9047-16959836232c" containerID="4040bf5eded5321be08e860e18a7066fbc1a53047d149d09b539d3d431cb0e20" exitCode=255 Jul 07 15:13:47.759866 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:47.759843 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-kb4ml" event={"ID":"3301f6ef-6671-4755-9047-16959836232c","Type":"ContainerDied","Data":"4040bf5eded5321be08e860e18a7066fbc1a53047d149d09b539d3d431cb0e20"} Jul 07 15:13:47.759921 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:47.759903 2549 scope.go:117] "RemoveContainer" containerID="8ae7d7e01eea76f3b80f6f9e88e0ee0675a2d35c1c6f6f967cbee570e36b4ba5" Jul 07 15:13:47.760124 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:47.760106 2549 scope.go:117] "RemoveContainer" containerID="4040bf5eded5321be08e860e18a7066fbc1a53047d149d09b539d3d431cb0e20" Jul 07 15:13:47.760360 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:47.760313 2549 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-kb4ml_openshift-console-operator(3301f6ef-6671-4755-9047-16959836232c)\"" pod="openshift-console-operator/console-operator-68b757fb67-kb4ml" podUID="3301f6ef-6671-4755-9047-16959836232c" Jul 07 15:13:48.088843 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:48.088812 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-v9726"] Jul 07 15:13:48.120438 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:48.120410 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-v9726"] Jul 07 15:13:48.120609 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:48.120529 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-v9726" Jul 07 15:13:48.122919 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:48.122896 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 07 15:13:48.123033 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:48.122990 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-265hp\"" Jul 07 15:13:48.123033 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:48.122990 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 07 15:13:48.139360 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:48.139333 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/85486152-4b69-4aac-948c-8234b1028748-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v9726\" (UID: \"85486152-4b69-4aac-948c-8234b1028748\") " pod="openshift-insights/insights-runtime-extractor-v9726" Jul 07 15:13:48.139479 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:48.139361 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/85486152-4b69-4aac-948c-8234b1028748-crio-socket\") pod \"insights-runtime-extractor-v9726\" (UID: \"85486152-4b69-4aac-948c-8234b1028748\") " pod="openshift-insights/insights-runtime-extractor-v9726" Jul 07 15:13:48.139479 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:48.139403 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/85486152-4b69-4aac-948c-8234b1028748-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v9726\" (UID: \"85486152-4b69-4aac-948c-8234b1028748\") " pod="openshift-insights/insights-runtime-extractor-v9726" Jul 07 15:13:48.139698 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:48.139480 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcdd4\" (UniqueName: \"kubernetes.io/projected/85486152-4b69-4aac-948c-8234b1028748-kube-api-access-lcdd4\") pod \"insights-runtime-extractor-v9726\" (UID: \"85486152-4b69-4aac-948c-8234b1028748\") " pod="openshift-insights/insights-runtime-extractor-v9726" Jul 07 15:13:48.139698 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:48.139509 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/85486152-4b69-4aac-948c-8234b1028748-data-volume\") pod \"insights-runtime-extractor-v9726\" (UID: \"85486152-4b69-4aac-948c-8234b1028748\") " pod="openshift-insights/insights-runtime-extractor-v9726" Jul 07 15:13:48.240495 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:48.240464 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/85486152-4b69-4aac-948c-8234b1028748-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v9726\" (UID: \"85486152-4b69-4aac-948c-8234b1028748\") " pod="openshift-insights/insights-runtime-extractor-v9726" Jul 07 15:13:48.240495 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:48.240496 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/85486152-4b69-4aac-948c-8234b1028748-crio-socket\") pod \"insights-runtime-extractor-v9726\" (UID: \"85486152-4b69-4aac-948c-8234b1028748\") " pod="openshift-insights/insights-runtime-extractor-v9726" Jul 07 15:13:48.240756 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:48.240541 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/85486152-4b69-4aac-948c-8234b1028748-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v9726\" (UID: \"85486152-4b69-4aac-948c-8234b1028748\") " pod="openshift-insights/insights-runtime-extractor-v9726" Jul 07 15:13:48.240756 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:48.240633 2549 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 15:13:48.240756 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:48.240671 2549 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 15:13:48.240756 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:48.240688 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcdd4\" (UniqueName: \"kubernetes.io/projected/85486152-4b69-4aac-948c-8234b1028748-kube-api-access-lcdd4\") pod \"insights-runtime-extractor-v9726\" (UID: \"85486152-4b69-4aac-948c-8234b1028748\") " pod="openshift-insights/insights-runtime-extractor-v9726" Jul 07 15:13:48.240756 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:48.240722 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/85486152-4b69-4aac-948c-8234b1028748-kube-rbac-proxy-cm podName:85486152-4b69-4aac-948c-8234b1028748 nodeName:}" failed. No retries permitted until 2026-07-07 15:13:48.740684234 +0000 UTC m=+46.748161677 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/85486152-4b69-4aac-948c-8234b1028748-kube-rbac-proxy-cm") pod "insights-runtime-extractor-v9726" (UID: "85486152-4b69-4aac-948c-8234b1028748") : configmap "kube-rbac-proxy" not found Jul 07 15:13:48.240756 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:48.240730 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/85486152-4b69-4aac-948c-8234b1028748-crio-socket\") pod \"insights-runtime-extractor-v9726\" (UID: \"85486152-4b69-4aac-948c-8234b1028748\") " pod="openshift-insights/insights-runtime-extractor-v9726" Jul 07 15:13:48.240756 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:48.240760 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85486152-4b69-4aac-948c-8234b1028748-insights-runtime-extractor-tls podName:85486152-4b69-4aac-948c-8234b1028748 nodeName:}" failed. No retries permitted until 2026-07-07 15:13:48.740750164 +0000 UTC m=+46.748227606 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/85486152-4b69-4aac-948c-8234b1028748-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v9726" (UID: "85486152-4b69-4aac-948c-8234b1028748") : secret "insights-runtime-extractor-tls" not found Jul 07 15:13:48.240975 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:48.240786 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/85486152-4b69-4aac-948c-8234b1028748-data-volume\") pod \"insights-runtime-extractor-v9726\" (UID: \"85486152-4b69-4aac-948c-8234b1028748\") " pod="openshift-insights/insights-runtime-extractor-v9726" Jul 07 15:13:48.241173 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:48.241158 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/85486152-4b69-4aac-948c-8234b1028748-data-volume\") pod \"insights-runtime-extractor-v9726\" (UID: \"85486152-4b69-4aac-948c-8234b1028748\") " pod="openshift-insights/insights-runtime-extractor-v9726" Jul 07 15:13:48.268074 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:48.268040 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcdd4\" (UniqueName: \"kubernetes.io/projected/85486152-4b69-4aac-948c-8234b1028748-kube-api-access-lcdd4\") pod \"insights-runtime-extractor-v9726\" (UID: \"85486152-4b69-4aac-948c-8234b1028748\") " pod="openshift-insights/insights-runtime-extractor-v9726" Jul 07 15:13:48.745489 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:48.745457 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/85486152-4b69-4aac-948c-8234b1028748-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v9726\" (UID: \"85486152-4b69-4aac-948c-8234b1028748\") " pod="openshift-insights/insights-runtime-extractor-v9726" Jul 07 15:13:48.745690 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:48.745513 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/85486152-4b69-4aac-948c-8234b1028748-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v9726\" (UID: \"85486152-4b69-4aac-948c-8234b1028748\") " pod="openshift-insights/insights-runtime-extractor-v9726" Jul 07 15:13:48.745690 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:48.745570 2549 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 15:13:48.745690 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:48.745622 2549 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 15:13:48.745690 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:48.745679 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85486152-4b69-4aac-948c-8234b1028748-insights-runtime-extractor-tls podName:85486152-4b69-4aac-948c-8234b1028748 nodeName:}" failed. No retries permitted until 2026-07-07 15:13:49.745664837 +0000 UTC m=+47.753142286 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/85486152-4b69-4aac-948c-8234b1028748-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v9726" (UID: "85486152-4b69-4aac-948c-8234b1028748") : secret "insights-runtime-extractor-tls" not found Jul 07 15:13:48.745833 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:48.745736 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/85486152-4b69-4aac-948c-8234b1028748-kube-rbac-proxy-cm podName:85486152-4b69-4aac-948c-8234b1028748 nodeName:}" failed. No retries permitted until 2026-07-07 15:13:49.745726711 +0000 UTC m=+47.753204145 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/85486152-4b69-4aac-948c-8234b1028748-kube-rbac-proxy-cm") pod "insights-runtime-extractor-v9726" (UID: "85486152-4b69-4aac-948c-8234b1028748") : configmap "kube-rbac-proxy" not found Jul 07 15:13:48.768671 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:48.768647 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kb4ml_3301f6ef-6671-4755-9047-16959836232c/console-operator/1.log" Jul 07 15:13:48.769035 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:48.768950 2549 scope.go:117] "RemoveContainer" containerID="4040bf5eded5321be08e860e18a7066fbc1a53047d149d09b539d3d431cb0e20" Jul 07 15:13:48.769115 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:48.769097 2549 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-kb4ml_openshift-console-operator(3301f6ef-6671-4755-9047-16959836232c)\"" pod="openshift-console-operator/console-operator-68b757fb67-kb4ml" podUID="3301f6ef-6671-4755-9047-16959836232c" Jul 07 15:13:49.754083 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:49.754039 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/85486152-4b69-4aac-948c-8234b1028748-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v9726\" (UID: \"85486152-4b69-4aac-948c-8234b1028748\") " pod="openshift-insights/insights-runtime-extractor-v9726" Jul 07 15:13:49.754289 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:49.754125 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/85486152-4b69-4aac-948c-8234b1028748-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v9726\" (UID: \"85486152-4b69-4aac-948c-8234b1028748\") " pod="openshift-insights/insights-runtime-extractor-v9726" Jul 07 15:13:49.754289 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:49.754183 2549 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 15:13:49.754289 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:49.754238 2549 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 15:13:49.754433 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:49.754304 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85486152-4b69-4aac-948c-8234b1028748-insights-runtime-extractor-tls podName:85486152-4b69-4aac-948c-8234b1028748 nodeName:}" failed. No retries permitted until 2026-07-07 15:13:51.754284773 +0000 UTC m=+49.761762203 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/85486152-4b69-4aac-948c-8234b1028748-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v9726" (UID: "85486152-4b69-4aac-948c-8234b1028748") : secret "insights-runtime-extractor-tls" not found Jul 07 15:13:49.754433 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:49.754330 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/85486152-4b69-4aac-948c-8234b1028748-kube-rbac-proxy-cm podName:85486152-4b69-4aac-948c-8234b1028748 nodeName:}" failed. No retries permitted until 2026-07-07 15:13:51.75431942 +0000 UTC m=+49.761796850 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/85486152-4b69-4aac-948c-8234b1028748-kube-rbac-proxy-cm") pod "insights-runtime-extractor-v9726" (UID: "85486152-4b69-4aac-948c-8234b1028748") : configmap "kube-rbac-proxy" not found Jul 07 15:13:51.021942 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:51.021913 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-t8hcc_da22234a-c3b5-450f-804c-6b6f6b238d3b/dns-node-resolver/0.log" Jul 07 15:13:51.773062 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:51.773025 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/85486152-4b69-4aac-948c-8234b1028748-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v9726\" (UID: \"85486152-4b69-4aac-948c-8234b1028748\") " pod="openshift-insights/insights-runtime-extractor-v9726" Jul 07 15:13:51.773244 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:51.773083 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/85486152-4b69-4aac-948c-8234b1028748-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v9726\" (UID: \"85486152-4b69-4aac-948c-8234b1028748\") " pod="openshift-insights/insights-runtime-extractor-v9726" Jul 07 15:13:51.773244 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:51.773192 2549 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 15:13:51.773372 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:51.773262 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/85486152-4b69-4aac-948c-8234b1028748-kube-rbac-proxy-cm podName:85486152-4b69-4aac-948c-8234b1028748 nodeName:}" failed. No retries permitted until 2026-07-07 15:13:55.773248197 +0000 UTC m=+53.780725625 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/85486152-4b69-4aac-948c-8234b1028748-kube-rbac-proxy-cm") pod "insights-runtime-extractor-v9726" (UID: "85486152-4b69-4aac-948c-8234b1028748") : configmap "kube-rbac-proxy" not found Jul 07 15:13:51.773372 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:51.773193 2549 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 15:13:51.773372 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:51.773352 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85486152-4b69-4aac-948c-8234b1028748-insights-runtime-extractor-tls podName:85486152-4b69-4aac-948c-8234b1028748 nodeName:}" failed. No retries permitted until 2026-07-07 15:13:55.773333089 +0000 UTC m=+53.780810528 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/85486152-4b69-4aac-948c-8234b1028748-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v9726" (UID: "85486152-4b69-4aac-948c-8234b1028748") : secret "insights-runtime-extractor-tls" not found Jul 07 15:13:51.824141 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:51.824114 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zd2vk_80a72d31-7f06-45f3-8c76-9dd94fdb476a/node-ca/0.log" Jul 07 15:13:52.426264 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:52.426237 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7smq7_4d2b93c0-1bf2-4ca8-913f-6c1e3d2e60ac/serve-healthcheck-canary/0.log" Jul 07 15:13:52.580164 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:52.580125 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/22d3c83b-a693-4f0a-8ae2-07d83467c620-registry-tls\") pod \"image-registry-689c879f5d-clmr6\" (UID: \"22d3c83b-a693-4f0a-8ae2-07d83467c620\") " pod="openshift-image-registry/image-registry-689c879f5d-clmr6" Jul 07 15:13:52.580330 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:52.580299 2549 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 15:13:52.580330 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:52.580314 2549 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-689c879f5d-clmr6: secret "image-registry-tls" not found Jul 07 15:13:52.580403 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:52.580369 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/22d3c83b-a693-4f0a-8ae2-07d83467c620-registry-tls podName:22d3c83b-a693-4f0a-8ae2-07d83467c620 nodeName:}" failed. No retries permitted until 2026-07-07 15:14:08.580353083 +0000 UTC m=+66.587830523 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/22d3c83b-a693-4f0a-8ae2-07d83467c620-registry-tls") pod "image-registry-689c879f5d-clmr6" (UID: "22d3c83b-a693-4f0a-8ae2-07d83467c620") : secret "image-registry-tls" not found Jul 07 15:13:52.681503 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:52.681413 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2a8ea65f-139e-4b35-8a9f-1471da0e27ed-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-66pbb\" (UID: \"2a8ea65f-139e-4b35-8a9f-1471da0e27ed\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-66pbb" Jul 07 15:13:52.681503 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:52.681453 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77ec27ef-75d0-454a-9cd5-803a776c4985-metrics-certs\") pod \"router-default-64478b6d7c-vxvp5\" (UID: \"77ec27ef-75d0-454a-9cd5-803a776c4985\") " pod="openshift-ingress/router-default-64478b6d7c-vxvp5" Jul 07 15:13:52.681503 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:52.681475 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8cb2a4ff-480a-47b8-b781-f695be5d74a1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-ddv8m\" (UID: \"8cb2a4ff-480a-47b8-b781-f695be5d74a1\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ddv8m" Jul 07 15:13:52.681503 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:52.681504 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77ec27ef-75d0-454a-9cd5-803a776c4985-service-ca-bundle\") pod \"router-default-64478b6d7c-vxvp5\" (UID: \"77ec27ef-75d0-454a-9cd5-803a776c4985\") " pod="openshift-ingress/router-default-64478b6d7c-vxvp5" Jul 07 15:13:52.681813 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:52.681532 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/42fc06ec-88cb-4308-9070-c4defaf37385-metrics-tls\") pod \"dns-default-dgd58\" (UID: \"42fc06ec-88cb-4308-9070-c4defaf37385\") " pod="openshift-dns/dns-default-dgd58" Jul 07 15:13:52.681813 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:52.681575 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4f1ac273-713a-4bb6-94d4-4ec09090a194-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-zh2mx\" (UID: \"4f1ac273-713a-4bb6-94d4-4ec09090a194\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-zh2mx" Jul 07 15:13:52.681813 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:52.681606 2549 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 15:13:52.681813 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:52.681612 2549 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 15:13:52.681813 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:52.681675 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77ec27ef-75d0-454a-9cd5-803a776c4985-metrics-certs podName:77ec27ef-75d0-454a-9cd5-803a776c4985 nodeName:}" failed. No retries permitted until 2026-07-07 15:14:08.681656933 +0000 UTC m=+66.689134373 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/77ec27ef-75d0-454a-9cd5-803a776c4985-metrics-certs") pod "router-default-64478b6d7c-vxvp5" (UID: "77ec27ef-75d0-454a-9cd5-803a776c4985") : secret "router-metrics-certs-default" not found Jul 07 15:13:52.681813 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:52.681685 2549 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 15:13:52.681813 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:52.681693 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/77ec27ef-75d0-454a-9cd5-803a776c4985-service-ca-bundle podName:77ec27ef-75d0-454a-9cd5-803a776c4985 nodeName:}" failed. No retries permitted until 2026-07-07 15:14:08.681684747 +0000 UTC m=+66.689162182 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/77ec27ef-75d0-454a-9cd5-803a776c4985-service-ca-bundle") pod "router-default-64478b6d7c-vxvp5" (UID: "77ec27ef-75d0-454a-9cd5-803a776c4985") : configmap references non-existent config key: service-ca.crt Jul 07 15:13:52.681813 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:52.681702 2549 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 15:13:52.681813 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:52.681704 2549 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 07 15:13:52.681813 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:52.681739 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cb2a4ff-480a-47b8-b781-f695be5d74a1-cluster-monitoring-operator-tls podName:8cb2a4ff-480a-47b8-b781-f695be5d74a1 nodeName:}" failed. No retries permitted until 2026-07-07 15:14:08.681722234 +0000 UTC m=+66.689199667 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/8cb2a4ff-480a-47b8-b781-f695be5d74a1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-ddv8m" (UID: "8cb2a4ff-480a-47b8-b781-f695be5d74a1") : secret "cluster-monitoring-operator-tls" not found Jul 07 15:13:52.681813 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:52.681756 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42fc06ec-88cb-4308-9070-c4defaf37385-metrics-tls podName:42fc06ec-88cb-4308-9070-c4defaf37385 nodeName:}" failed. No retries permitted until 2026-07-07 15:14:08.681748558 +0000 UTC m=+66.689225986 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/42fc06ec-88cb-4308-9070-c4defaf37385-metrics-tls") pod "dns-default-dgd58" (UID: "42fc06ec-88cb-4308-9070-c4defaf37385") : secret "dns-default-metrics-tls" not found Jul 07 15:13:52.681813 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:52.681766 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a8ea65f-139e-4b35-8a9f-1471da0e27ed-samples-operator-tls podName:2a8ea65f-139e-4b35-8a9f-1471da0e27ed nodeName:}" failed. No retries permitted until 2026-07-07 15:14:08.681761355 +0000 UTC m=+66.689238784 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2a8ea65f-139e-4b35-8a9f-1471da0e27ed-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-66pbb" (UID: "2a8ea65f-139e-4b35-8a9f-1471da0e27ed") : secret "samples-operator-tls" not found Jul 07 15:13:52.681813 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:52.681777 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f1ac273-713a-4bb6-94d4-4ec09090a194-networking-console-plugin-cert podName:4f1ac273-713a-4bb6-94d4-4ec09090a194 nodeName:}" failed. No retries permitted until 2026-07-07 15:14:08.681772188 +0000 UTC m=+66.689249617 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4f1ac273-713a-4bb6-94d4-4ec09090a194-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-zh2mx" (UID: "4f1ac273-713a-4bb6-94d4-4ec09090a194") : secret "networking-console-plugin-cert" not found Jul 07 15:13:53.223237 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:53.223208 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-88b42_665ed2cb-7aaf-42ab-8731-03e7cea27fc4/kube-storage-version-migrator-operator/0.log" Jul 07 15:13:55.808887 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:55.808855 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/85486152-4b69-4aac-948c-8234b1028748-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v9726\" (UID: \"85486152-4b69-4aac-948c-8234b1028748\") " pod="openshift-insights/insights-runtime-extractor-v9726" Jul 07 15:13:55.809324 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:55.808914 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/85486152-4b69-4aac-948c-8234b1028748-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v9726\" (UID: \"85486152-4b69-4aac-948c-8234b1028748\") " pod="openshift-insights/insights-runtime-extractor-v9726" Jul 07 15:13:55.809324 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:55.808993 2549 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 15:13:55.809324 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:55.809014 2549 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 15:13:55.809324 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:55.809052 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85486152-4b69-4aac-948c-8234b1028748-insights-runtime-extractor-tls podName:85486152-4b69-4aac-948c-8234b1028748 nodeName:}" failed. No retries permitted until 2026-07-07 15:14:03.809038008 +0000 UTC m=+61.816515438 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/85486152-4b69-4aac-948c-8234b1028748-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v9726" (UID: "85486152-4b69-4aac-948c-8234b1028748") : secret "insights-runtime-extractor-tls" not found Jul 07 15:13:55.809324 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:55.809068 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/85486152-4b69-4aac-948c-8234b1028748-kube-rbac-proxy-cm podName:85486152-4b69-4aac-948c-8234b1028748 nodeName:}" failed. No retries permitted until 2026-07-07 15:14:03.80906134 +0000 UTC m=+61.816538769 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/85486152-4b69-4aac-948c-8234b1028748-kube-rbac-proxy-cm") pod "insights-runtime-extractor-v9726" (UID: "85486152-4b69-4aac-948c-8234b1028748") : configmap "kube-rbac-proxy" not found Jul 07 15:13:57.138707 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:57.138671 2549 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-kb4ml" Jul 07 15:13:57.139198 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:57.138730 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-kb4ml" Jul 07 15:13:57.139198 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:57.139083 2549 scope.go:117] "RemoveContainer" containerID="4040bf5eded5321be08e860e18a7066fbc1a53047d149d09b539d3d431cb0e20" Jul 07 15:13:57.796842 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:57.796814 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kb4ml_3301f6ef-6671-4755-9047-16959836232c/console-operator/2.log" Jul 07 15:13:57.797199 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:57.797184 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kb4ml_3301f6ef-6671-4755-9047-16959836232c/console-operator/1.log" Jul 07 15:13:57.797251 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:57.797216 2549 generic.go:356] "Generic (PLEG): container finished" podID="3301f6ef-6671-4755-9047-16959836232c" containerID="c52b276cfa1addd875b8f56ebb7a09f9d8244250854fa74b5e069cf1e3479fff" exitCode=255 Jul 07 15:13:57.797285 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:57.797265 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-kb4ml" event={"ID":"3301f6ef-6671-4755-9047-16959836232c","Type":"ContainerDied","Data":"c52b276cfa1addd875b8f56ebb7a09f9d8244250854fa74b5e069cf1e3479fff"} Jul 07 15:13:57.797319 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:57.797310 2549 scope.go:117] "RemoveContainer" containerID="4040bf5eded5321be08e860e18a7066fbc1a53047d149d09b539d3d431cb0e20" Jul 07 15:13:57.797680 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:57.797651 2549 scope.go:117] "RemoveContainer" containerID="c52b276cfa1addd875b8f56ebb7a09f9d8244250854fa74b5e069cf1e3479fff" Jul 07 15:13:57.797836 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:13:57.797815 2549 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-kb4ml_openshift-console-operator(3301f6ef-6671-4755-9047-16959836232c)\"" pod="openshift-console-operator/console-operator-68b757fb67-kb4ml" podUID="3301f6ef-6671-4755-9047-16959836232c" Jul 07 15:13:58.802220 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:13:58.802192 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kb4ml_3301f6ef-6671-4755-9047-16959836232c/console-operator/2.log" Jul 07 15:14:02.690018 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:02.689990 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9p7dn" Jul 07 15:14:03.879193 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:03.879159 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/85486152-4b69-4aac-948c-8234b1028748-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v9726\" (UID: \"85486152-4b69-4aac-948c-8234b1028748\") " pod="openshift-insights/insights-runtime-extractor-v9726" Jul 07 15:14:03.879694 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:03.879219 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/85486152-4b69-4aac-948c-8234b1028748-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v9726\" (UID: \"85486152-4b69-4aac-948c-8234b1028748\") " pod="openshift-insights/insights-runtime-extractor-v9726" Jul 07 15:14:03.879694 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:14:03.879340 2549 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 15:14:03.879694 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:14:03.879423 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85486152-4b69-4aac-948c-8234b1028748-insights-runtime-extractor-tls podName:85486152-4b69-4aac-948c-8234b1028748 nodeName:}" failed. No retries permitted until 2026-07-07 15:14:19.879401334 +0000 UTC m=+77.886878785 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/85486152-4b69-4aac-948c-8234b1028748-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v9726" (UID: "85486152-4b69-4aac-948c-8234b1028748") : secret "insights-runtime-extractor-tls" not found Jul 07 15:14:03.879694 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:03.879664 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/85486152-4b69-4aac-948c-8234b1028748-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v9726\" (UID: \"85486152-4b69-4aac-948c-8234b1028748\") " pod="openshift-insights/insights-runtime-extractor-v9726" Jul 07 15:14:07.138243 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:07.138198 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-kb4ml" Jul 07 15:14:07.138243 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:07.138254 2549 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-kb4ml" Jul 07 15:14:07.138700 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:07.138542 2549 scope.go:117] "RemoveContainer" containerID="c52b276cfa1addd875b8f56ebb7a09f9d8244250854fa74b5e069cf1e3479fff" Jul 07 15:14:07.138738 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:14:07.138717 2549 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-kb4ml_openshift-console-operator(3301f6ef-6671-4755-9047-16959836232c)\"" pod="openshift-console-operator/console-operator-68b757fb67-kb4ml" podUID="3301f6ef-6671-4755-9047-16959836232c" Jul 07 15:14:08.119426 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:08.119365 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/274e9d34-35d9-452a-996f-1183641bf618-metrics-certs\") pod \"network-metrics-daemon-xqp6p\" (UID: \"274e9d34-35d9-452a-996f-1183641bf618\") " pod="openshift-multus/network-metrics-daemon-xqp6p" Jul 07 15:14:08.121928 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:08.121898 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 15:14:08.132541 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:08.132497 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/274e9d34-35d9-452a-996f-1183641bf618-metrics-certs\") pod \"network-metrics-daemon-xqp6p\" (UID: \"274e9d34-35d9-452a-996f-1183641bf618\") " pod="openshift-multus/network-metrics-daemon-xqp6p" Jul 07 15:14:08.143817 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:08.143779 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-lhgw4\"" Jul 07 15:14:08.151769 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:08.151748 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xqp6p" Jul 07 15:14:08.220235 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:08.220201 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgrk9\" (UniqueName: \"kubernetes.io/projected/95c962c7-1672-403f-9d88-acc1fc891e91-kube-api-access-rgrk9\") pod \"network-check-target-9dwk4\" (UID: \"95c962c7-1672-403f-9d88-acc1fc891e91\") " pod="openshift-network-diagnostics/network-check-target-9dwk4" Jul 07 15:14:08.220390 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:08.220246 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/473572dc-5da2-44c1-918d-a409370c493f-original-pull-secret\") pod \"global-pull-secret-syncer-xt54w\" (UID: \"473572dc-5da2-44c1-918d-a409370c493f\") " pod="kube-system/global-pull-secret-syncer-xt54w" Jul 07 15:14:08.222746 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:08.222698 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgrk9\" (UniqueName: \"kubernetes.io/projected/95c962c7-1672-403f-9d88-acc1fc891e91-kube-api-access-rgrk9\") pod \"network-check-target-9dwk4\" (UID: \"95c962c7-1672-403f-9d88-acc1fc891e91\") " pod="openshift-network-diagnostics/network-check-target-9dwk4" Jul 07 15:14:08.222875 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:08.222799 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 15:14:08.233792 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:08.233735 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/473572dc-5da2-44c1-918d-a409370c493f-original-pull-secret\") pod \"global-pull-secret-syncer-xt54w\" (UID: \"473572dc-5da2-44c1-918d-a409370c493f\") " pod="kube-system/global-pull-secret-syncer-xt54w" Jul 07 15:14:08.242689 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:08.242659 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xt54w" Jul 07 15:14:08.280856 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:08.280825 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xqp6p"] Jul 07 15:14:08.285428 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:14:08.285405 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod274e9d34_35d9_452a_996f_1183641bf618.slice/crio-785ee02d7be6db34d02d930169c033a228c5a3bc510d726703dd580435a5295c WatchSource:0}: Error finding container 785ee02d7be6db34d02d930169c033a228c5a3bc510d726703dd580435a5295c: Status 404 returned error can't find the container with id 785ee02d7be6db34d02d930169c033a228c5a3bc510d726703dd580435a5295c Jul 07 15:14:08.370941 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:08.370855 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xt54w"] Jul 07 15:14:08.373781 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:14:08.373754 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod473572dc_5da2_44c1_918d_a409370c493f.slice/crio-50597c3b5bc2fdb678381d3b98bb69c786dfc2b5a02ab7c03ea7b3e725bacd26 WatchSource:0}: Error finding container 50597c3b5bc2fdb678381d3b98bb69c786dfc2b5a02ab7c03ea7b3e725bacd26: Status 404 returned error can't find the container with id 50597c3b5bc2fdb678381d3b98bb69c786dfc2b5a02ab7c03ea7b3e725bacd26 Jul 07 15:14:08.449402 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:08.449371 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-5bbhh\"" Jul 07 15:14:08.457299 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:08.457279 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9dwk4" Jul 07 15:14:08.583008 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:08.582977 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9dwk4"] Jul 07 15:14:08.586367 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:14:08.586333 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95c962c7_1672_403f_9d88_acc1fc891e91.slice/crio-50eb1614ecdcb7156a670074f25eada29e0afa7b300e976aa0b8bc57fe4b0473 WatchSource:0}: Error finding container 50eb1614ecdcb7156a670074f25eada29e0afa7b300e976aa0b8bc57fe4b0473: Status 404 returned error can't find the container with id 50eb1614ecdcb7156a670074f25eada29e0afa7b300e976aa0b8bc57fe4b0473 Jul 07 15:14:08.624652 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:08.624612 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/22d3c83b-a693-4f0a-8ae2-07d83467c620-registry-tls\") pod \"image-registry-689c879f5d-clmr6\" (UID: \"22d3c83b-a693-4f0a-8ae2-07d83467c620\") " pod="openshift-image-registry/image-registry-689c879f5d-clmr6" Jul 07 15:14:08.626873 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:08.626849 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/22d3c83b-a693-4f0a-8ae2-07d83467c620-registry-tls\") pod \"image-registry-689c879f5d-clmr6\" (UID: \"22d3c83b-a693-4f0a-8ae2-07d83467c620\") " pod="openshift-image-registry/image-registry-689c879f5d-clmr6" Jul 07 15:14:08.726125 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:08.726089 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2a8ea65f-139e-4b35-8a9f-1471da0e27ed-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-66pbb\" (UID: \"2a8ea65f-139e-4b35-8a9f-1471da0e27ed\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-66pbb" Jul 07 15:14:08.726306 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:08.726133 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77ec27ef-75d0-454a-9cd5-803a776c4985-metrics-certs\") pod \"router-default-64478b6d7c-vxvp5\" (UID: \"77ec27ef-75d0-454a-9cd5-803a776c4985\") " pod="openshift-ingress/router-default-64478b6d7c-vxvp5" Jul 07 15:14:08.726306 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:08.726249 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8cb2a4ff-480a-47b8-b781-f695be5d74a1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-ddv8m\" (UID: \"8cb2a4ff-480a-47b8-b781-f695be5d74a1\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ddv8m" Jul 07 15:14:08.726306 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:08.726303 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77ec27ef-75d0-454a-9cd5-803a776c4985-service-ca-bundle\") pod \"router-default-64478b6d7c-vxvp5\" (UID: \"77ec27ef-75d0-454a-9cd5-803a776c4985\") " pod="openshift-ingress/router-default-64478b6d7c-vxvp5" Jul 07 15:14:08.726468 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:08.726352 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/42fc06ec-88cb-4308-9070-c4defaf37385-metrics-tls\") pod \"dns-default-dgd58\" (UID: \"42fc06ec-88cb-4308-9070-c4defaf37385\") " pod="openshift-dns/dns-default-dgd58" Jul 07 15:14:08.726468 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:08.726394 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4f1ac273-713a-4bb6-94d4-4ec09090a194-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-zh2mx\" (UID: \"4f1ac273-713a-4bb6-94d4-4ec09090a194\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-zh2mx" Jul 07 15:14:08.727317 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:08.727251 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77ec27ef-75d0-454a-9cd5-803a776c4985-service-ca-bundle\") pod \"router-default-64478b6d7c-vxvp5\" (UID: \"77ec27ef-75d0-454a-9cd5-803a776c4985\") " pod="openshift-ingress/router-default-64478b6d7c-vxvp5" Jul 07 15:14:08.729203 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:08.729154 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/42fc06ec-88cb-4308-9070-c4defaf37385-metrics-tls\") pod \"dns-default-dgd58\" (UID: \"42fc06ec-88cb-4308-9070-c4defaf37385\") " pod="openshift-dns/dns-default-dgd58" Jul 07 15:14:08.729327 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:08.729233 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4f1ac273-713a-4bb6-94d4-4ec09090a194-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-zh2mx\" (UID: \"4f1ac273-713a-4bb6-94d4-4ec09090a194\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-zh2mx" Jul 07 15:14:08.729371 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:08.729351 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8cb2a4ff-480a-47b8-b781-f695be5d74a1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-ddv8m\" (UID: \"8cb2a4ff-480a-47b8-b781-f695be5d74a1\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ddv8m" Jul 07 15:14:08.729613 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:08.729571 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2a8ea65f-139e-4b35-8a9f-1471da0e27ed-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-66pbb\" (UID: \"2a8ea65f-139e-4b35-8a9f-1471da0e27ed\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-66pbb" Jul 07 15:14:08.729924 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:08.729903 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77ec27ef-75d0-454a-9cd5-803a776c4985-metrics-certs\") pod \"router-default-64478b6d7c-vxvp5\" (UID: \"77ec27ef-75d0-454a-9cd5-803a776c4985\") " pod="openshift-ingress/router-default-64478b6d7c-vxvp5" Jul 07 15:14:08.837707 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:08.837667 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9dwk4" event={"ID":"95c962c7-1672-403f-9d88-acc1fc891e91","Type":"ContainerStarted","Data":"7d63b9ececb8b18c82652c9ece79e086613dd340c09365eb0cb87d678b40a234"} Jul 07 15:14:08.837902 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:08.837727 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9dwk4" event={"ID":"95c962c7-1672-403f-9d88-acc1fc891e91","Type":"ContainerStarted","Data":"50eb1614ecdcb7156a670074f25eada29e0afa7b300e976aa0b8bc57fe4b0473"} Jul 07 15:14:08.837902 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:08.837768 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-9dwk4" Jul 07 15:14:08.839642 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:08.839611 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xt54w" event={"ID":"473572dc-5da2-44c1-918d-a409370c493f","Type":"ContainerStarted","Data":"50597c3b5bc2fdb678381d3b98bb69c786dfc2b5a02ab7c03ea7b3e725bacd26"} Jul 07 15:14:08.841258 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:08.841230 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xqp6p" event={"ID":"274e9d34-35d9-452a-996f-1183641bf618","Type":"ContainerStarted","Data":"785ee02d7be6db34d02d930169c033a228c5a3bc510d726703dd580435a5295c"} Jul 07 15:14:08.851824 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:08.851763 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-st7wq\"" Jul 07 15:14:08.854412 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:08.854363 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-9dwk4" podStartSLOduration=66.854346172 podStartE2EDuration="1m6.854346172s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:13:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:14:08.854183633 +0000 UTC m=+66.861661087" watchObservedRunningTime="2026-07-07 15:14:08.854346172 +0000 UTC m=+66.861823624" Jul 07 15:14:08.860478 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:08.860449 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-689c879f5d-clmr6" Jul 07 15:14:08.878151 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:08.878078 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-pb8qd\"" Jul 07 15:14:08.886157 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:08.886067 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dgd58" Jul 07 15:14:08.912634 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:08.912533 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-kspmr\"" Jul 07 15:14:08.921085 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:08.920855 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-64478b6d7c-vxvp5" Jul 07 15:14:08.927475 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:08.927412 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-cxv4p\"" Jul 07 15:14:08.934768 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:08.934553 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ddv8m" Jul 07 15:14:08.935369 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:08.935014 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-4wpvl\"" Jul 07 15:14:08.942849 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:08.942705 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-66pbb" Jul 07 15:14:08.956451 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:08.956252 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-c5fvs\"" Jul 07 15:14:08.964486 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:08.964429 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-zh2mx" Jul 07 15:14:09.014179 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:09.014143 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-689c879f5d-clmr6"] Jul 07 15:14:09.170429 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:14:09.170356 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22d3c83b_a693_4f0a_8ae2_07d83467c620.slice/crio-b56eee08114041845c2541ba6ac09723a3749b5b28a560ab833e0cf652d792f1 WatchSource:0}: Error finding container b56eee08114041845c2541ba6ac09723a3749b5b28a560ab833e0cf652d792f1: Status 404 returned error can't find the container with id b56eee08114041845c2541ba6ac09723a3749b5b28a560ab833e0cf652d792f1 Jul 07 15:14:09.388623 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:09.388524 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dgd58"] Jul 07 15:14:09.390727 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:14:09.390702 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42fc06ec_88cb_4308_9070_c4defaf37385.slice/crio-ecd36447480f3aa0deefc84373e6c460e0730753476864f2272004671b2909fc WatchSource:0}: Error finding container ecd36447480f3aa0deefc84373e6c460e0730753476864f2272004671b2909fc: Status 404 returned error can't find the container with id ecd36447480f3aa0deefc84373e6c460e0730753476864f2272004671b2909fc Jul 07 15:14:09.616607 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:09.616562 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ddv8m"] Jul 07 15:14:09.620002 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:09.619982 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-zh2mx"] Jul 07 15:14:09.621441 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:09.621378 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-66pbb"] Jul 07 15:14:09.623203 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:14:09.622926 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f1ac273_713a_4bb6_94d4_4ec09090a194.slice/crio-1db197d82a813ccd72c6801fac3f092a5a838ddb15d8999d665d7d8cdf9ee152 WatchSource:0}: Error finding container 1db197d82a813ccd72c6801fac3f092a5a838ddb15d8999d665d7d8cdf9ee152: Status 404 returned error can't find the container with id 1db197d82a813ccd72c6801fac3f092a5a838ddb15d8999d665d7d8cdf9ee152 Jul 07 15:14:09.629262 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:09.629237 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-64478b6d7c-vxvp5"] Jul 07 15:14:09.637996 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:14:09.637970 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77ec27ef_75d0_454a_9cd5_803a776c4985.slice/crio-ea616dfb42ba12ceca681e87ceaa20c2d30e6407e60f9dcb94405c4b2badb2ce WatchSource:0}: Error finding container ea616dfb42ba12ceca681e87ceaa20c2d30e6407e60f9dcb94405c4b2badb2ce: Status 404 returned error can't find the container with id ea616dfb42ba12ceca681e87ceaa20c2d30e6407e60f9dcb94405c4b2badb2ce Jul 07 15:14:09.851300 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:09.851255 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-66pbb" event={"ID":"2a8ea65f-139e-4b35-8a9f-1471da0e27ed","Type":"ContainerStarted","Data":"8c9ec3e1c7490bfb38eaaef8ca7096fab648318c637518fb4b1c34adc0579543"} Jul 07 15:14:09.853162 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:09.853134 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-64478b6d7c-vxvp5" event={"ID":"77ec27ef-75d0-454a-9cd5-803a776c4985","Type":"ContainerStarted","Data":"814b60d3722fb56141e736776f95b7dc37a16e04eb7188ef012272bb1a4b86fe"} Jul 07 15:14:09.853280 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:09.853179 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-64478b6d7c-vxvp5" event={"ID":"77ec27ef-75d0-454a-9cd5-803a776c4985","Type":"ContainerStarted","Data":"ea616dfb42ba12ceca681e87ceaa20c2d30e6407e60f9dcb94405c4b2badb2ce"} Jul 07 15:14:09.854564 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:09.854534 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dgd58" event={"ID":"42fc06ec-88cb-4308-9070-c4defaf37385","Type":"ContainerStarted","Data":"ecd36447480f3aa0deefc84373e6c460e0730753476864f2272004671b2909fc"} Jul 07 15:14:09.856147 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:09.856126 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-689c879f5d-clmr6" event={"ID":"22d3c83b-a693-4f0a-8ae2-07d83467c620","Type":"ContainerStarted","Data":"de6e500aa9a3c4ff9709aa23213388baa66768696c8532aeff266f74597a38e5"} Jul 07 15:14:09.856249 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:09.856164 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-689c879f5d-clmr6" event={"ID":"22d3c83b-a693-4f0a-8ae2-07d83467c620","Type":"ContainerStarted","Data":"b56eee08114041845c2541ba6ac09723a3749b5b28a560ab833e0cf652d792f1"} Jul 07 15:14:09.856249 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:09.856190 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-689c879f5d-clmr6" Jul 07 15:14:09.857639 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:09.857619 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ddv8m" event={"ID":"8cb2a4ff-480a-47b8-b781-f695be5d74a1","Type":"ContainerStarted","Data":"2c1defbf54010d02c0a878093ebff21be3370d19c8a74c19a4ff4e6ccb225784"} Jul 07 15:14:09.859660 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:09.859639 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xqp6p" event={"ID":"274e9d34-35d9-452a-996f-1183641bf618","Type":"ContainerStarted","Data":"2a98cd952271c9befdcec617174d39e26d361608c7d3aef9c79c579c0e604460"} Jul 07 15:14:09.859817 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:09.859802 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xqp6p" event={"ID":"274e9d34-35d9-452a-996f-1183641bf618","Type":"ContainerStarted","Data":"c8e5a4fd50c1127c232fb3b5cce2d8038e42ad6b56d89fc2406f93ad42ec1dbb"} Jul 07 15:14:09.860969 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:09.860945 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-zh2mx" event={"ID":"4f1ac273-713a-4bb6-94d4-4ec09090a194","Type":"ContainerStarted","Data":"1db197d82a813ccd72c6801fac3f092a5a838ddb15d8999d665d7d8cdf9ee152"} Jul 07 15:14:09.921523 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:09.921450 2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-64478b6d7c-vxvp5" Jul 07 15:14:09.924844 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:09.924621 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-689c879f5d-clmr6" podStartSLOduration=67.92460263 podStartE2EDuration="1m7.92460263s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:13:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:14:09.924116771 +0000 UTC m=+67.931594224" watchObservedRunningTime="2026-07-07 15:14:09.92460263 +0000 UTC m=+67.932080074" Jul 07 15:14:09.924844 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:09.924823 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-64478b6d7c-vxvp5" podStartSLOduration=67.924814629 podStartE2EDuration="1m7.924814629s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:13:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:14:09.891000509 +0000 UTC m=+67.898477962" watchObservedRunningTime="2026-07-07 15:14:09.924814629 +0000 UTC m=+67.932292083" Jul 07 15:14:09.925027 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:09.924851 2549 patch_prober.go:28] interesting pod/router-default-64478b6d7c-vxvp5 container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 07 15:14:09.925027 ip-10-0-138-11 kubenswrapper[2549]: [+]has-synced ok Jul 07 15:14:09.925027 ip-10-0-138-11 kubenswrapper[2549]: [+]process-running ok Jul 07 15:14:09.925027 ip-10-0-138-11 kubenswrapper[2549]: healthz check failed Jul 07 15:14:09.925027 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:09.924896 2549 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-64478b6d7c-vxvp5" podUID="77ec27ef-75d0-454a-9cd5-803a776c4985" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 07 15:14:09.961742 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:09.961619 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-xqp6p" podStartSLOduration=66.871413844 podStartE2EDuration="1m7.961598584s" totalImagesPullingTime="1.09018474s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:13:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:14:09.960917403 +0000 UTC m=+67.968394860" watchObservedRunningTime="2026-07-07 15:14:09.961598584 +0000 UTC m=+67.969076037" Jul 07 15:14:10.923910 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:10.923696 2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-64478b6d7c-vxvp5" Jul 07 15:14:11.869237 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:11.869202 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-64478b6d7c-vxvp5" Jul 07 15:14:11.870714 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:11.870692 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-64478b6d7c-vxvp5" Jul 07 15:14:15.885295 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:15.885209 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xt54w" event={"ID":"473572dc-5da2-44c1-918d-a409370c493f","Type":"ContainerStarted","Data":"ad0a569e4e3e2716fec9006355f88b76873068797820763653fa7c89c3ed426a"} Jul 07 15:14:15.886964 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:15.886935 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-zh2mx" event={"ID":"4f1ac273-713a-4bb6-94d4-4ec09090a194","Type":"ContainerStarted","Data":"2f28d46287c8da197d0da0c09d7a856f49f72996bc673ce7b9d8836a6c6ba320"} Jul 07 15:14:15.888822 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:15.888798 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-66pbb" event={"ID":"2a8ea65f-139e-4b35-8a9f-1471da0e27ed","Type":"ContainerStarted","Data":"adfcb452b176984242f0f34f8075e69ae0832c3ad4d631a7786f11d5e7ddb1b4"} Jul 07 15:14:15.888926 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:15.888845 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-66pbb" event={"ID":"2a8ea65f-139e-4b35-8a9f-1471da0e27ed","Type":"ContainerStarted","Data":"20ac712c7fd1c367554745f99117e9bcec6827a7d375982d44fd5e4a72c03e2e"} Jul 07 15:14:15.890550 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:15.890524 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dgd58" event={"ID":"42fc06ec-88cb-4308-9070-c4defaf37385","Type":"ContainerStarted","Data":"e74d712c7be4b9eb7ba89fb47fdd3046db3576a2d3909a7752142af3b144c9f8"} Jul 07 15:14:15.890670 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:15.890576 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-dgd58" Jul 07 15:14:15.890670 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:15.890616 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dgd58" event={"ID":"42fc06ec-88cb-4308-9070-c4defaf37385","Type":"ContainerStarted","Data":"6a083af6d82cfdd55461358d30b97d5b8d5680338c2ef1914460f3f563a267b8"} Jul 07 15:14:15.891968 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:15.891949 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ddv8m" event={"ID":"8cb2a4ff-480a-47b8-b781-f695be5d74a1","Type":"ContainerStarted","Data":"3c20410fa169ac2c60c92070a0e0423dc642e66578488c99ecb65efcf4404f9e"} Jul 07 15:14:15.937499 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:15.937434 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-xt54w" podStartSLOduration=66.997203444 podStartE2EDuration="1m13.93741261s" totalImagesPullingTime="6.940209166s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:13:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:14:15.911741552 +0000 UTC m=+73.919219004" watchObservedRunningTime="2026-07-07 15:14:15.93741261 +0000 UTC m=+73.944890042" Jul 07 15:14:15.938752 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:15.938717 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-66pbb" podStartSLOduration=68.325660653 podStartE2EDuration="1m13.938703644s" totalImagesPullingTime="5.613042991s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:13:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:14:15.937176734 +0000 UTC m=+73.944654239" watchObservedRunningTime="2026-07-07 15:14:15.938703644 +0000 UTC m=+73.946181096" Jul 07 15:14:15.960636 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:15.960568 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ddv8m" podStartSLOduration=68.267809423 podStartE2EDuration="1m13.960554135s" totalImagesPullingTime="5.692744712s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:13:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:14:15.960026916 +0000 UTC m=+73.967504367" watchObservedRunningTime="2026-07-07 15:14:15.960554135 +0000 UTC m=+73.968031597" Jul 07 15:14:15.973174 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:15.973138 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-7qqd8"] Jul 07 15:14:15.986761 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:15.986708 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-zh2mx" podStartSLOduration=42.294523506 podStartE2EDuration="47.98669267s" totalImagesPullingTime="5.692169164s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:13:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:14:15.985667641 +0000 UTC m=+73.993145092" watchObservedRunningTime="2026-07-07 15:14:15.98669267 +0000 UTC m=+73.994170119" Jul 07 15:14:15.988444 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:15.988424 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-7qqd8" Jul 07 15:14:15.988569 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:15.988523 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-7qqd8"] Jul 07 15:14:15.990721 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:15.990698 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 07 15:14:15.990813 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:15.990718 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-t452r\"" Jul 07 15:14:16.043234 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:16.043172 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-dgd58" podStartSLOduration=34.121197624 podStartE2EDuration="40.043149473s" totalImagesPullingTime="5.921951849s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:13:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:14:16.011914844 +0000 UTC m=+74.019392296" watchObservedRunningTime="2026-07-07 15:14:16.043149473 +0000 UTC m=+74.050626926" Jul 07 15:14:16.093753 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:16.093717 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d73abd9e-6eb2-4aa5-8ce9-edaeac1b8721-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-7qqd8\" (UID: \"d73abd9e-6eb2-4aa5-8ce9-edaeac1b8721\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-7qqd8" Jul 07 15:14:16.195146 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:16.195052 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d73abd9e-6eb2-4aa5-8ce9-edaeac1b8721-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-7qqd8\" (UID: \"d73abd9e-6eb2-4aa5-8ce9-edaeac1b8721\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-7qqd8" Jul 07 15:14:16.195299 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:14:16.195191 2549 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 07 15:14:16.195299 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:14:16.195256 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d73abd9e-6eb2-4aa5-8ce9-edaeac1b8721-tls-certificates podName:d73abd9e-6eb2-4aa5-8ce9-edaeac1b8721 nodeName:}" failed. No retries permitted until 2026-07-07 15:14:16.69524104 +0000 UTC m=+74.702718469 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/d73abd9e-6eb2-4aa5-8ce9-edaeac1b8721-tls-certificates") pod "prometheus-operator-admission-webhook-d559d66d5-7qqd8" (UID: "d73abd9e-6eb2-4aa5-8ce9-edaeac1b8721") : secret "prometheus-operator-admission-webhook-tls" not found Jul 07 15:14:16.698838 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:16.698794 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d73abd9e-6eb2-4aa5-8ce9-edaeac1b8721-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-7qqd8\" (UID: \"d73abd9e-6eb2-4aa5-8ce9-edaeac1b8721\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-7qqd8" Jul 07 15:14:16.701213 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:16.701180 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d73abd9e-6eb2-4aa5-8ce9-edaeac1b8721-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-7qqd8\" (UID: \"d73abd9e-6eb2-4aa5-8ce9-edaeac1b8721\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-7qqd8" Jul 07 15:14:16.903804 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:16.903770 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-7qqd8" Jul 07 15:14:17.067370 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:17.067337 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-7qqd8"] Jul 07 15:14:17.071217 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:14:17.071191 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd73abd9e_6eb2_4aa5_8ce9_edaeac1b8721.slice/crio-df01df30995dc35bb7b8070808cbaf542c67e8c6685f071bfa11aad5f144f37a WatchSource:0}: Error finding container df01df30995dc35bb7b8070808cbaf542c67e8c6685f071bfa11aad5f144f37a: Status 404 returned error can't find the container with id df01df30995dc35bb7b8070808cbaf542c67e8c6685f071bfa11aad5f144f37a Jul 07 15:14:17.529004 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:17.528975 2549 scope.go:117] "RemoveContainer" containerID="c52b276cfa1addd875b8f56ebb7a09f9d8244250854fa74b5e069cf1e3479fff" Jul 07 15:14:17.902118 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:17.902047 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-7qqd8" event={"ID":"d73abd9e-6eb2-4aa5-8ce9-edaeac1b8721","Type":"ContainerStarted","Data":"df01df30995dc35bb7b8070808cbaf542c67e8c6685f071bfa11aad5f144f37a"} Jul 07 15:14:17.904280 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:17.904257 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kb4ml_3301f6ef-6671-4755-9047-16959836232c/console-operator/2.log" Jul 07 15:14:17.904704 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:17.904330 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-kb4ml" event={"ID":"3301f6ef-6671-4755-9047-16959836232c","Type":"ContainerStarted","Data":"29716ebc1d0c2cfa0474895713ae3aafb71d3be316dea8c73ffa05388c2d4c42"} Jul 07 15:14:17.904704 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:17.904615 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-kb4ml" Jul 07 15:14:17.926464 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:17.926401 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-kb4ml" podStartSLOduration=64.049986176 podStartE2EDuration="1m12.926382059s" totalImagesPullingTime="8.876395883s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:13:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:14:17.92609638 +0000 UTC m=+75.933573832" watchObservedRunningTime="2026-07-07 15:14:17.926382059 +0000 UTC m=+75.933859512" Jul 07 15:14:18.581834 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:18.581799 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-kb4ml" Jul 07 15:14:18.909666 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:18.909629 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-7qqd8" event={"ID":"d73abd9e-6eb2-4aa5-8ce9-edaeac1b8721","Type":"ContainerStarted","Data":"680ebea9cb25ac719aa9bbaf0687d80496c04fc319bf3375918579e40d304b24"} Jul 07 15:14:18.910069 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:18.909841 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-7qqd8" Jul 07 15:14:18.914090 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:18.914071 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-7qqd8" Jul 07 15:14:18.987211 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:18.987159 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-7qqd8" podStartSLOduration=2.601241665 podStartE2EDuration="3.987137747s" totalImagesPullingTime="1.385896082s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:14:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:14:18.947673425 +0000 UTC m=+76.955150887" watchObservedRunningTime="2026-07-07 15:14:18.987137747 +0000 UTC m=+76.994615199" Jul 07 15:14:19.924870 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:19.924842 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/85486152-4b69-4aac-948c-8234b1028748-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v9726\" (UID: \"85486152-4b69-4aac-948c-8234b1028748\") " pod="openshift-insights/insights-runtime-extractor-v9726" Jul 07 15:14:19.927083 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:19.927061 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/85486152-4b69-4aac-948c-8234b1028748-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v9726\" (UID: \"85486152-4b69-4aac-948c-8234b1028748\") " pod="openshift-insights/insights-runtime-extractor-v9726" Jul 07 15:14:19.932655 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:19.932632 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-265hp\"" Jul 07 15:14:19.940856 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:19.940833 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-v9726" Jul 07 15:14:20.075602 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:20.075544 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-v9726"] Jul 07 15:14:20.078690 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:14:20.078658 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85486152_4b69_4aac_948c_8234b1028748.slice/crio-ee8ac9779434aa6101f8a6f9c6a78d71b8fba5b41c2d18703f25eae41f75171d WatchSource:0}: Error finding container ee8ac9779434aa6101f8a6f9c6a78d71b8fba5b41c2d18703f25eae41f75171d: Status 404 returned error can't find the container with id ee8ac9779434aa6101f8a6f9c6a78d71b8fba5b41c2d18703f25eae41f75171d Jul 07 15:14:20.920760 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:20.920638 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v9726" event={"ID":"85486152-4b69-4aac-948c-8234b1028748","Type":"ContainerStarted","Data":"ee8ac9779434aa6101f8a6f9c6a78d71b8fba5b41c2d18703f25eae41f75171d"} Jul 07 15:14:23.933327 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:23.933189 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v9726" event={"ID":"85486152-4b69-4aac-948c-8234b1028748","Type":"ContainerStarted","Data":"f4401b45f877b14f9e8885dce770335662d0b821044b101818131679522aee08"} Jul 07 15:14:24.939660 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:24.939616 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v9726" event={"ID":"85486152-4b69-4aac-948c-8234b1028748","Type":"ContainerStarted","Data":"6d01aa5f5b66a462d62600714eeb14425e5c1bf304d9969dd4907083ac03746d"} Jul 07 15:14:25.898389 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:25.898360 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-dgd58" Jul 07 15:14:26.949576 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:26.949487 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v9726" event={"ID":"85486152-4b69-4aac-948c-8234b1028748","Type":"ContainerStarted","Data":"5f75d7c213bc4e50d01df690d69a9f23fd5e1688f6c6f094ed969876fd55c5c7"} Jul 07 15:14:26.988663 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:26.988608 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-v9726" podStartSLOduration=32.50754875 podStartE2EDuration="38.988573601s" totalImagesPullingTime="6.481024851s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:13:48 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:14:26.987815097 +0000 UTC m=+84.995292543" watchObservedRunningTime="2026-07-07 15:14:26.988573601 +0000 UTC m=+84.996051050" Jul 07 15:14:27.479635 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:27.479603 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-5pxlr"] Jul 07 15:14:27.482232 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:27.482205 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-5pxlr" Jul 07 15:14:27.484835 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:27.484814 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-xnvxx\"" Jul 07 15:14:27.484952 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:27.484823 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 07 15:14:27.485531 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:27.485516 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 07 15:14:27.485595 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:27.485541 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 07 15:14:27.502239 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:27.502212 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-5pxlr"] Jul 07 15:14:27.503424 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:27.503399 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-bthjj"] Jul 07 15:14:27.505632 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:27.505615 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-bthjj" Jul 07 15:14:27.508022 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:27.507950 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 07 15:14:27.508125 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:27.508110 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-fzzpb\"" Jul 07 15:14:27.508298 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:27.508285 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 07 15:14:27.509367 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:27.509351 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 07 15:14:27.588549 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:27.588514 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7745743b-cc63-477b-b67c-5f2f81872dcc-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bthjj\" (UID: \"7745743b-cc63-477b-b67c-5f2f81872dcc\") " pod="openshift-monitoring/node-exporter-bthjj" Jul 07 15:14:27.588549 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:27.588553 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvrlz\" (UniqueName: \"kubernetes.io/projected/45876fcf-512b-4537-a4f3-6c16985e00f2-kube-api-access-wvrlz\") pod \"openshift-state-metrics-54cf96dfcf-5pxlr\" (UID: \"45876fcf-512b-4537-a4f3-6c16985e00f2\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-5pxlr" Jul 07 15:14:27.588812 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:27.588573 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7745743b-cc63-477b-b67c-5f2f81872dcc-node-exporter-accelerators-collector-config\") pod \"node-exporter-bthjj\" (UID: \"7745743b-cc63-477b-b67c-5f2f81872dcc\") " pod="openshift-monitoring/node-exporter-bthjj" Jul 07 15:14:27.588812 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:27.588668 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/45876fcf-512b-4537-a4f3-6c16985e00f2-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-5pxlr\" (UID: \"45876fcf-512b-4537-a4f3-6c16985e00f2\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-5pxlr" Jul 07 15:14:27.588812 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:27.588724 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7745743b-cc63-477b-b67c-5f2f81872dcc-node-exporter-wtmp\") pod \"node-exporter-bthjj\" (UID: \"7745743b-cc63-477b-b67c-5f2f81872dcc\") " pod="openshift-monitoring/node-exporter-bthjj" Jul 07 15:14:27.588812 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:27.588749 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjvcn\" (UniqueName: \"kubernetes.io/projected/7745743b-cc63-477b-b67c-5f2f81872dcc-kube-api-access-kjvcn\") pod \"node-exporter-bthjj\" (UID: \"7745743b-cc63-477b-b67c-5f2f81872dcc\") " pod="openshift-monitoring/node-exporter-bthjj" Jul 07 15:14:27.588812 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:27.588795 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7745743b-cc63-477b-b67c-5f2f81872dcc-metrics-client-ca\") pod \"node-exporter-bthjj\" (UID: \"7745743b-cc63-477b-b67c-5f2f81872dcc\") " pod="openshift-monitoring/node-exporter-bthjj" Jul 07 15:14:27.589035 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:27.588867 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7745743b-cc63-477b-b67c-5f2f81872dcc-node-exporter-textfile\") pod \"node-exporter-bthjj\" (UID: \"7745743b-cc63-477b-b67c-5f2f81872dcc\") " pod="openshift-monitoring/node-exporter-bthjj" Jul 07 15:14:27.589035 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:27.588888 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7745743b-cc63-477b-b67c-5f2f81872dcc-node-exporter-tls\") pod \"node-exporter-bthjj\" (UID: \"7745743b-cc63-477b-b67c-5f2f81872dcc\") " pod="openshift-monitoring/node-exporter-bthjj" Jul 07 15:14:27.589035 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:27.588906 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/45876fcf-512b-4537-a4f3-6c16985e00f2-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-5pxlr\" (UID: \"45876fcf-512b-4537-a4f3-6c16985e00f2\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-5pxlr" Jul 07 15:14:27.589035 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:27.588925 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7745743b-cc63-477b-b67c-5f2f81872dcc-root\") pod \"node-exporter-bthjj\" (UID: \"7745743b-cc63-477b-b67c-5f2f81872dcc\") " pod="openshift-monitoring/node-exporter-bthjj" Jul 07 15:14:27.589035 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:27.588952 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7745743b-cc63-477b-b67c-5f2f81872dcc-sys\") pod \"node-exporter-bthjj\" (UID: \"7745743b-cc63-477b-b67c-5f2f81872dcc\") " pod="openshift-monitoring/node-exporter-bthjj" Jul 07 15:14:27.589035 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:27.588966 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45876fcf-512b-4537-a4f3-6c16985e00f2-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-5pxlr\" (UID: \"45876fcf-512b-4537-a4f3-6c16985e00f2\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-5pxlr" Jul 07 15:14:27.689920 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:27.689882 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7745743b-cc63-477b-b67c-5f2f81872dcc-node-exporter-accelerators-collector-config\") pod \"node-exporter-bthjj\" (UID: \"7745743b-cc63-477b-b67c-5f2f81872dcc\") " pod="openshift-monitoring/node-exporter-bthjj" Jul 07 15:14:27.689920 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:27.689924 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/45876fcf-512b-4537-a4f3-6c16985e00f2-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-5pxlr\" (UID: \"45876fcf-512b-4537-a4f3-6c16985e00f2\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-5pxlr" Jul 07 15:14:27.690175 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:27.689949 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7745743b-cc63-477b-b67c-5f2f81872dcc-node-exporter-wtmp\") pod \"node-exporter-bthjj\" (UID: \"7745743b-cc63-477b-b67c-5f2f81872dcc\") " pod="openshift-monitoring/node-exporter-bthjj" Jul 07 15:14:27.690175 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:27.689966 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjvcn\" (UniqueName: \"kubernetes.io/projected/7745743b-cc63-477b-b67c-5f2f81872dcc-kube-api-access-kjvcn\") pod \"node-exporter-bthjj\" (UID: \"7745743b-cc63-477b-b67c-5f2f81872dcc\") " pod="openshift-monitoring/node-exporter-bthjj" Jul 07 15:14:27.690175 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:27.689992 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7745743b-cc63-477b-b67c-5f2f81872dcc-metrics-client-ca\") pod \"node-exporter-bthjj\" (UID: \"7745743b-cc63-477b-b67c-5f2f81872dcc\") " pod="openshift-monitoring/node-exporter-bthjj" Jul 07 15:14:27.690175 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:27.690016 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7745743b-cc63-477b-b67c-5f2f81872dcc-node-exporter-textfile\") pod \"node-exporter-bthjj\" (UID: \"7745743b-cc63-477b-b67c-5f2f81872dcc\") " pod="openshift-monitoring/node-exporter-bthjj" Jul 07 15:14:27.690175 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:27.690031 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7745743b-cc63-477b-b67c-5f2f81872dcc-node-exporter-tls\") pod \"node-exporter-bthjj\" (UID: \"7745743b-cc63-477b-b67c-5f2f81872dcc\") " pod="openshift-monitoring/node-exporter-bthjj" Jul 07 15:14:27.690175 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:27.690050 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/45876fcf-512b-4537-a4f3-6c16985e00f2-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-5pxlr\" (UID: \"45876fcf-512b-4537-a4f3-6c16985e00f2\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-5pxlr" Jul 07 15:14:27.690175 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:14:27.690072 2549 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 07 15:14:27.690175 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:27.690111 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7745743b-cc63-477b-b67c-5f2f81872dcc-root\") pod \"node-exporter-bthjj\" (UID: \"7745743b-cc63-477b-b67c-5f2f81872dcc\") " pod="openshift-monitoring/node-exporter-bthjj" Jul 07 15:14:27.690175 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:27.690078 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7745743b-cc63-477b-b67c-5f2f81872dcc-node-exporter-wtmp\") pod \"node-exporter-bthjj\" (UID: \"7745743b-cc63-477b-b67c-5f2f81872dcc\") " pod="openshift-monitoring/node-exporter-bthjj" Jul 07 15:14:27.690175 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:14:27.690149 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45876fcf-512b-4537-a4f3-6c16985e00f2-openshift-state-metrics-tls podName:45876fcf-512b-4537-a4f3-6c16985e00f2 nodeName:}" failed. No retries permitted until 2026-07-07 15:14:28.190127966 +0000 UTC m=+86.197605395 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/45876fcf-512b-4537-a4f3-6c16985e00f2-openshift-state-metrics-tls") pod "openshift-state-metrics-54cf96dfcf-5pxlr" (UID: "45876fcf-512b-4537-a4f3-6c16985e00f2") : secret "openshift-state-metrics-tls" not found Jul 07 15:14:27.690175 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:27.690075 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7745743b-cc63-477b-b67c-5f2f81872dcc-root\") pod \"node-exporter-bthjj\" (UID: \"7745743b-cc63-477b-b67c-5f2f81872dcc\") " pod="openshift-monitoring/node-exporter-bthjj" Jul 07 15:14:27.690175 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:14:27.690167 2549 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 07 15:14:27.690777 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:27.690211 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7745743b-cc63-477b-b67c-5f2f81872dcc-sys\") pod \"node-exporter-bthjj\" (UID: \"7745743b-cc63-477b-b67c-5f2f81872dcc\") " pod="openshift-monitoring/node-exporter-bthjj" Jul 07 15:14:27.690777 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:14:27.690223 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7745743b-cc63-477b-b67c-5f2f81872dcc-node-exporter-tls podName:7745743b-cc63-477b-b67c-5f2f81872dcc nodeName:}" failed. No retries permitted until 2026-07-07 15:14:28.190206209 +0000 UTC m=+86.197683638 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/7745743b-cc63-477b-b67c-5f2f81872dcc-node-exporter-tls") pod "node-exporter-bthjj" (UID: "7745743b-cc63-477b-b67c-5f2f81872dcc") : secret "node-exporter-tls" not found Jul 07 15:14:27.690777 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:27.690254 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7745743b-cc63-477b-b67c-5f2f81872dcc-sys\") pod \"node-exporter-bthjj\" (UID: \"7745743b-cc63-477b-b67c-5f2f81872dcc\") " pod="openshift-monitoring/node-exporter-bthjj" Jul 07 15:14:27.690777 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:27.690259 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45876fcf-512b-4537-a4f3-6c16985e00f2-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-5pxlr\" (UID: \"45876fcf-512b-4537-a4f3-6c16985e00f2\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-5pxlr" Jul 07 15:14:27.690777 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:27.690309 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7745743b-cc63-477b-b67c-5f2f81872dcc-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bthjj\" (UID: \"7745743b-cc63-477b-b67c-5f2f81872dcc\") " pod="openshift-monitoring/node-exporter-bthjj" Jul 07 15:14:27.690777 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:27.690352 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvrlz\" (UniqueName: \"kubernetes.io/projected/45876fcf-512b-4537-a4f3-6c16985e00f2-kube-api-access-wvrlz\") pod \"openshift-state-metrics-54cf96dfcf-5pxlr\" (UID: \"45876fcf-512b-4537-a4f3-6c16985e00f2\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-5pxlr" Jul 07 15:14:27.690777 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:27.690508 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7745743b-cc63-477b-b67c-5f2f81872dcc-node-exporter-textfile\") pod \"node-exporter-bthjj\" (UID: \"7745743b-cc63-477b-b67c-5f2f81872dcc\") " pod="openshift-monitoring/node-exporter-bthjj" Jul 07 15:14:27.690777 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:27.690776 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7745743b-cc63-477b-b67c-5f2f81872dcc-metrics-client-ca\") pod \"node-exporter-bthjj\" (UID: \"7745743b-cc63-477b-b67c-5f2f81872dcc\") " pod="openshift-monitoring/node-exporter-bthjj" Jul 07 15:14:27.691034 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:27.690789 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7745743b-cc63-477b-b67c-5f2f81872dcc-node-exporter-accelerators-collector-config\") pod \"node-exporter-bthjj\" (UID: \"7745743b-cc63-477b-b67c-5f2f81872dcc\") " pod="openshift-monitoring/node-exporter-bthjj" Jul 07 15:14:27.691370 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:27.691349 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45876fcf-512b-4537-a4f3-6c16985e00f2-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-5pxlr\" (UID: \"45876fcf-512b-4537-a4f3-6c16985e00f2\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-5pxlr" Jul 07 15:14:27.692565 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:27.692542 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7745743b-cc63-477b-b67c-5f2f81872dcc-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bthjj\" (UID: \"7745743b-cc63-477b-b67c-5f2f81872dcc\") " pod="openshift-monitoring/node-exporter-bthjj" Jul 07 15:14:27.692715 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:27.692699 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/45876fcf-512b-4537-a4f3-6c16985e00f2-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-5pxlr\" (UID: \"45876fcf-512b-4537-a4f3-6c16985e00f2\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-5pxlr" Jul 07 15:14:27.702124 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:27.702090 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvrlz\" (UniqueName: \"kubernetes.io/projected/45876fcf-512b-4537-a4f3-6c16985e00f2-kube-api-access-wvrlz\") pod \"openshift-state-metrics-54cf96dfcf-5pxlr\" (UID: \"45876fcf-512b-4537-a4f3-6c16985e00f2\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-5pxlr" Jul 07 15:14:27.702451 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:27.702427 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjvcn\" (UniqueName: \"kubernetes.io/projected/7745743b-cc63-477b-b67c-5f2f81872dcc-kube-api-access-kjvcn\") pod \"node-exporter-bthjj\" (UID: \"7745743b-cc63-477b-b67c-5f2f81872dcc\") " pod="openshift-monitoring/node-exporter-bthjj" Jul 07 15:14:28.194771 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.194732 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7745743b-cc63-477b-b67c-5f2f81872dcc-node-exporter-tls\") pod \"node-exporter-bthjj\" (UID: \"7745743b-cc63-477b-b67c-5f2f81872dcc\") " pod="openshift-monitoring/node-exporter-bthjj" Jul 07 15:14:28.195265 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.194828 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/45876fcf-512b-4537-a4f3-6c16985e00f2-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-5pxlr\" (UID: \"45876fcf-512b-4537-a4f3-6c16985e00f2\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-5pxlr" Jul 07 15:14:28.195265 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:14:28.194946 2549 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 07 15:14:28.195265 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:14:28.195036 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45876fcf-512b-4537-a4f3-6c16985e00f2-openshift-state-metrics-tls podName:45876fcf-512b-4537-a4f3-6c16985e00f2 nodeName:}" failed. No retries permitted until 2026-07-07 15:14:29.19501644 +0000 UTC m=+87.202493883 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/45876fcf-512b-4537-a4f3-6c16985e00f2-openshift-state-metrics-tls") pod "openshift-state-metrics-54cf96dfcf-5pxlr" (UID: "45876fcf-512b-4537-a4f3-6c16985e00f2") : secret "openshift-state-metrics-tls" not found Jul 07 15:14:28.197093 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.197067 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7745743b-cc63-477b-b67c-5f2f81872dcc-node-exporter-tls\") pod \"node-exporter-bthjj\" (UID: \"7745743b-cc63-477b-b67c-5f2f81872dcc\") " pod="openshift-monitoring/node-exporter-bthjj" Jul 07 15:14:28.415218 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.415175 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-bthjj" Jul 07 15:14:28.427255 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:14:28.427230 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7745743b_cc63_477b_b67c_5f2f81872dcc.slice/crio-6d7ed29c9a04cdf8d74457922987cda3143306ebf1d17d448be154a5cce3ab48 WatchSource:0}: Error finding container 6d7ed29c9a04cdf8d74457922987cda3143306ebf1d17d448be154a5cce3ab48: Status 404 returned error can't find the container with id 6d7ed29c9a04cdf8d74457922987cda3143306ebf1d17d448be154a5cce3ab48 Jul 07 15:14:28.583112 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.583084 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 15:14:28.586432 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.586412 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:14:28.599627 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.599567 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 07 15:14:28.599761 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.599664 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 07 15:14:28.599761 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.599738 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 07 15:14:28.599899 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.599870 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 07 15:14:28.602958 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.602940 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 07 15:14:28.603064 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.603024 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 07 15:14:28.603126 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.602957 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 07 15:14:28.603126 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.602953 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 07 15:14:28.603126 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.602965 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 07 15:14:28.603262 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.602995 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-xjrrg\"" Jul 07 15:14:28.620202 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.620175 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 15:14:28.699463 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.699426 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b2e65842-a1c8-4e50-801e-b949cc3a410e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b2e65842-a1c8-4e50-801e-b949cc3a410e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:14:28.699463 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.699466 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b2e65842-a1c8-4e50-801e-b949cc3a410e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b2e65842-a1c8-4e50-801e-b949cc3a410e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:14:28.699690 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.699488 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b2e65842-a1c8-4e50-801e-b949cc3a410e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b2e65842-a1c8-4e50-801e-b949cc3a410e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:14:28.699690 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.699598 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b2e65842-a1c8-4e50-801e-b949cc3a410e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b2e65842-a1c8-4e50-801e-b949cc3a410e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:14:28.699690 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.699639 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b2e65842-a1c8-4e50-801e-b949cc3a410e-config-volume\") pod \"alertmanager-main-0\" (UID: \"b2e65842-a1c8-4e50-801e-b949cc3a410e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:14:28.699690 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.699658 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b2e65842-a1c8-4e50-801e-b949cc3a410e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b2e65842-a1c8-4e50-801e-b949cc3a410e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:14:28.699826 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.699715 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b2e65842-a1c8-4e50-801e-b949cc3a410e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b2e65842-a1c8-4e50-801e-b949cc3a410e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:14:28.699826 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.699734 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fjcs\" (UniqueName: \"kubernetes.io/projected/b2e65842-a1c8-4e50-801e-b949cc3a410e-kube-api-access-7fjcs\") pod \"alertmanager-main-0\" (UID: \"b2e65842-a1c8-4e50-801e-b949cc3a410e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:14:28.699826 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.699755 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b2e65842-a1c8-4e50-801e-b949cc3a410e-config-out\") pod \"alertmanager-main-0\" (UID: \"b2e65842-a1c8-4e50-801e-b949cc3a410e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:14:28.699826 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.699785 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b2e65842-a1c8-4e50-801e-b949cc3a410e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b2e65842-a1c8-4e50-801e-b949cc3a410e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:14:28.699826 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.699812 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b2e65842-a1c8-4e50-801e-b949cc3a410e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b2e65842-a1c8-4e50-801e-b949cc3a410e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:14:28.700023 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.699834 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b2e65842-a1c8-4e50-801e-b949cc3a410e-web-config\") pod \"alertmanager-main-0\" (UID: \"b2e65842-a1c8-4e50-801e-b949cc3a410e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:14:28.700023 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.699886 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b2e65842-a1c8-4e50-801e-b949cc3a410e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b2e65842-a1c8-4e50-801e-b949cc3a410e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:14:28.801183 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.801141 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b2e65842-a1c8-4e50-801e-b949cc3a410e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b2e65842-a1c8-4e50-801e-b949cc3a410e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:14:28.801357 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.801205 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b2e65842-a1c8-4e50-801e-b949cc3a410e-config-volume\") pod \"alertmanager-main-0\" (UID: \"b2e65842-a1c8-4e50-801e-b949cc3a410e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:14:28.801357 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.801230 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b2e65842-a1c8-4e50-801e-b949cc3a410e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b2e65842-a1c8-4e50-801e-b949cc3a410e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:14:28.801357 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.801270 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b2e65842-a1c8-4e50-801e-b949cc3a410e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b2e65842-a1c8-4e50-801e-b949cc3a410e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:14:28.801357 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.801295 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fjcs\" (UniqueName: \"kubernetes.io/projected/b2e65842-a1c8-4e50-801e-b949cc3a410e-kube-api-access-7fjcs\") pod \"alertmanager-main-0\" (UID: \"b2e65842-a1c8-4e50-801e-b949cc3a410e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:14:28.801357 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.801320 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b2e65842-a1c8-4e50-801e-b949cc3a410e-config-out\") pod \"alertmanager-main-0\" (UID: \"b2e65842-a1c8-4e50-801e-b949cc3a410e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:14:28.801357 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.801348 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b2e65842-a1c8-4e50-801e-b949cc3a410e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b2e65842-a1c8-4e50-801e-b949cc3a410e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:14:28.801703 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.801383 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b2e65842-a1c8-4e50-801e-b949cc3a410e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b2e65842-a1c8-4e50-801e-b949cc3a410e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:14:28.801703 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.801417 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b2e65842-a1c8-4e50-801e-b949cc3a410e-web-config\") pod \"alertmanager-main-0\" (UID: \"b2e65842-a1c8-4e50-801e-b949cc3a410e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:14:28.801703 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.801456 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b2e65842-a1c8-4e50-801e-b949cc3a410e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b2e65842-a1c8-4e50-801e-b949cc3a410e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:14:28.801703 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.801494 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b2e65842-a1c8-4e50-801e-b949cc3a410e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b2e65842-a1c8-4e50-801e-b949cc3a410e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:14:28.801703 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.801520 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b2e65842-a1c8-4e50-801e-b949cc3a410e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b2e65842-a1c8-4e50-801e-b949cc3a410e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:14:28.801703 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.801548 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b2e65842-a1c8-4e50-801e-b949cc3a410e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b2e65842-a1c8-4e50-801e-b949cc3a410e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:14:28.801977 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:14:28.801906 2549 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 07 15:14:28.802022 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:14:28.802005 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2e65842-a1c8-4e50-801e-b949cc3a410e-secret-alertmanager-main-tls podName:b2e65842-a1c8-4e50-801e-b949cc3a410e nodeName:}" failed. No retries permitted until 2026-07-07 15:14:29.301985285 +0000 UTC m=+87.309462715 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/b2e65842-a1c8-4e50-801e-b949cc3a410e-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "b2e65842-a1c8-4e50-801e-b949cc3a410e") : secret "alertmanager-main-tls" not found Jul 07 15:14:28.802087 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.802044 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b2e65842-a1c8-4e50-801e-b949cc3a410e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b2e65842-a1c8-4e50-801e-b949cc3a410e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:14:28.802364 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.802338 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b2e65842-a1c8-4e50-801e-b949cc3a410e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b2e65842-a1c8-4e50-801e-b949cc3a410e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:14:28.802857 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.802821 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b2e65842-a1c8-4e50-801e-b949cc3a410e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b2e65842-a1c8-4e50-801e-b949cc3a410e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:14:28.805533 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.805019 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b2e65842-a1c8-4e50-801e-b949cc3a410e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b2e65842-a1c8-4e50-801e-b949cc3a410e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:14:28.805533 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.805481 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b2e65842-a1c8-4e50-801e-b949cc3a410e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b2e65842-a1c8-4e50-801e-b949cc3a410e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:14:28.805533 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.805498 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b2e65842-a1c8-4e50-801e-b949cc3a410e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b2e65842-a1c8-4e50-801e-b949cc3a410e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:14:28.805533 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.805511 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b2e65842-a1c8-4e50-801e-b949cc3a410e-web-config\") pod \"alertmanager-main-0\" (UID: \"b2e65842-a1c8-4e50-801e-b949cc3a410e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:14:28.805920 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.805899 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b2e65842-a1c8-4e50-801e-b949cc3a410e-config-out\") pod \"alertmanager-main-0\" (UID: \"b2e65842-a1c8-4e50-801e-b949cc3a410e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:14:28.805983 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.805958 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b2e65842-a1c8-4e50-801e-b949cc3a410e-config-volume\") pod \"alertmanager-main-0\" (UID: \"b2e65842-a1c8-4e50-801e-b949cc3a410e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:14:28.806378 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.806356 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b2e65842-a1c8-4e50-801e-b949cc3a410e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b2e65842-a1c8-4e50-801e-b949cc3a410e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:14:28.807137 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.807117 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b2e65842-a1c8-4e50-801e-b949cc3a410e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b2e65842-a1c8-4e50-801e-b949cc3a410e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:14:28.811480 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.811457 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fjcs\" (UniqueName: \"kubernetes.io/projected/b2e65842-a1c8-4e50-801e-b949cc3a410e-kube-api-access-7fjcs\") pod \"alertmanager-main-0\" (UID: \"b2e65842-a1c8-4e50-801e-b949cc3a410e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:14:28.865482 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.865397 2549 patch_prober.go:28] interesting pod/image-registry-689c879f5d-clmr6 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 07 15:14:28.865482 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.865451 2549 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-689c879f5d-clmr6" podUID="22d3c83b-a693-4f0a-8ae2-07d83467c620" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 15:14:28.959314 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:28.959279 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bthjj" event={"ID":"7745743b-cc63-477b-b67c-5f2f81872dcc","Type":"ContainerStarted","Data":"6d7ed29c9a04cdf8d74457922987cda3143306ebf1d17d448be154a5cce3ab48"} Jul 07 15:14:29.204873 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:29.204789 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/45876fcf-512b-4537-a4f3-6c16985e00f2-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-5pxlr\" (UID: \"45876fcf-512b-4537-a4f3-6c16985e00f2\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-5pxlr" Jul 07 15:14:29.207117 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:29.207098 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/45876fcf-512b-4537-a4f3-6c16985e00f2-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-5pxlr\" (UID: \"45876fcf-512b-4537-a4f3-6c16985e00f2\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-5pxlr" Jul 07 15:14:29.291270 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:29.291231 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-5pxlr" Jul 07 15:14:29.305522 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:29.305488 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b2e65842-a1c8-4e50-801e-b949cc3a410e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b2e65842-a1c8-4e50-801e-b949cc3a410e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:14:29.308154 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:29.308129 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b2e65842-a1c8-4e50-801e-b949cc3a410e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b2e65842-a1c8-4e50-801e-b949cc3a410e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:14:29.440759 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:29.440716 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-5pxlr"] Jul 07 15:14:29.496068 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:29.495995 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:14:29.640483 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:14:29.640450 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45876fcf_512b_4537_a4f3_6c16985e00f2.slice/crio-abc70c06dd0bdda29f7707942905be364f21aa6c206bece923cbd5472bfdacfd WatchSource:0}: Error finding container abc70c06dd0bdda29f7707942905be364f21aa6c206bece923cbd5472bfdacfd: Status 404 returned error can't find the container with id abc70c06dd0bdda29f7707942905be364f21aa6c206bece923cbd5472bfdacfd Jul 07 15:14:29.796669 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:29.796645 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 15:14:29.804928 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:14:29.804884 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2e65842_a1c8_4e50_801e_b949cc3a410e.slice/crio-16e3a5a74a59703f7ebd76ea340baa19781aee42c88d883f559fd365136d9a9c WatchSource:0}: Error finding container 16e3a5a74a59703f7ebd76ea340baa19781aee42c88d883f559fd365136d9a9c: Status 404 returned error can't find the container with id 16e3a5a74a59703f7ebd76ea340baa19781aee42c88d883f559fd365136d9a9c Jul 07 15:14:29.965846 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:29.965813 2549 generic.go:356] "Generic (PLEG): container finished" podID="7745743b-cc63-477b-b67c-5f2f81872dcc" containerID="fc73fcf851c6efc657b65920667d89ae00b8c3b4357f8d931a50eaa9b0572e5f" exitCode=0 Jul 07 15:14:29.966038 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:29.965907 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bthjj" event={"ID":"7745743b-cc63-477b-b67c-5f2f81872dcc","Type":"ContainerDied","Data":"fc73fcf851c6efc657b65920667d89ae00b8c3b4357f8d931a50eaa9b0572e5f"} Jul 07 15:14:29.967945 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:29.967919 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-5pxlr" event={"ID":"45876fcf-512b-4537-a4f3-6c16985e00f2","Type":"ContainerStarted","Data":"aeed24eb552a43ebf1dc7ecf2a1d0d4b01613c4f622c327471bca6ed954371bf"} Jul 07 15:14:29.968062 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:29.967971 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-5pxlr" event={"ID":"45876fcf-512b-4537-a4f3-6c16985e00f2","Type":"ContainerStarted","Data":"933539196fa8a44d1a537fee786cf787325b661bcd0ab16f0ead0fcc20e34a65"} Jul 07 15:14:29.968062 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:29.967999 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-5pxlr" event={"ID":"45876fcf-512b-4537-a4f3-6c16985e00f2","Type":"ContainerStarted","Data":"abc70c06dd0bdda29f7707942905be364f21aa6c206bece923cbd5472bfdacfd"} Jul 07 15:14:29.969181 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:29.969159 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b2e65842-a1c8-4e50-801e-b949cc3a410e","Type":"ContainerStarted","Data":"16e3a5a74a59703f7ebd76ea340baa19781aee42c88d883f559fd365136d9a9c"} Jul 07 15:14:30.505263 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:30.505229 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-67656c78c5-x6f5l"] Jul 07 15:14:30.508850 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:30.508828 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-67656c78c5-x6f5l" Jul 07 15:14:30.513130 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:30.512976 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 07 15:14:30.513130 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:30.513062 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 07 15:14:30.513442 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:30.513418 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 07 15:14:30.513914 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:30.513530 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 07 15:14:30.513914 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:30.513673 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 07 15:14:30.513914 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:30.513688 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-76fab4n3flc0f\"" Jul 07 15:14:30.514113 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:30.513965 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-4bkd6\"" Jul 07 15:14:30.562058 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:30.562019 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-67656c78c5-x6f5l"] Jul 07 15:14:30.622002 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:30.621969 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2e8905ad-6c70-43de-aba9-19eda719d58b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-67656c78c5-x6f5l\" (UID: \"2e8905ad-6c70-43de-aba9-19eda719d58b\") " pod="openshift-monitoring/thanos-querier-67656c78c5-x6f5l" Jul 07 15:14:30.622002 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:30.622007 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2e8905ad-6c70-43de-aba9-19eda719d58b-metrics-client-ca\") pod \"thanos-querier-67656c78c5-x6f5l\" (UID: \"2e8905ad-6c70-43de-aba9-19eda719d58b\") " pod="openshift-monitoring/thanos-querier-67656c78c5-x6f5l" Jul 07 15:14:30.622230 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:30.622029 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4h4db\" (UniqueName: \"kubernetes.io/projected/2e8905ad-6c70-43de-aba9-19eda719d58b-kube-api-access-4h4db\") pod \"thanos-querier-67656c78c5-x6f5l\" (UID: \"2e8905ad-6c70-43de-aba9-19eda719d58b\") " pod="openshift-monitoring/thanos-querier-67656c78c5-x6f5l" Jul 07 15:14:30.622230 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:30.622053 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/2e8905ad-6c70-43de-aba9-19eda719d58b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-67656c78c5-x6f5l\" (UID: \"2e8905ad-6c70-43de-aba9-19eda719d58b\") " pod="openshift-monitoring/thanos-querier-67656c78c5-x6f5l" Jul 07 15:14:30.622230 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:30.622102 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2e8905ad-6c70-43de-aba9-19eda719d58b-secret-grpc-tls\") pod \"thanos-querier-67656c78c5-x6f5l\" (UID: \"2e8905ad-6c70-43de-aba9-19eda719d58b\") " pod="openshift-monitoring/thanos-querier-67656c78c5-x6f5l" Jul 07 15:14:30.622230 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:30.622137 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/2e8905ad-6c70-43de-aba9-19eda719d58b-secret-thanos-querier-tls\") pod \"thanos-querier-67656c78c5-x6f5l\" (UID: \"2e8905ad-6c70-43de-aba9-19eda719d58b\") " pod="openshift-monitoring/thanos-querier-67656c78c5-x6f5l" Jul 07 15:14:30.622230 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:30.622167 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/2e8905ad-6c70-43de-aba9-19eda719d58b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-67656c78c5-x6f5l\" (UID: \"2e8905ad-6c70-43de-aba9-19eda719d58b\") " pod="openshift-monitoring/thanos-querier-67656c78c5-x6f5l" Jul 07 15:14:30.622447 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:30.622238 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2e8905ad-6c70-43de-aba9-19eda719d58b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-67656c78c5-x6f5l\" (UID: \"2e8905ad-6c70-43de-aba9-19eda719d58b\") " pod="openshift-monitoring/thanos-querier-67656c78c5-x6f5l" Jul 07 15:14:30.723366 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:30.723313 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2e8905ad-6c70-43de-aba9-19eda719d58b-secret-grpc-tls\") pod \"thanos-querier-67656c78c5-x6f5l\" (UID: \"2e8905ad-6c70-43de-aba9-19eda719d58b\") " pod="openshift-monitoring/thanos-querier-67656c78c5-x6f5l" Jul 07 15:14:30.723520 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:30.723471 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/2e8905ad-6c70-43de-aba9-19eda719d58b-secret-thanos-querier-tls\") pod \"thanos-querier-67656c78c5-x6f5l\" (UID: \"2e8905ad-6c70-43de-aba9-19eda719d58b\") " pod="openshift-monitoring/thanos-querier-67656c78c5-x6f5l" Jul 07 15:14:30.723620 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:30.723518 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/2e8905ad-6c70-43de-aba9-19eda719d58b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-67656c78c5-x6f5l\" (UID: \"2e8905ad-6c70-43de-aba9-19eda719d58b\") " pod="openshift-monitoring/thanos-querier-67656c78c5-x6f5l" Jul 07 15:14:30.723620 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:30.723570 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2e8905ad-6c70-43de-aba9-19eda719d58b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-67656c78c5-x6f5l\" (UID: \"2e8905ad-6c70-43de-aba9-19eda719d58b\") " pod="openshift-monitoring/thanos-querier-67656c78c5-x6f5l" Jul 07 15:14:30.723737 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:30.723637 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2e8905ad-6c70-43de-aba9-19eda719d58b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-67656c78c5-x6f5l\" (UID: \"2e8905ad-6c70-43de-aba9-19eda719d58b\") " pod="openshift-monitoring/thanos-querier-67656c78c5-x6f5l" Jul 07 15:14:30.723737 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:30.723673 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2e8905ad-6c70-43de-aba9-19eda719d58b-metrics-client-ca\") pod \"thanos-querier-67656c78c5-x6f5l\" (UID: \"2e8905ad-6c70-43de-aba9-19eda719d58b\") " pod="openshift-monitoring/thanos-querier-67656c78c5-x6f5l" Jul 07 15:14:30.723737 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:30.723707 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4h4db\" (UniqueName: \"kubernetes.io/projected/2e8905ad-6c70-43de-aba9-19eda719d58b-kube-api-access-4h4db\") pod \"thanos-querier-67656c78c5-x6f5l\" (UID: \"2e8905ad-6c70-43de-aba9-19eda719d58b\") " pod="openshift-monitoring/thanos-querier-67656c78c5-x6f5l" Jul 07 15:14:30.723868 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:30.723742 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/2e8905ad-6c70-43de-aba9-19eda719d58b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-67656c78c5-x6f5l\" (UID: \"2e8905ad-6c70-43de-aba9-19eda719d58b\") " pod="openshift-monitoring/thanos-querier-67656c78c5-x6f5l" Jul 07 15:14:30.724831 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:30.724753 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2e8905ad-6c70-43de-aba9-19eda719d58b-metrics-client-ca\") pod \"thanos-querier-67656c78c5-x6f5l\" (UID: \"2e8905ad-6c70-43de-aba9-19eda719d58b\") " pod="openshift-monitoring/thanos-querier-67656c78c5-x6f5l" Jul 07 15:14:30.726453 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:30.726425 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/2e8905ad-6c70-43de-aba9-19eda719d58b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-67656c78c5-x6f5l\" (UID: \"2e8905ad-6c70-43de-aba9-19eda719d58b\") " pod="openshift-monitoring/thanos-querier-67656c78c5-x6f5l" Jul 07 15:14:30.726686 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:30.726660 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2e8905ad-6c70-43de-aba9-19eda719d58b-secret-grpc-tls\") pod \"thanos-querier-67656c78c5-x6f5l\" (UID: \"2e8905ad-6c70-43de-aba9-19eda719d58b\") " pod="openshift-monitoring/thanos-querier-67656c78c5-x6f5l" Jul 07 15:14:30.726767 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:30.726698 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/2e8905ad-6c70-43de-aba9-19eda719d58b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-67656c78c5-x6f5l\" (UID: \"2e8905ad-6c70-43de-aba9-19eda719d58b\") " pod="openshift-monitoring/thanos-querier-67656c78c5-x6f5l" Jul 07 15:14:30.727395 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:30.727371 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2e8905ad-6c70-43de-aba9-19eda719d58b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-67656c78c5-x6f5l\" (UID: \"2e8905ad-6c70-43de-aba9-19eda719d58b\") " pod="openshift-monitoring/thanos-querier-67656c78c5-x6f5l" Jul 07 15:14:30.727493 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:30.727382 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/2e8905ad-6c70-43de-aba9-19eda719d58b-secret-thanos-querier-tls\") pod \"thanos-querier-67656c78c5-x6f5l\" (UID: \"2e8905ad-6c70-43de-aba9-19eda719d58b\") " pod="openshift-monitoring/thanos-querier-67656c78c5-x6f5l" Jul 07 15:14:30.727569 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:30.727550 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2e8905ad-6c70-43de-aba9-19eda719d58b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-67656c78c5-x6f5l\" (UID: \"2e8905ad-6c70-43de-aba9-19eda719d58b\") " pod="openshift-monitoring/thanos-querier-67656c78c5-x6f5l" Jul 07 15:14:30.732443 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:30.732420 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4h4db\" (UniqueName: \"kubernetes.io/projected/2e8905ad-6c70-43de-aba9-19eda719d58b-kube-api-access-4h4db\") pod \"thanos-querier-67656c78c5-x6f5l\" (UID: \"2e8905ad-6c70-43de-aba9-19eda719d58b\") " pod="openshift-monitoring/thanos-querier-67656c78c5-x6f5l" Jul 07 15:14:30.822285 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:30.822243 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-67656c78c5-x6f5l" Jul 07 15:14:30.870817 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:30.870790 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-689c879f5d-clmr6" Jul 07 15:14:30.976970 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:30.976661 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bthjj" event={"ID":"7745743b-cc63-477b-b67c-5f2f81872dcc","Type":"ContainerStarted","Data":"5008878b02903051925fb85be51e58fc34ab0b6786d9ccb861502c23e7615f8e"} Jul 07 15:14:30.976970 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:30.976739 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bthjj" event={"ID":"7745743b-cc63-477b-b67c-5f2f81872dcc","Type":"ContainerStarted","Data":"b228507c52755894f0dd0d75c251a6eff24e7cfe89b7f8732f5aba8e0cc7ba59"} Jul 07 15:14:30.979710 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:30.979685 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-5pxlr" event={"ID":"45876fcf-512b-4537-a4f3-6c16985e00f2","Type":"ContainerStarted","Data":"698986718ee13965582e9bf71d4e23ee1fdcb5a0f50e54947f2991b1d810a022"} Jul 07 15:14:30.985725 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:30.985671 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-67656c78c5-x6f5l"] Jul 07 15:14:30.988177 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:14:30.988155 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e8905ad_6c70_43de_aba9_19eda719d58b.slice/crio-a5057515ce44830fc1ddaaa4aa15c638c66edeffa938e15d9ab6347d0f925002 WatchSource:0}: Error finding container a5057515ce44830fc1ddaaa4aa15c638c66edeffa938e15d9ab6347d0f925002: Status 404 returned error can't find the container with id a5057515ce44830fc1ddaaa4aa15c638c66edeffa938e15d9ab6347d0f925002 Jul 07 15:14:30.998429 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:30.998366 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-bthjj" podStartSLOduration=2.677025549 podStartE2EDuration="3.998353331s" totalImagesPullingTime="1.244844743s" totalInitContainerRuntime="76.483039ms" isStatefulPod=true podCreationTimestamp="2026-07-07 15:14:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:14:30.997451319 +0000 UTC m=+89.004928818" watchObservedRunningTime="2026-07-07 15:14:30.998353331 +0000 UTC m=+89.005830824" Jul 07 15:14:31.018716 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:31.018669 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-5pxlr" podStartSLOduration=2.95255616 podStartE2EDuration="4.018652494s" totalImagesPullingTime="1.066096334s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:14:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:14:31.01794355 +0000 UTC m=+89.025421001" watchObservedRunningTime="2026-07-07 15:14:31.018652494 +0000 UTC m=+89.026129946" Jul 07 15:14:31.854026 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:31.853991 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6894f65b55-6lfnz"] Jul 07 15:14:31.856317 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:31.856290 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6894f65b55-6lfnz" Jul 07 15:14:31.861178 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:31.861155 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 07 15:14:31.861314 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:31.861292 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 07 15:14:31.861997 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:31.861972 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 07 15:14:31.862108 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:31.861976 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 07 15:14:31.862108 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:31.862019 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-3l4f5fb59r08d\"" Jul 07 15:14:31.862108 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:31.862038 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-v75gx\"" Jul 07 15:14:31.873317 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:31.873296 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6894f65b55-6lfnz"] Jul 07 15:14:31.985207 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:31.985167 2549 generic.go:356] "Generic (PLEG): container finished" podID="b2e65842-a1c8-4e50-801e-b949cc3a410e" containerID="6e28c5a9ed93e91b87cee99182e66454e2bdcedd69ab6994688a772cac7b674d" exitCode=0 Jul 07 15:14:31.985372 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:31.985254 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b2e65842-a1c8-4e50-801e-b949cc3a410e","Type":"ContainerDied","Data":"6e28c5a9ed93e91b87cee99182e66454e2bdcedd69ab6994688a772cac7b674d"} Jul 07 15:14:31.986497 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:31.986475 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-67656c78c5-x6f5l" event={"ID":"2e8905ad-6c70-43de-aba9-19eda719d58b","Type":"ContainerStarted","Data":"a5057515ce44830fc1ddaaa4aa15c638c66edeffa938e15d9ab6347d0f925002"} Jul 07 15:14:32.036941 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:32.036911 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c8d6306d-fa3e-49e3-964e-9666a18a5477-secret-metrics-server-client-certs\") pod \"metrics-server-6894f65b55-6lfnz\" (UID: \"c8d6306d-fa3e-49e3-964e-9666a18a5477\") " pod="openshift-monitoring/metrics-server-6894f65b55-6lfnz" Jul 07 15:14:32.036941 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:32.036946 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c8d6306d-fa3e-49e3-964e-9666a18a5477-audit-log\") pod \"metrics-server-6894f65b55-6lfnz\" (UID: \"c8d6306d-fa3e-49e3-964e-9666a18a5477\") " pod="openshift-monitoring/metrics-server-6894f65b55-6lfnz" Jul 07 15:14:32.037132 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:32.036965 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c8d6306d-fa3e-49e3-964e-9666a18a5477-secret-metrics-server-tls\") pod \"metrics-server-6894f65b55-6lfnz\" (UID: \"c8d6306d-fa3e-49e3-964e-9666a18a5477\") " pod="openshift-monitoring/metrics-server-6894f65b55-6lfnz" Jul 07 15:14:32.037132 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:32.036986 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdv6h\" (UniqueName: \"kubernetes.io/projected/c8d6306d-fa3e-49e3-964e-9666a18a5477-kube-api-access-sdv6h\") pod \"metrics-server-6894f65b55-6lfnz\" (UID: \"c8d6306d-fa3e-49e3-964e-9666a18a5477\") " pod="openshift-monitoring/metrics-server-6894f65b55-6lfnz" Jul 07 15:14:32.037132 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:32.037008 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c8d6306d-fa3e-49e3-964e-9666a18a5477-metrics-server-audit-profiles\") pod \"metrics-server-6894f65b55-6lfnz\" (UID: \"c8d6306d-fa3e-49e3-964e-9666a18a5477\") " pod="openshift-monitoring/metrics-server-6894f65b55-6lfnz" Jul 07 15:14:32.037132 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:32.037115 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c8d6306d-fa3e-49e3-964e-9666a18a5477-client-ca-bundle\") pod \"metrics-server-6894f65b55-6lfnz\" (UID: \"c8d6306d-fa3e-49e3-964e-9666a18a5477\") " pod="openshift-monitoring/metrics-server-6894f65b55-6lfnz" Jul 07 15:14:32.037283 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:32.037157 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8d6306d-fa3e-49e3-964e-9666a18a5477-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6894f65b55-6lfnz\" (UID: \"c8d6306d-fa3e-49e3-964e-9666a18a5477\") " pod="openshift-monitoring/metrics-server-6894f65b55-6lfnz" Jul 07 15:14:32.138129 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:32.137692 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c8d6306d-fa3e-49e3-964e-9666a18a5477-secret-metrics-server-client-certs\") pod \"metrics-server-6894f65b55-6lfnz\" (UID: \"c8d6306d-fa3e-49e3-964e-9666a18a5477\") " pod="openshift-monitoring/metrics-server-6894f65b55-6lfnz" Jul 07 15:14:32.138129 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:32.137770 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c8d6306d-fa3e-49e3-964e-9666a18a5477-audit-log\") pod \"metrics-server-6894f65b55-6lfnz\" (UID: \"c8d6306d-fa3e-49e3-964e-9666a18a5477\") " pod="openshift-monitoring/metrics-server-6894f65b55-6lfnz" Jul 07 15:14:32.138129 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:32.137831 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c8d6306d-fa3e-49e3-964e-9666a18a5477-secret-metrics-server-tls\") pod \"metrics-server-6894f65b55-6lfnz\" (UID: \"c8d6306d-fa3e-49e3-964e-9666a18a5477\") " pod="openshift-monitoring/metrics-server-6894f65b55-6lfnz" Jul 07 15:14:32.138129 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:32.137860 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdv6h\" (UniqueName: \"kubernetes.io/projected/c8d6306d-fa3e-49e3-964e-9666a18a5477-kube-api-access-sdv6h\") pod \"metrics-server-6894f65b55-6lfnz\" (UID: \"c8d6306d-fa3e-49e3-964e-9666a18a5477\") " pod="openshift-monitoring/metrics-server-6894f65b55-6lfnz" Jul 07 15:14:32.138129 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:32.137904 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c8d6306d-fa3e-49e3-964e-9666a18a5477-metrics-server-audit-profiles\") pod \"metrics-server-6894f65b55-6lfnz\" (UID: \"c8d6306d-fa3e-49e3-964e-9666a18a5477\") " pod="openshift-monitoring/metrics-server-6894f65b55-6lfnz" Jul 07 15:14:32.138129 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:32.138124 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c8d6306d-fa3e-49e3-964e-9666a18a5477-client-ca-bundle\") pod \"metrics-server-6894f65b55-6lfnz\" (UID: \"c8d6306d-fa3e-49e3-964e-9666a18a5477\") " pod="openshift-monitoring/metrics-server-6894f65b55-6lfnz" Jul 07 15:14:32.138716 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:32.138234 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8d6306d-fa3e-49e3-964e-9666a18a5477-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6894f65b55-6lfnz\" (UID: \"c8d6306d-fa3e-49e3-964e-9666a18a5477\") " pod="openshift-monitoring/metrics-server-6894f65b55-6lfnz" Jul 07 15:14:32.139323 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:32.139293 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c8d6306d-fa3e-49e3-964e-9666a18a5477-audit-log\") pod \"metrics-server-6894f65b55-6lfnz\" (UID: \"c8d6306d-fa3e-49e3-964e-9666a18a5477\") " pod="openshift-monitoring/metrics-server-6894f65b55-6lfnz" Jul 07 15:14:32.141431 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:32.141405 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8d6306d-fa3e-49e3-964e-9666a18a5477-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6894f65b55-6lfnz\" (UID: \"c8d6306d-fa3e-49e3-964e-9666a18a5477\") " pod="openshift-monitoring/metrics-server-6894f65b55-6lfnz" Jul 07 15:14:32.143034 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:32.142959 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c8d6306d-fa3e-49e3-964e-9666a18a5477-secret-metrics-server-client-certs\") pod \"metrics-server-6894f65b55-6lfnz\" (UID: \"c8d6306d-fa3e-49e3-964e-9666a18a5477\") " pod="openshift-monitoring/metrics-server-6894f65b55-6lfnz" Jul 07 15:14:32.151005 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:32.144555 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c8d6306d-fa3e-49e3-964e-9666a18a5477-metrics-server-audit-profiles\") pod \"metrics-server-6894f65b55-6lfnz\" (UID: \"c8d6306d-fa3e-49e3-964e-9666a18a5477\") " pod="openshift-monitoring/metrics-server-6894f65b55-6lfnz" Jul 07 15:14:32.151005 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:32.146098 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c8d6306d-fa3e-49e3-964e-9666a18a5477-secret-metrics-server-tls\") pod \"metrics-server-6894f65b55-6lfnz\" (UID: \"c8d6306d-fa3e-49e3-964e-9666a18a5477\") " pod="openshift-monitoring/metrics-server-6894f65b55-6lfnz" Jul 07 15:14:32.151005 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:32.150506 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdv6h\" (UniqueName: \"kubernetes.io/projected/c8d6306d-fa3e-49e3-964e-9666a18a5477-kube-api-access-sdv6h\") pod \"metrics-server-6894f65b55-6lfnz\" (UID: \"c8d6306d-fa3e-49e3-964e-9666a18a5477\") " pod="openshift-monitoring/metrics-server-6894f65b55-6lfnz" Jul 07 15:14:32.154186 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:32.154151 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c8d6306d-fa3e-49e3-964e-9666a18a5477-client-ca-bundle\") pod \"metrics-server-6894f65b55-6lfnz\" (UID: \"c8d6306d-fa3e-49e3-964e-9666a18a5477\") " pod="openshift-monitoring/metrics-server-6894f65b55-6lfnz" Jul 07 15:14:32.167558 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:32.167528 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6894f65b55-6lfnz" Jul 07 15:14:32.313450 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:32.313350 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6894f65b55-6lfnz"] Jul 07 15:14:32.316183 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:14:32.316077 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8d6306d_fa3e_49e3_964e_9666a18a5477.slice/crio-4911e6f46ce8a390ba355db414455b9b53c6ba5f44c1c665beb9b32066f1f629 WatchSource:0}: Error finding container 4911e6f46ce8a390ba355db414455b9b53c6ba5f44c1c665beb9b32066f1f629: Status 404 returned error can't find the container with id 4911e6f46ce8a390ba355db414455b9b53c6ba5f44c1c665beb9b32066f1f629 Jul 07 15:14:32.993456 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:32.993415 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6894f65b55-6lfnz" event={"ID":"c8d6306d-fa3e-49e3-964e-9666a18a5477","Type":"ContainerStarted","Data":"4911e6f46ce8a390ba355db414455b9b53c6ba5f44c1c665beb9b32066f1f629"} Jul 07 15:14:34.001594 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:34.001553 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-67656c78c5-x6f5l" event={"ID":"2e8905ad-6c70-43de-aba9-19eda719d58b","Type":"ContainerStarted","Data":"0fe023e2179ea5098e45dbbf9c15cf85157544d0d2dd35c6199b9454d428ddfe"} Jul 07 15:14:34.002079 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:34.001657 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-67656c78c5-x6f5l" event={"ID":"2e8905ad-6c70-43de-aba9-19eda719d58b","Type":"ContainerStarted","Data":"ca8563253d3ea6a56fef82577053246eecffa2dab03e84dde69feb489555aa2d"} Jul 07 15:14:34.002079 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:34.001702 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-67656c78c5-x6f5l" event={"ID":"2e8905ad-6c70-43de-aba9-19eda719d58b","Type":"ContainerStarted","Data":"46d68d4ec860938ad917b9894ecbb0fcd044984d84323fcb7abb2635b86770c0"} Jul 07 15:14:35.016067 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:35.016022 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b2e65842-a1c8-4e50-801e-b949cc3a410e","Type":"ContainerStarted","Data":"bf94db224dc6b4b4289e1b888e000ca8121354a2ef330b3248a8d34755c17db3"} Jul 07 15:14:35.016406 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:35.016159 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b2e65842-a1c8-4e50-801e-b949cc3a410e","Type":"ContainerStarted","Data":"3064c74a4f9db75a445fc4762815ec9bf6ed992ff805562bc28ebd75e704262e"} Jul 07 15:14:35.016406 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:35.016209 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b2e65842-a1c8-4e50-801e-b949cc3a410e","Type":"ContainerStarted","Data":"35c70690c7b95d59e392303a868cef85334eab1af0326bf19577f38a30e5a808"} Jul 07 15:14:35.016406 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:35.016257 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b2e65842-a1c8-4e50-801e-b949cc3a410e","Type":"ContainerStarted","Data":"7f38e590b961779527f985850610948329752bfe67208ef1e2d7e629570cf4ca"} Jul 07 15:14:35.016406 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:35.016302 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b2e65842-a1c8-4e50-801e-b949cc3a410e","Type":"ContainerStarted","Data":"0577cbf008f9fc82e6910fb991b845188dceb869fae2b7ea8f1b65005d378dbc"} Jul 07 15:14:35.019920 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:35.019882 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6894f65b55-6lfnz" event={"ID":"c8d6306d-fa3e-49e3-964e-9666a18a5477","Type":"ContainerStarted","Data":"6e30a0d2d75f3f0fa280d4bd79864824db2d955d45104d331ef1bef53bf971b4"} Jul 07 15:14:35.041860 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:35.041780 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6894f65b55-6lfnz" podStartSLOduration=1.820448904 podStartE2EDuration="4.041759621s" totalImagesPullingTime="2.221310717s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:14:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:14:35.039908953 +0000 UTC m=+93.047386401" watchObservedRunningTime="2026-07-07 15:14:35.041759621 +0000 UTC m=+93.049237072" Jul 07 15:14:35.884389 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:35.884357 2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-689c879f5d-clmr6"] Jul 07 15:14:36.027376 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:36.027343 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b2e65842-a1c8-4e50-801e-b949cc3a410e","Type":"ContainerStarted","Data":"ecffba129361af7f4955403120ebc05565a6fd1f0a33a3e005c0e709b8813adb"} Jul 07 15:14:36.030969 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:36.030941 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-67656c78c5-x6f5l" event={"ID":"2e8905ad-6c70-43de-aba9-19eda719d58b","Type":"ContainerStarted","Data":"beb6984f15b49597dcf7bb196069bf75efe41485ec431282ec9fb6a0a445a7e8"} Jul 07 15:14:36.031125 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:36.031015 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-67656c78c5-x6f5l" event={"ID":"2e8905ad-6c70-43de-aba9-19eda719d58b","Type":"ContainerStarted","Data":"d315d46f15bd47c5bcfc90391fc3b98fe3724e06cdc27cdde7802a51a51a65a7"} Jul 07 15:14:36.031125 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:36.031045 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-67656c78c5-x6f5l" event={"ID":"2e8905ad-6c70-43de-aba9-19eda719d58b","Type":"ContainerStarted","Data":"fc9317950db441fa84018f10aac35fee0eb96e25789442ba32385873336174d1"} Jul 07 15:14:36.071547 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:36.071493 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.390615309 podStartE2EDuration="8.071477734s" totalImagesPullingTime="4.664943216s" totalInitContainerRuntime="15.919209ms" isStatefulPod=false podCreationTimestamp="2026-07-07 15:14:28 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:14:36.070251415 +0000 UTC m=+94.077728904" watchObservedRunningTime="2026-07-07 15:14:36.071477734 +0000 UTC m=+94.078955235" Jul 07 15:14:36.104135 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:36.104082 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-67656c78c5-x6f5l" podStartSLOduration=2.276175131 podStartE2EDuration="6.104063548s" totalImagesPullingTime="3.827888417s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:14:30 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:14:36.102000137 +0000 UTC m=+94.109477627" watchObservedRunningTime="2026-07-07 15:14:36.104063548 +0000 UTC m=+94.111541033" Jul 07 15:14:37.034915 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:37.034886 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-67656c78c5-x6f5l" Jul 07 15:14:38.047636 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:38.047609 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-67656c78c5-x6f5l" Jul 07 15:14:39.863683 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:39.863648 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-9dwk4" Jul 07 15:14:40.502291 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:40.502259 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-57b986b797-tf2fq"] Jul 07 15:14:40.505153 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:40.505136 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57b986b797-tf2fq" Jul 07 15:14:40.512598 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:40.512365 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bc6442d-60c1-4296-ba74-aeb9a087c67b-trusted-ca-bundle\") pod \"console-57b986b797-tf2fq\" (UID: \"2bc6442d-60c1-4296-ba74-aeb9a087c67b\") " pod="openshift-console/console-57b986b797-tf2fq" Jul 07 15:14:40.512598 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:40.512438 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2bc6442d-60c1-4296-ba74-aeb9a087c67b-oauth-serving-cert\") pod \"console-57b986b797-tf2fq\" (UID: \"2bc6442d-60c1-4296-ba74-aeb9a087c67b\") " pod="openshift-console/console-57b986b797-tf2fq" Jul 07 15:14:40.512598 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:40.512488 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2bc6442d-60c1-4296-ba74-aeb9a087c67b-service-ca\") pod \"console-57b986b797-tf2fq\" (UID: \"2bc6442d-60c1-4296-ba74-aeb9a087c67b\") " pod="openshift-console/console-57b986b797-tf2fq" Jul 07 15:14:40.512598 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:40.512567 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxbj6\" (UniqueName: \"kubernetes.io/projected/2bc6442d-60c1-4296-ba74-aeb9a087c67b-kube-api-access-jxbj6\") pod \"console-57b986b797-tf2fq\" (UID: \"2bc6442d-60c1-4296-ba74-aeb9a087c67b\") " pod="openshift-console/console-57b986b797-tf2fq" Jul 07 15:14:40.512830 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:40.512619 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2bc6442d-60c1-4296-ba74-aeb9a087c67b-console-config\") pod \"console-57b986b797-tf2fq\" (UID: \"2bc6442d-60c1-4296-ba74-aeb9a087c67b\") " pod="openshift-console/console-57b986b797-tf2fq" Jul 07 15:14:40.512830 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:40.512655 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2bc6442d-60c1-4296-ba74-aeb9a087c67b-console-serving-cert\") pod \"console-57b986b797-tf2fq\" (UID: \"2bc6442d-60c1-4296-ba74-aeb9a087c67b\") " pod="openshift-console/console-57b986b797-tf2fq" Jul 07 15:14:40.512830 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:40.512678 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2bc6442d-60c1-4296-ba74-aeb9a087c67b-console-oauth-config\") pod \"console-57b986b797-tf2fq\" (UID: \"2bc6442d-60c1-4296-ba74-aeb9a087c67b\") " pod="openshift-console/console-57b986b797-tf2fq" Jul 07 15:14:40.530771 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:40.530746 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 07 15:14:40.530908 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:40.530822 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 07 15:14:40.530908 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:40.530822 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 07 15:14:40.531542 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:40.531524 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 07 15:14:40.566396 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:40.566366 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 07 15:14:40.567198 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:40.567173 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 07 15:14:40.567310 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:40.567180 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-nqx2c\"" Jul 07 15:14:40.567794 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:40.567778 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 07 15:14:40.569631 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:40.569614 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57b986b797-tf2fq"] Jul 07 15:14:40.605715 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:40.605693 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 07 15:14:40.613127 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:40.613107 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2bc6442d-60c1-4296-ba74-aeb9a087c67b-service-ca\") pod \"console-57b986b797-tf2fq\" (UID: \"2bc6442d-60c1-4296-ba74-aeb9a087c67b\") " pod="openshift-console/console-57b986b797-tf2fq" Jul 07 15:14:40.613233 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:40.613137 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxbj6\" (UniqueName: \"kubernetes.io/projected/2bc6442d-60c1-4296-ba74-aeb9a087c67b-kube-api-access-jxbj6\") pod \"console-57b986b797-tf2fq\" (UID: \"2bc6442d-60c1-4296-ba74-aeb9a087c67b\") " pod="openshift-console/console-57b986b797-tf2fq" Jul 07 15:14:40.613233 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:40.613161 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2bc6442d-60c1-4296-ba74-aeb9a087c67b-console-config\") pod \"console-57b986b797-tf2fq\" (UID: \"2bc6442d-60c1-4296-ba74-aeb9a087c67b\") " pod="openshift-console/console-57b986b797-tf2fq" Jul 07 15:14:40.613233 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:40.613181 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2bc6442d-60c1-4296-ba74-aeb9a087c67b-console-serving-cert\") pod \"console-57b986b797-tf2fq\" (UID: \"2bc6442d-60c1-4296-ba74-aeb9a087c67b\") " pod="openshift-console/console-57b986b797-tf2fq" Jul 07 15:14:40.613233 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:40.613205 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2bc6442d-60c1-4296-ba74-aeb9a087c67b-console-oauth-config\") pod \"console-57b986b797-tf2fq\" (UID: \"2bc6442d-60c1-4296-ba74-aeb9a087c67b\") " pod="openshift-console/console-57b986b797-tf2fq" Jul 07 15:14:40.613418 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:40.613259 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bc6442d-60c1-4296-ba74-aeb9a087c67b-trusted-ca-bundle\") pod \"console-57b986b797-tf2fq\" (UID: \"2bc6442d-60c1-4296-ba74-aeb9a087c67b\") " pod="openshift-console/console-57b986b797-tf2fq" Jul 07 15:14:40.613418 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:40.613312 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2bc6442d-60c1-4296-ba74-aeb9a087c67b-oauth-serving-cert\") pod \"console-57b986b797-tf2fq\" (UID: \"2bc6442d-60c1-4296-ba74-aeb9a087c67b\") " pod="openshift-console/console-57b986b797-tf2fq" Jul 07 15:14:40.613971 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:40.613951 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2bc6442d-60c1-4296-ba74-aeb9a087c67b-service-ca\") pod \"console-57b986b797-tf2fq\" (UID: \"2bc6442d-60c1-4296-ba74-aeb9a087c67b\") " pod="openshift-console/console-57b986b797-tf2fq" Jul 07 15:14:40.614119 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:40.614039 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2bc6442d-60c1-4296-ba74-aeb9a087c67b-oauth-serving-cert\") pod \"console-57b986b797-tf2fq\" (UID: \"2bc6442d-60c1-4296-ba74-aeb9a087c67b\") " pod="openshift-console/console-57b986b797-tf2fq" Jul 07 15:14:40.614212 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:40.614130 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bc6442d-60c1-4296-ba74-aeb9a087c67b-trusted-ca-bundle\") pod \"console-57b986b797-tf2fq\" (UID: \"2bc6442d-60c1-4296-ba74-aeb9a087c67b\") " pod="openshift-console/console-57b986b797-tf2fq" Jul 07 15:14:40.614308 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:40.614261 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2bc6442d-60c1-4296-ba74-aeb9a087c67b-console-config\") pod \"console-57b986b797-tf2fq\" (UID: \"2bc6442d-60c1-4296-ba74-aeb9a087c67b\") " pod="openshift-console/console-57b986b797-tf2fq" Jul 07 15:14:40.616203 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:40.616175 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2bc6442d-60c1-4296-ba74-aeb9a087c67b-console-serving-cert\") pod \"console-57b986b797-tf2fq\" (UID: \"2bc6442d-60c1-4296-ba74-aeb9a087c67b\") " pod="openshift-console/console-57b986b797-tf2fq" Jul 07 15:14:40.616285 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:40.616225 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2bc6442d-60c1-4296-ba74-aeb9a087c67b-console-oauth-config\") pod \"console-57b986b797-tf2fq\" (UID: \"2bc6442d-60c1-4296-ba74-aeb9a087c67b\") " pod="openshift-console/console-57b986b797-tf2fq" Jul 07 15:14:40.668770 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:40.668738 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxbj6\" (UniqueName: \"kubernetes.io/projected/2bc6442d-60c1-4296-ba74-aeb9a087c67b-kube-api-access-jxbj6\") pod \"console-57b986b797-tf2fq\" (UID: \"2bc6442d-60c1-4296-ba74-aeb9a087c67b\") " pod="openshift-console/console-57b986b797-tf2fq" Jul 07 15:14:40.815159 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:40.815125 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57b986b797-tf2fq" Jul 07 15:14:40.978499 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:40.978467 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57b986b797-tf2fq"] Jul 07 15:14:40.982418 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:14:40.982386 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bc6442d_60c1_4296_ba74_aeb9a087c67b.slice/crio-d170ca05b73b84282d770ebf16fbb92e0d4c7e70f8b7ddd374030b9d1a140e94 WatchSource:0}: Error finding container d170ca05b73b84282d770ebf16fbb92e0d4c7e70f8b7ddd374030b9d1a140e94: Status 404 returned error can't find the container with id d170ca05b73b84282d770ebf16fbb92e0d4c7e70f8b7ddd374030b9d1a140e94 Jul 07 15:14:41.054136 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:41.054089 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57b986b797-tf2fq" event={"ID":"2bc6442d-60c1-4296-ba74-aeb9a087c67b","Type":"ContainerStarted","Data":"d170ca05b73b84282d770ebf16fbb92e0d4c7e70f8b7ddd374030b9d1a140e94"} Jul 07 15:14:45.075219 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:45.075179 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57b986b797-tf2fq" event={"ID":"2bc6442d-60c1-4296-ba74-aeb9a087c67b","Type":"ContainerStarted","Data":"4c842d163561414225a97479f53a5e6b1387e5a9e9b26e73971f67b280b506d2"} Jul 07 15:14:50.815685 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:50.815640 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-57b986b797-tf2fq" Jul 07 15:14:50.816188 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:50.815707 2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-57b986b797-tf2fq" Jul 07 15:14:50.820558 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:50.820528 2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-57b986b797-tf2fq" Jul 07 15:14:50.846641 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:50.846569 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-57b986b797-tf2fq" podStartSLOduration=7.71226991 podStartE2EDuration="10.846553982s" totalImagesPullingTime="3.134284072s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:14:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:14:45.119920413 +0000 UTC m=+103.127397863" watchObservedRunningTime="2026-07-07 15:14:50.846553982 +0000 UTC m=+108.854031432" Jul 07 15:14:51.103452 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:51.103378 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-57b986b797-tf2fq" Jul 07 15:14:52.167725 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:52.167674 2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6894f65b55-6lfnz" Jul 07 15:14:52.168157 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:52.167758 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6894f65b55-6lfnz" Jul 07 15:14:53.110168 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:53.110135 2549 generic.go:356] "Generic (PLEG): container finished" podID="8056aeaf-ce8a-4d1d-a6b6-c0df02057284" containerID="0554bd916eeb0b28bc46c9f67b259f43d5011908edc77bbec5efbe1a422e64a4" exitCode=0 Jul 07 15:14:53.110331 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:53.110209 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-wdtl4" event={"ID":"8056aeaf-ce8a-4d1d-a6b6-c0df02057284","Type":"ContainerDied","Data":"0554bd916eeb0b28bc46c9f67b259f43d5011908edc77bbec5efbe1a422e64a4"} Jul 07 15:14:53.110535 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:53.110522 2549 scope.go:117] "RemoveContainer" containerID="0554bd916eeb0b28bc46c9f67b259f43d5011908edc77bbec5efbe1a422e64a4" Jul 07 15:14:54.116482 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:14:54.116446 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-wdtl4" event={"ID":"8056aeaf-ce8a-4d1d-a6b6-c0df02057284","Type":"ContainerStarted","Data":"dca0737d9561977b7e6c674cc632fa9a2221d29b18748e702333f0950cf01f74"} Jul 07 15:15:00.904660 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:00.904571 2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-689c879f5d-clmr6" podUID="22d3c83b-a693-4f0a-8ae2-07d83467c620" containerName="registry" containerID="cri-o://de6e500aa9a3c4ff9709aa23213388baa66768696c8532aeff266f74597a38e5" gracePeriod=30 Jul 07 15:15:01.146690 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:01.146663 2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-689c879f5d-clmr6" Jul 07 15:15:01.147149 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:01.147119 2549 generic.go:356] "Generic (PLEG): container finished" podID="22d3c83b-a693-4f0a-8ae2-07d83467c620" containerID="de6e500aa9a3c4ff9709aa23213388baa66768696c8532aeff266f74597a38e5" exitCode=0 Jul 07 15:15:01.147291 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:01.147191 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-689c879f5d-clmr6" event={"ID":"22d3c83b-a693-4f0a-8ae2-07d83467c620","Type":"ContainerDied","Data":"de6e500aa9a3c4ff9709aa23213388baa66768696c8532aeff266f74597a38e5"} Jul 07 15:15:01.147291 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:01.147255 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-689c879f5d-clmr6" event={"ID":"22d3c83b-a693-4f0a-8ae2-07d83467c620","Type":"ContainerDied","Data":"b56eee08114041845c2541ba6ac09723a3749b5b28a560ab833e0cf652d792f1"} Jul 07 15:15:01.147291 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:01.147283 2549 scope.go:117] "RemoveContainer" containerID="de6e500aa9a3c4ff9709aa23213388baa66768696c8532aeff266f74597a38e5" Jul 07 15:15:01.155638 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:01.155619 2549 scope.go:117] "RemoveContainer" containerID="de6e500aa9a3c4ff9709aa23213388baa66768696c8532aeff266f74597a38e5" Jul 07 15:15:01.155907 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:15:01.155887 2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de6e500aa9a3c4ff9709aa23213388baa66768696c8532aeff266f74597a38e5\": container with ID starting with de6e500aa9a3c4ff9709aa23213388baa66768696c8532aeff266f74597a38e5 not found: ID does not exist" containerID="de6e500aa9a3c4ff9709aa23213388baa66768696c8532aeff266f74597a38e5" Jul 07 15:15:01.155961 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:01.155918 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de6e500aa9a3c4ff9709aa23213388baa66768696c8532aeff266f74597a38e5"} err="failed to get container status \"de6e500aa9a3c4ff9709aa23213388baa66768696c8532aeff266f74597a38e5\": rpc error: code = NotFound desc = could not find container \"de6e500aa9a3c4ff9709aa23213388baa66768696c8532aeff266f74597a38e5\": container with ID starting with de6e500aa9a3c4ff9709aa23213388baa66768696c8532aeff266f74597a38e5 not found: ID does not exist" Jul 07 15:15:01.192158 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:01.192130 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/22d3c83b-a693-4f0a-8ae2-07d83467c620-trusted-ca\") pod \"22d3c83b-a693-4f0a-8ae2-07d83467c620\" (UID: \"22d3c83b-a693-4f0a-8ae2-07d83467c620\") " Jul 07 15:15:01.192312 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:01.192187 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/22d3c83b-a693-4f0a-8ae2-07d83467c620-ca-trust-extracted\") pod \"22d3c83b-a693-4f0a-8ae2-07d83467c620\" (UID: \"22d3c83b-a693-4f0a-8ae2-07d83467c620\") " Jul 07 15:15:01.192312 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:01.192221 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/22d3c83b-a693-4f0a-8ae2-07d83467c620-bound-sa-token\") pod \"22d3c83b-a693-4f0a-8ae2-07d83467c620\" (UID: \"22d3c83b-a693-4f0a-8ae2-07d83467c620\") " Jul 07 15:15:01.192312 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:01.192251 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/22d3c83b-a693-4f0a-8ae2-07d83467c620-image-registry-private-configuration\") pod \"22d3c83b-a693-4f0a-8ae2-07d83467c620\" (UID: \"22d3c83b-a693-4f0a-8ae2-07d83467c620\") " Jul 07 15:15:01.192312 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:01.192286 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zflwn\" (UniqueName: \"kubernetes.io/projected/22d3c83b-a693-4f0a-8ae2-07d83467c620-kube-api-access-zflwn\") pod \"22d3c83b-a693-4f0a-8ae2-07d83467c620\" (UID: \"22d3c83b-a693-4f0a-8ae2-07d83467c620\") " Jul 07 15:15:01.192518 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:01.192351 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/22d3c83b-a693-4f0a-8ae2-07d83467c620-installation-pull-secrets\") pod \"22d3c83b-a693-4f0a-8ae2-07d83467c620\" (UID: \"22d3c83b-a693-4f0a-8ae2-07d83467c620\") " Jul 07 15:15:01.192518 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:01.192387 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/22d3c83b-a693-4f0a-8ae2-07d83467c620-registry-certificates\") pod \"22d3c83b-a693-4f0a-8ae2-07d83467c620\" (UID: \"22d3c83b-a693-4f0a-8ae2-07d83467c620\") " Jul 07 15:15:01.192518 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:01.192425 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/22d3c83b-a693-4f0a-8ae2-07d83467c620-registry-tls\") pod \"22d3c83b-a693-4f0a-8ae2-07d83467c620\" (UID: \"22d3c83b-a693-4f0a-8ae2-07d83467c620\") " Jul 07 15:15:01.192707 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:01.192569 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22d3c83b-a693-4f0a-8ae2-07d83467c620-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "22d3c83b-a693-4f0a-8ae2-07d83467c620" (UID: "22d3c83b-a693-4f0a-8ae2-07d83467c620"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:15:01.192767 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:01.192753 2549 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/22d3c83b-a693-4f0a-8ae2-07d83467c620-trusted-ca\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:15:01.193244 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:01.193204 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22d3c83b-a693-4f0a-8ae2-07d83467c620-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "22d3c83b-a693-4f0a-8ae2-07d83467c620" (UID: "22d3c83b-a693-4f0a-8ae2-07d83467c620"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:15:01.195126 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:01.195094 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22d3c83b-a693-4f0a-8ae2-07d83467c620-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "22d3c83b-a693-4f0a-8ae2-07d83467c620" (UID: "22d3c83b-a693-4f0a-8ae2-07d83467c620"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:15:01.195236 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:01.195149 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22d3c83b-a693-4f0a-8ae2-07d83467c620-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "22d3c83b-a693-4f0a-8ae2-07d83467c620" (UID: "22d3c83b-a693-4f0a-8ae2-07d83467c620"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:15:01.195300 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:01.195273 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22d3c83b-a693-4f0a-8ae2-07d83467c620-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "22d3c83b-a693-4f0a-8ae2-07d83467c620" (UID: "22d3c83b-a693-4f0a-8ae2-07d83467c620"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:15:01.195497 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:01.195472 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22d3c83b-a693-4f0a-8ae2-07d83467c620-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "22d3c83b-a693-4f0a-8ae2-07d83467c620" (UID: "22d3c83b-a693-4f0a-8ae2-07d83467c620"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:15:01.195563 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:01.195538 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22d3c83b-a693-4f0a-8ae2-07d83467c620-kube-api-access-zflwn" (OuterVolumeSpecName: "kube-api-access-zflwn") pod "22d3c83b-a693-4f0a-8ae2-07d83467c620" (UID: "22d3c83b-a693-4f0a-8ae2-07d83467c620"). InnerVolumeSpecName "kube-api-access-zflwn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:15:01.201431 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:01.201402 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22d3c83b-a693-4f0a-8ae2-07d83467c620-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "22d3c83b-a693-4f0a-8ae2-07d83467c620" (UID: "22d3c83b-a693-4f0a-8ae2-07d83467c620"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:15:01.293446 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:01.293413 2549 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/22d3c83b-a693-4f0a-8ae2-07d83467c620-installation-pull-secrets\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:15:01.293446 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:01.293439 2549 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/22d3c83b-a693-4f0a-8ae2-07d83467c620-registry-certificates\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:15:01.293446 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:01.293449 2549 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/22d3c83b-a693-4f0a-8ae2-07d83467c620-registry-tls\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:15:01.293446 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:01.293457 2549 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/22d3c83b-a693-4f0a-8ae2-07d83467c620-ca-trust-extracted\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:15:01.293729 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:01.293466 2549 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/22d3c83b-a693-4f0a-8ae2-07d83467c620-bound-sa-token\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:15:01.293729 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:01.293476 2549 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/22d3c83b-a693-4f0a-8ae2-07d83467c620-image-registry-private-configuration\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:15:01.293729 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:01.293485 2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zflwn\" (UniqueName: \"kubernetes.io/projected/22d3c83b-a693-4f0a-8ae2-07d83467c620-kube-api-access-zflwn\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:15:02.152054 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:02.152027 2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-689c879f5d-clmr6" Jul 07 15:15:02.182166 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:02.182137 2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-689c879f5d-clmr6"] Jul 07 15:15:02.190061 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:02.190037 2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-689c879f5d-clmr6"] Jul 07 15:15:02.539653 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:02.539559 2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22d3c83b-a693-4f0a-8ae2-07d83467c620" path="/var/lib/kubelet/pods/22d3c83b-a693-4f0a-8ae2-07d83467c620/volumes" Jul 07 15:15:12.173480 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:12.173450 2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6894f65b55-6lfnz" Jul 07 15:15:12.177325 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:12.177294 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6894f65b55-6lfnz" Jul 07 15:15:18.223836 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:18.223804 2549 generic.go:356] "Generic (PLEG): container finished" podID="665ed2cb-7aaf-42ab-8731-03e7cea27fc4" containerID="abcd8f8a9c9509e5bd3b61083fe484c238ad10705875605d8975e1830db2e160" exitCode=0 Jul 07 15:15:18.224316 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:18.223876 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-88b42" event={"ID":"665ed2cb-7aaf-42ab-8731-03e7cea27fc4","Type":"ContainerDied","Data":"abcd8f8a9c9509e5bd3b61083fe484c238ad10705875605d8975e1830db2e160"} Jul 07 15:15:18.224316 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:18.224241 2549 scope.go:117] "RemoveContainer" containerID="abcd8f8a9c9509e5bd3b61083fe484c238ad10705875605d8975e1830db2e160" Jul 07 15:15:18.225332 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:18.225310 2549 generic.go:356] "Generic (PLEG): container finished" podID="6bd34c29-c215-4974-98fd-7759cb30f6b8" containerID="dd229155a484ee0e23f23e0cd5197e41645ad3b3b67114907ab8e81eb262fb77" exitCode=0 Jul 07 15:15:18.225419 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:18.225336 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-ntm4v" event={"ID":"6bd34c29-c215-4974-98fd-7759cb30f6b8","Type":"ContainerDied","Data":"dd229155a484ee0e23f23e0cd5197e41645ad3b3b67114907ab8e81eb262fb77"} Jul 07 15:15:18.225648 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:18.225635 2549 scope.go:117] "RemoveContainer" containerID="dd229155a484ee0e23f23e0cd5197e41645ad3b3b67114907ab8e81eb262fb77" Jul 07 15:15:19.231486 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:19.231453 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-ntm4v" event={"ID":"6bd34c29-c215-4974-98fd-7759cb30f6b8","Type":"ContainerStarted","Data":"5c14ca1d1729478e26feb10d769f15a4e1f0a90d0afe93301692cfba38d7039f"} Jul 07 15:15:19.233203 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:19.233182 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-88b42" event={"ID":"665ed2cb-7aaf-42ab-8731-03e7cea27fc4","Type":"ContainerStarted","Data":"6942f9e466325c78c6f68f84b7be435799238bd25f3bc5048e08a4e8147c842b"} Jul 07 15:15:48.077325 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:48.077292 2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 15:15:48.077899 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:48.077595 2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b2e65842-a1c8-4e50-801e-b949cc3a410e" containerName="alertmanager" containerID="cri-o://0577cbf008f9fc82e6910fb991b845188dceb869fae2b7ea8f1b65005d378dbc" gracePeriod=120 Jul 07 15:15:48.077899 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:48.077632 2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b2e65842-a1c8-4e50-801e-b949cc3a410e" containerName="kube-rbac-proxy-metric" containerID="cri-o://bf94db224dc6b4b4289e1b888e000ca8121354a2ef330b3248a8d34755c17db3" gracePeriod=120 Jul 07 15:15:48.077899 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:48.077697 2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b2e65842-a1c8-4e50-801e-b949cc3a410e" containerName="prom-label-proxy" containerID="cri-o://ecffba129361af7f4955403120ebc05565a6fd1f0a33a3e005c0e709b8813adb" gracePeriod=120 Jul 07 15:15:48.077899 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:48.077718 2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b2e65842-a1c8-4e50-801e-b949cc3a410e" containerName="kube-rbac-proxy-web" containerID="cri-o://35c70690c7b95d59e392303a868cef85334eab1af0326bf19577f38a30e5a808" gracePeriod=120 Jul 07 15:15:48.077899 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:48.077725 2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b2e65842-a1c8-4e50-801e-b949cc3a410e" containerName="config-reloader" containerID="cri-o://7f38e590b961779527f985850610948329752bfe67208ef1e2d7e629570cf4ca" gracePeriod=120 Jul 07 15:15:48.077899 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:48.077722 2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b2e65842-a1c8-4e50-801e-b949cc3a410e" containerName="kube-rbac-proxy" containerID="cri-o://3064c74a4f9db75a445fc4762815ec9bf6ed992ff805562bc28ebd75e704262e" gracePeriod=120 Jul 07 15:15:48.373952 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:48.373865 2549 generic.go:356] "Generic (PLEG): container finished" podID="b2e65842-a1c8-4e50-801e-b949cc3a410e" containerID="ecffba129361af7f4955403120ebc05565a6fd1f0a33a3e005c0e709b8813adb" exitCode=0 Jul 07 15:15:48.373952 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:48.373891 2549 generic.go:356] "Generic (PLEG): container finished" podID="b2e65842-a1c8-4e50-801e-b949cc3a410e" containerID="3064c74a4f9db75a445fc4762815ec9bf6ed992ff805562bc28ebd75e704262e" exitCode=0 Jul 07 15:15:48.373952 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:48.373897 2549 generic.go:356] "Generic (PLEG): container finished" podID="b2e65842-a1c8-4e50-801e-b949cc3a410e" containerID="7f38e590b961779527f985850610948329752bfe67208ef1e2d7e629570cf4ca" exitCode=0 Jul 07 15:15:48.373952 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:48.373905 2549 generic.go:356] "Generic (PLEG): container finished" podID="b2e65842-a1c8-4e50-801e-b949cc3a410e" containerID="0577cbf008f9fc82e6910fb991b845188dceb869fae2b7ea8f1b65005d378dbc" exitCode=0 Jul 07 15:15:48.373952 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:48.373937 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b2e65842-a1c8-4e50-801e-b949cc3a410e","Type":"ContainerDied","Data":"ecffba129361af7f4955403120ebc05565a6fd1f0a33a3e005c0e709b8813adb"} Jul 07 15:15:48.374242 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:48.374017 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b2e65842-a1c8-4e50-801e-b949cc3a410e","Type":"ContainerDied","Data":"3064c74a4f9db75a445fc4762815ec9bf6ed992ff805562bc28ebd75e704262e"} Jul 07 15:15:48.374242 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:48.374057 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b2e65842-a1c8-4e50-801e-b949cc3a410e","Type":"ContainerDied","Data":"7f38e590b961779527f985850610948329752bfe67208ef1e2d7e629570cf4ca"} Jul 07 15:15:48.374242 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:48.374092 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b2e65842-a1c8-4e50-801e-b949cc3a410e","Type":"ContainerDied","Data":"0577cbf008f9fc82e6910fb991b845188dceb869fae2b7ea8f1b65005d378dbc"} Jul 07 15:15:49.320172 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.320145 2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:15:49.382504 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.382424 2549 generic.go:356] "Generic (PLEG): container finished" podID="b2e65842-a1c8-4e50-801e-b949cc3a410e" containerID="bf94db224dc6b4b4289e1b888e000ca8121354a2ef330b3248a8d34755c17db3" exitCode=0 Jul 07 15:15:49.382504 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.382449 2549 generic.go:356] "Generic (PLEG): container finished" podID="b2e65842-a1c8-4e50-801e-b949cc3a410e" containerID="35c70690c7b95d59e392303a868cef85334eab1af0326bf19577f38a30e5a808" exitCode=0 Jul 07 15:15:49.382504 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.382468 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b2e65842-a1c8-4e50-801e-b949cc3a410e","Type":"ContainerDied","Data":"bf94db224dc6b4b4289e1b888e000ca8121354a2ef330b3248a8d34755c17db3"} Jul 07 15:15:49.382504 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.382498 2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:15:49.382800 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.382531 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b2e65842-a1c8-4e50-801e-b949cc3a410e","Type":"ContainerDied","Data":"35c70690c7b95d59e392303a868cef85334eab1af0326bf19577f38a30e5a808"} Jul 07 15:15:49.382800 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.382559 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b2e65842-a1c8-4e50-801e-b949cc3a410e","Type":"ContainerDied","Data":"16e3a5a74a59703f7ebd76ea340baa19781aee42c88d883f559fd365136d9a9c"} Jul 07 15:15:49.382800 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.382608 2549 scope.go:117] "RemoveContainer" containerID="ecffba129361af7f4955403120ebc05565a6fd1f0a33a3e005c0e709b8813adb" Jul 07 15:15:49.392105 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.392087 2549 scope.go:117] "RemoveContainer" containerID="bf94db224dc6b4b4289e1b888e000ca8121354a2ef330b3248a8d34755c17db3" Jul 07 15:15:49.399148 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.399125 2549 scope.go:117] "RemoveContainer" containerID="3064c74a4f9db75a445fc4762815ec9bf6ed992ff805562bc28ebd75e704262e" Jul 07 15:15:49.407437 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.407420 2549 scope.go:117] "RemoveContainer" containerID="35c70690c7b95d59e392303a868cef85334eab1af0326bf19577f38a30e5a808" Jul 07 15:15:49.414717 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.414699 2549 scope.go:117] "RemoveContainer" containerID="7f38e590b961779527f985850610948329752bfe67208ef1e2d7e629570cf4ca" Jul 07 15:15:49.423097 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.423074 2549 scope.go:117] "RemoveContainer" containerID="0577cbf008f9fc82e6910fb991b845188dceb869fae2b7ea8f1b65005d378dbc" Jul 07 15:15:49.425252 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.425232 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b2e65842-a1c8-4e50-801e-b949cc3a410e-web-config\") pod \"b2e65842-a1c8-4e50-801e-b949cc3a410e\" (UID: \"b2e65842-a1c8-4e50-801e-b949cc3a410e\") " Jul 07 15:15:49.425371 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.425287 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b2e65842-a1c8-4e50-801e-b949cc3a410e-alertmanager-trusted-ca-bundle\") pod \"b2e65842-a1c8-4e50-801e-b949cc3a410e\" (UID: \"b2e65842-a1c8-4e50-801e-b949cc3a410e\") " Jul 07 15:15:49.425371 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.425316 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b2e65842-a1c8-4e50-801e-b949cc3a410e-cluster-tls-config\") pod \"b2e65842-a1c8-4e50-801e-b949cc3a410e\" (UID: \"b2e65842-a1c8-4e50-801e-b949cc3a410e\") " Jul 07 15:15:49.425371 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.425337 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b2e65842-a1c8-4e50-801e-b949cc3a410e-config-out\") pod \"b2e65842-a1c8-4e50-801e-b949cc3a410e\" (UID: \"b2e65842-a1c8-4e50-801e-b949cc3a410e\") " Jul 07 15:15:49.425371 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.425363 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7fjcs\" (UniqueName: \"kubernetes.io/projected/b2e65842-a1c8-4e50-801e-b949cc3a410e-kube-api-access-7fjcs\") pod \"b2e65842-a1c8-4e50-801e-b949cc3a410e\" (UID: \"b2e65842-a1c8-4e50-801e-b949cc3a410e\") " Jul 07 15:15:49.425595 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.425393 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b2e65842-a1c8-4e50-801e-b949cc3a410e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"b2e65842-a1c8-4e50-801e-b949cc3a410e\" (UID: \"b2e65842-a1c8-4e50-801e-b949cc3a410e\") " Jul 07 15:15:49.425595 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.425422 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b2e65842-a1c8-4e50-801e-b949cc3a410e-metrics-client-ca\") pod \"b2e65842-a1c8-4e50-801e-b949cc3a410e\" (UID: \"b2e65842-a1c8-4e50-801e-b949cc3a410e\") " Jul 07 15:15:49.425595 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.425447 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b2e65842-a1c8-4e50-801e-b949cc3a410e-alertmanager-main-db\") pod \"b2e65842-a1c8-4e50-801e-b949cc3a410e\" (UID: \"b2e65842-a1c8-4e50-801e-b949cc3a410e\") " Jul 07 15:15:49.425595 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.425491 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b2e65842-a1c8-4e50-801e-b949cc3a410e-secret-alertmanager-kube-rbac-proxy\") pod \"b2e65842-a1c8-4e50-801e-b949cc3a410e\" (UID: \"b2e65842-a1c8-4e50-801e-b949cc3a410e\") " Jul 07 15:15:49.425595 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.425521 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b2e65842-a1c8-4e50-801e-b949cc3a410e-secret-alertmanager-kube-rbac-proxy-web\") pod \"b2e65842-a1c8-4e50-801e-b949cc3a410e\" (UID: \"b2e65842-a1c8-4e50-801e-b949cc3a410e\") " Jul 07 15:15:49.425595 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.425544 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b2e65842-a1c8-4e50-801e-b949cc3a410e-tls-assets\") pod \"b2e65842-a1c8-4e50-801e-b949cc3a410e\" (UID: \"b2e65842-a1c8-4e50-801e-b949cc3a410e\") " Jul 07 15:15:49.425893 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.425627 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b2e65842-a1c8-4e50-801e-b949cc3a410e-config-volume\") pod \"b2e65842-a1c8-4e50-801e-b949cc3a410e\" (UID: \"b2e65842-a1c8-4e50-801e-b949cc3a410e\") " Jul 07 15:15:49.425893 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.425654 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b2e65842-a1c8-4e50-801e-b949cc3a410e-secret-alertmanager-main-tls\") pod \"b2e65842-a1c8-4e50-801e-b949cc3a410e\" (UID: \"b2e65842-a1c8-4e50-801e-b949cc3a410e\") " Jul 07 15:15:49.425893 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.425796 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2e65842-a1c8-4e50-801e-b949cc3a410e-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "b2e65842-a1c8-4e50-801e-b949cc3a410e" (UID: "b2e65842-a1c8-4e50-801e-b949cc3a410e"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:15:49.426046 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.425922 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2e65842-a1c8-4e50-801e-b949cc3a410e-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "b2e65842-a1c8-4e50-801e-b949cc3a410e" (UID: "b2e65842-a1c8-4e50-801e-b949cc3a410e"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:15:49.426046 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.425948 2549 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b2e65842-a1c8-4e50-801e-b949cc3a410e-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:15:49.426349 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.426236 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2e65842-a1c8-4e50-801e-b949cc3a410e-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "b2e65842-a1c8-4e50-801e-b949cc3a410e" (UID: "b2e65842-a1c8-4e50-801e-b949cc3a410e"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:15:49.430526 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.430485 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2e65842-a1c8-4e50-801e-b949cc3a410e-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "b2e65842-a1c8-4e50-801e-b949cc3a410e" (UID: "b2e65842-a1c8-4e50-801e-b949cc3a410e"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:15:49.430814 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.430768 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2e65842-a1c8-4e50-801e-b949cc3a410e-kube-api-access-7fjcs" (OuterVolumeSpecName: "kube-api-access-7fjcs") pod "b2e65842-a1c8-4e50-801e-b949cc3a410e" (UID: "b2e65842-a1c8-4e50-801e-b949cc3a410e"). InnerVolumeSpecName "kube-api-access-7fjcs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:15:49.431680 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.430813 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2e65842-a1c8-4e50-801e-b949cc3a410e-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "b2e65842-a1c8-4e50-801e-b949cc3a410e" (UID: "b2e65842-a1c8-4e50-801e-b949cc3a410e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:15:49.431680 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.430841 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2e65842-a1c8-4e50-801e-b949cc3a410e-config-out" (OuterVolumeSpecName: "config-out") pod "b2e65842-a1c8-4e50-801e-b949cc3a410e" (UID: "b2e65842-a1c8-4e50-801e-b949cc3a410e"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:15:49.431819 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.431743 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2e65842-a1c8-4e50-801e-b949cc3a410e-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "b2e65842-a1c8-4e50-801e-b949cc3a410e" (UID: "b2e65842-a1c8-4e50-801e-b949cc3a410e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:15:49.431819 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.431747 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2e65842-a1c8-4e50-801e-b949cc3a410e-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "b2e65842-a1c8-4e50-801e-b949cc3a410e" (UID: "b2e65842-a1c8-4e50-801e-b949cc3a410e"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:15:49.431819 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.431763 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2e65842-a1c8-4e50-801e-b949cc3a410e-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "b2e65842-a1c8-4e50-801e-b949cc3a410e" (UID: "b2e65842-a1c8-4e50-801e-b949cc3a410e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:15:49.432133 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.432114 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2e65842-a1c8-4e50-801e-b949cc3a410e-config-volume" (OuterVolumeSpecName: "config-volume") pod "b2e65842-a1c8-4e50-801e-b949cc3a410e" (UID: "b2e65842-a1c8-4e50-801e-b949cc3a410e"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:15:49.433459 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.433414 2549 scope.go:117] "RemoveContainer" containerID="6e28c5a9ed93e91b87cee99182e66454e2bdcedd69ab6994688a772cac7b674d" Jul 07 15:15:49.435373 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.435349 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2e65842-a1c8-4e50-801e-b949cc3a410e-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "b2e65842-a1c8-4e50-801e-b949cc3a410e" (UID: "b2e65842-a1c8-4e50-801e-b949cc3a410e"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:15:49.443493 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.443469 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2e65842-a1c8-4e50-801e-b949cc3a410e-web-config" (OuterVolumeSpecName: "web-config") pod "b2e65842-a1c8-4e50-801e-b949cc3a410e" (UID: "b2e65842-a1c8-4e50-801e-b949cc3a410e"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:15:49.446138 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.446114 2549 scope.go:117] "RemoveContainer" containerID="ecffba129361af7f4955403120ebc05565a6fd1f0a33a3e005c0e709b8813adb" Jul 07 15:15:49.446412 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:15:49.446391 2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ecffba129361af7f4955403120ebc05565a6fd1f0a33a3e005c0e709b8813adb\": container with ID starting with ecffba129361af7f4955403120ebc05565a6fd1f0a33a3e005c0e709b8813adb not found: ID does not exist" containerID="ecffba129361af7f4955403120ebc05565a6fd1f0a33a3e005c0e709b8813adb" Jul 07 15:15:49.446478 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.446420 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ecffba129361af7f4955403120ebc05565a6fd1f0a33a3e005c0e709b8813adb"} err="failed to get container status \"ecffba129361af7f4955403120ebc05565a6fd1f0a33a3e005c0e709b8813adb\": rpc error: code = NotFound desc = could not find container \"ecffba129361af7f4955403120ebc05565a6fd1f0a33a3e005c0e709b8813adb\": container with ID starting with ecffba129361af7f4955403120ebc05565a6fd1f0a33a3e005c0e709b8813adb not found: ID does not exist" Jul 07 15:15:49.446478 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.446441 2549 scope.go:117] "RemoveContainer" containerID="bf94db224dc6b4b4289e1b888e000ca8121354a2ef330b3248a8d34755c17db3" Jul 07 15:15:49.446703 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:15:49.446687 2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf94db224dc6b4b4289e1b888e000ca8121354a2ef330b3248a8d34755c17db3\": container with ID starting with bf94db224dc6b4b4289e1b888e000ca8121354a2ef330b3248a8d34755c17db3 not found: ID does not exist" containerID="bf94db224dc6b4b4289e1b888e000ca8121354a2ef330b3248a8d34755c17db3" Jul 07 15:15:49.446767 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.446709 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf94db224dc6b4b4289e1b888e000ca8121354a2ef330b3248a8d34755c17db3"} err="failed to get container status \"bf94db224dc6b4b4289e1b888e000ca8121354a2ef330b3248a8d34755c17db3\": rpc error: code = NotFound desc = could not find container \"bf94db224dc6b4b4289e1b888e000ca8121354a2ef330b3248a8d34755c17db3\": container with ID starting with bf94db224dc6b4b4289e1b888e000ca8121354a2ef330b3248a8d34755c17db3 not found: ID does not exist" Jul 07 15:15:49.446767 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.446726 2549 scope.go:117] "RemoveContainer" containerID="3064c74a4f9db75a445fc4762815ec9bf6ed992ff805562bc28ebd75e704262e" Jul 07 15:15:49.446974 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:15:49.446952 2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3064c74a4f9db75a445fc4762815ec9bf6ed992ff805562bc28ebd75e704262e\": container with ID starting with 3064c74a4f9db75a445fc4762815ec9bf6ed992ff805562bc28ebd75e704262e not found: ID does not exist" containerID="3064c74a4f9db75a445fc4762815ec9bf6ed992ff805562bc28ebd75e704262e" Jul 07 15:15:49.447081 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.446978 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3064c74a4f9db75a445fc4762815ec9bf6ed992ff805562bc28ebd75e704262e"} err="failed to get container status \"3064c74a4f9db75a445fc4762815ec9bf6ed992ff805562bc28ebd75e704262e\": rpc error: code = NotFound desc = could not find container \"3064c74a4f9db75a445fc4762815ec9bf6ed992ff805562bc28ebd75e704262e\": container with ID starting with 3064c74a4f9db75a445fc4762815ec9bf6ed992ff805562bc28ebd75e704262e not found: ID does not exist" Jul 07 15:15:49.447081 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.446998 2549 scope.go:117] "RemoveContainer" containerID="35c70690c7b95d59e392303a868cef85334eab1af0326bf19577f38a30e5a808" Jul 07 15:15:49.447251 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:15:49.447236 2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35c70690c7b95d59e392303a868cef85334eab1af0326bf19577f38a30e5a808\": container with ID starting with 35c70690c7b95d59e392303a868cef85334eab1af0326bf19577f38a30e5a808 not found: ID does not exist" containerID="35c70690c7b95d59e392303a868cef85334eab1af0326bf19577f38a30e5a808" Jul 07 15:15:49.447287 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.447254 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35c70690c7b95d59e392303a868cef85334eab1af0326bf19577f38a30e5a808"} err="failed to get container status \"35c70690c7b95d59e392303a868cef85334eab1af0326bf19577f38a30e5a808\": rpc error: code = NotFound desc = could not find container \"35c70690c7b95d59e392303a868cef85334eab1af0326bf19577f38a30e5a808\": container with ID starting with 35c70690c7b95d59e392303a868cef85334eab1af0326bf19577f38a30e5a808 not found: ID does not exist" Jul 07 15:15:49.447287 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.447269 2549 scope.go:117] "RemoveContainer" containerID="7f38e590b961779527f985850610948329752bfe67208ef1e2d7e629570cf4ca" Jul 07 15:15:49.447537 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:15:49.447520 2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f38e590b961779527f985850610948329752bfe67208ef1e2d7e629570cf4ca\": container with ID starting with 7f38e590b961779527f985850610948329752bfe67208ef1e2d7e629570cf4ca not found: ID does not exist" containerID="7f38e590b961779527f985850610948329752bfe67208ef1e2d7e629570cf4ca" Jul 07 15:15:49.447619 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.447542 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f38e590b961779527f985850610948329752bfe67208ef1e2d7e629570cf4ca"} err="failed to get container status \"7f38e590b961779527f985850610948329752bfe67208ef1e2d7e629570cf4ca\": rpc error: code = NotFound desc = could not find container \"7f38e590b961779527f985850610948329752bfe67208ef1e2d7e629570cf4ca\": container with ID starting with 7f38e590b961779527f985850610948329752bfe67208ef1e2d7e629570cf4ca not found: ID does not exist" Jul 07 15:15:49.447619 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.447562 2549 scope.go:117] "RemoveContainer" containerID="0577cbf008f9fc82e6910fb991b845188dceb869fae2b7ea8f1b65005d378dbc" Jul 07 15:15:49.447816 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:15:49.447797 2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0577cbf008f9fc82e6910fb991b845188dceb869fae2b7ea8f1b65005d378dbc\": container with ID starting with 0577cbf008f9fc82e6910fb991b845188dceb869fae2b7ea8f1b65005d378dbc not found: ID does not exist" containerID="0577cbf008f9fc82e6910fb991b845188dceb869fae2b7ea8f1b65005d378dbc" Jul 07 15:15:49.447854 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.447820 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0577cbf008f9fc82e6910fb991b845188dceb869fae2b7ea8f1b65005d378dbc"} err="failed to get container status \"0577cbf008f9fc82e6910fb991b845188dceb869fae2b7ea8f1b65005d378dbc\": rpc error: code = NotFound desc = could not find container \"0577cbf008f9fc82e6910fb991b845188dceb869fae2b7ea8f1b65005d378dbc\": container with ID starting with 0577cbf008f9fc82e6910fb991b845188dceb869fae2b7ea8f1b65005d378dbc not found: ID does not exist" Jul 07 15:15:49.447854 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.447834 2549 scope.go:117] "RemoveContainer" containerID="6e28c5a9ed93e91b87cee99182e66454e2bdcedd69ab6994688a772cac7b674d" Jul 07 15:15:49.448053 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:15:49.448035 2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e28c5a9ed93e91b87cee99182e66454e2bdcedd69ab6994688a772cac7b674d\": container with ID starting with 6e28c5a9ed93e91b87cee99182e66454e2bdcedd69ab6994688a772cac7b674d not found: ID does not exist" containerID="6e28c5a9ed93e91b87cee99182e66454e2bdcedd69ab6994688a772cac7b674d" Jul 07 15:15:49.448100 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.448056 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e28c5a9ed93e91b87cee99182e66454e2bdcedd69ab6994688a772cac7b674d"} err="failed to get container status \"6e28c5a9ed93e91b87cee99182e66454e2bdcedd69ab6994688a772cac7b674d\": rpc error: code = NotFound desc = could not find container \"6e28c5a9ed93e91b87cee99182e66454e2bdcedd69ab6994688a772cac7b674d\": container with ID starting with 6e28c5a9ed93e91b87cee99182e66454e2bdcedd69ab6994688a772cac7b674d not found: ID does not exist" Jul 07 15:15:49.448100 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.448068 2549 scope.go:117] "RemoveContainer" containerID="ecffba129361af7f4955403120ebc05565a6fd1f0a33a3e005c0e709b8813adb" Jul 07 15:15:49.448287 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.448272 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ecffba129361af7f4955403120ebc05565a6fd1f0a33a3e005c0e709b8813adb"} err="failed to get container status \"ecffba129361af7f4955403120ebc05565a6fd1f0a33a3e005c0e709b8813adb\": rpc error: code = NotFound desc = could not find container \"ecffba129361af7f4955403120ebc05565a6fd1f0a33a3e005c0e709b8813adb\": container with ID starting with ecffba129361af7f4955403120ebc05565a6fd1f0a33a3e005c0e709b8813adb not found: ID does not exist" Jul 07 15:15:49.448382 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.448287 2549 scope.go:117] "RemoveContainer" containerID="bf94db224dc6b4b4289e1b888e000ca8121354a2ef330b3248a8d34755c17db3" Jul 07 15:15:49.448493 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.448479 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf94db224dc6b4b4289e1b888e000ca8121354a2ef330b3248a8d34755c17db3"} err="failed to get container status \"bf94db224dc6b4b4289e1b888e000ca8121354a2ef330b3248a8d34755c17db3\": rpc error: code = NotFound desc = could not find container \"bf94db224dc6b4b4289e1b888e000ca8121354a2ef330b3248a8d34755c17db3\": container with ID starting with bf94db224dc6b4b4289e1b888e000ca8121354a2ef330b3248a8d34755c17db3 not found: ID does not exist" Jul 07 15:15:49.448536 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.448493 2549 scope.go:117] "RemoveContainer" containerID="3064c74a4f9db75a445fc4762815ec9bf6ed992ff805562bc28ebd75e704262e" Jul 07 15:15:49.448717 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.448698 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3064c74a4f9db75a445fc4762815ec9bf6ed992ff805562bc28ebd75e704262e"} err="failed to get container status \"3064c74a4f9db75a445fc4762815ec9bf6ed992ff805562bc28ebd75e704262e\": rpc error: code = NotFound desc = could not find container \"3064c74a4f9db75a445fc4762815ec9bf6ed992ff805562bc28ebd75e704262e\": container with ID starting with 3064c74a4f9db75a445fc4762815ec9bf6ed992ff805562bc28ebd75e704262e not found: ID does not exist" Jul 07 15:15:49.448795 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.448719 2549 scope.go:117] "RemoveContainer" containerID="35c70690c7b95d59e392303a868cef85334eab1af0326bf19577f38a30e5a808" Jul 07 15:15:49.448929 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.448912 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35c70690c7b95d59e392303a868cef85334eab1af0326bf19577f38a30e5a808"} err="failed to get container status \"35c70690c7b95d59e392303a868cef85334eab1af0326bf19577f38a30e5a808\": rpc error: code = NotFound desc = could not find container \"35c70690c7b95d59e392303a868cef85334eab1af0326bf19577f38a30e5a808\": container with ID starting with 35c70690c7b95d59e392303a868cef85334eab1af0326bf19577f38a30e5a808 not found: ID does not exist" Jul 07 15:15:49.448980 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.448931 2549 scope.go:117] "RemoveContainer" containerID="7f38e590b961779527f985850610948329752bfe67208ef1e2d7e629570cf4ca" Jul 07 15:15:49.449143 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.449125 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f38e590b961779527f985850610948329752bfe67208ef1e2d7e629570cf4ca"} err="failed to get container status \"7f38e590b961779527f985850610948329752bfe67208ef1e2d7e629570cf4ca\": rpc error: code = NotFound desc = could not find container \"7f38e590b961779527f985850610948329752bfe67208ef1e2d7e629570cf4ca\": container with ID starting with 7f38e590b961779527f985850610948329752bfe67208ef1e2d7e629570cf4ca not found: ID does not exist" Jul 07 15:15:49.449187 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.449144 2549 scope.go:117] "RemoveContainer" containerID="0577cbf008f9fc82e6910fb991b845188dceb869fae2b7ea8f1b65005d378dbc" Jul 07 15:15:49.449342 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.449327 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0577cbf008f9fc82e6910fb991b845188dceb869fae2b7ea8f1b65005d378dbc"} err="failed to get container status \"0577cbf008f9fc82e6910fb991b845188dceb869fae2b7ea8f1b65005d378dbc\": rpc error: code = NotFound desc = could not find container \"0577cbf008f9fc82e6910fb991b845188dceb869fae2b7ea8f1b65005d378dbc\": container with ID starting with 0577cbf008f9fc82e6910fb991b845188dceb869fae2b7ea8f1b65005d378dbc not found: ID does not exist" Jul 07 15:15:49.449384 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.449342 2549 scope.go:117] "RemoveContainer" containerID="6e28c5a9ed93e91b87cee99182e66454e2bdcedd69ab6994688a772cac7b674d" Jul 07 15:15:49.449561 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.449543 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e28c5a9ed93e91b87cee99182e66454e2bdcedd69ab6994688a772cac7b674d"} err="failed to get container status \"6e28c5a9ed93e91b87cee99182e66454e2bdcedd69ab6994688a772cac7b674d\": rpc error: code = NotFound desc = could not find container \"6e28c5a9ed93e91b87cee99182e66454e2bdcedd69ab6994688a772cac7b674d\": container with ID starting with 6e28c5a9ed93e91b87cee99182e66454e2bdcedd69ab6994688a772cac7b674d not found: ID does not exist" Jul 07 15:15:49.527301 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.527270 2549 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b2e65842-a1c8-4e50-801e-b949cc3a410e-web-config\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:15:49.527301 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.527296 2549 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b2e65842-a1c8-4e50-801e-b949cc3a410e-cluster-tls-config\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:15:49.527301 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.527305 2549 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b2e65842-a1c8-4e50-801e-b949cc3a410e-config-out\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:15:49.527505 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.527315 2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7fjcs\" (UniqueName: \"kubernetes.io/projected/b2e65842-a1c8-4e50-801e-b949cc3a410e-kube-api-access-7fjcs\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:15:49.527505 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.527327 2549 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b2e65842-a1c8-4e50-801e-b949cc3a410e-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:15:49.527505 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.527336 2549 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b2e65842-a1c8-4e50-801e-b949cc3a410e-metrics-client-ca\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:15:49.527505 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.527344 2549 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b2e65842-a1c8-4e50-801e-b949cc3a410e-alertmanager-main-db\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:15:49.527505 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.527352 2549 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b2e65842-a1c8-4e50-801e-b949cc3a410e-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:15:49.527505 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.527362 2549 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b2e65842-a1c8-4e50-801e-b949cc3a410e-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:15:49.527505 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.527371 2549 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b2e65842-a1c8-4e50-801e-b949cc3a410e-tls-assets\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:15:49.527505 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.527379 2549 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b2e65842-a1c8-4e50-801e-b949cc3a410e-config-volume\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:15:49.527505 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.527386 2549 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b2e65842-a1c8-4e50-801e-b949cc3a410e-secret-alertmanager-main-tls\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:15:49.710770 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.710740 2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 15:15:49.715276 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.715253 2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 15:15:49.762200 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.762157 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 15:15:49.762546 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.762529 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2e65842-a1c8-4e50-801e-b949cc3a410e" containerName="init-config-reloader" Jul 07 15:15:49.762651 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.762550 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2e65842-a1c8-4e50-801e-b949cc3a410e" containerName="init-config-reloader" Jul 07 15:15:49.762651 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.762566 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2e65842-a1c8-4e50-801e-b949cc3a410e" containerName="alertmanager" Jul 07 15:15:49.762651 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.762575 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2e65842-a1c8-4e50-801e-b949cc3a410e" containerName="alertmanager" Jul 07 15:15:49.762651 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.762604 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2e65842-a1c8-4e50-801e-b949cc3a410e" containerName="kube-rbac-proxy-metric" Jul 07 15:15:49.762651 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.762613 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2e65842-a1c8-4e50-801e-b949cc3a410e" containerName="kube-rbac-proxy-metric" Jul 07 15:15:49.762651 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.762630 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2e65842-a1c8-4e50-801e-b949cc3a410e" containerName="config-reloader" Jul 07 15:15:49.762651 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.762638 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2e65842-a1c8-4e50-801e-b949cc3a410e" containerName="config-reloader" Jul 07 15:15:49.762651 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.762647 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2e65842-a1c8-4e50-801e-b949cc3a410e" containerName="kube-rbac-proxy" Jul 07 15:15:49.762651 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.762655 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2e65842-a1c8-4e50-801e-b949cc3a410e" containerName="kube-rbac-proxy" Jul 07 15:15:49.763093 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.762669 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2e65842-a1c8-4e50-801e-b949cc3a410e" containerName="kube-rbac-proxy-web" Jul 07 15:15:49.763093 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.762678 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2e65842-a1c8-4e50-801e-b949cc3a410e" containerName="kube-rbac-proxy-web" Jul 07 15:15:49.763093 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.762688 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2e65842-a1c8-4e50-801e-b949cc3a410e" containerName="prom-label-proxy" Jul 07 15:15:49.763093 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.762696 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2e65842-a1c8-4e50-801e-b949cc3a410e" containerName="prom-label-proxy" Jul 07 15:15:49.763093 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.762715 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22d3c83b-a693-4f0a-8ae2-07d83467c620" containerName="registry" Jul 07 15:15:49.763093 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.762722 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="22d3c83b-a693-4f0a-8ae2-07d83467c620" containerName="registry" Jul 07 15:15:49.763093 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.762813 2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2e65842-a1c8-4e50-801e-b949cc3a410e" containerName="config-reloader" Jul 07 15:15:49.763093 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.762824 2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2e65842-a1c8-4e50-801e-b949cc3a410e" containerName="kube-rbac-proxy-metric" Jul 07 15:15:49.763093 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.762835 2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2e65842-a1c8-4e50-801e-b949cc3a410e" containerName="alertmanager" Jul 07 15:15:49.763093 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.762844 2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2e65842-a1c8-4e50-801e-b949cc3a410e" containerName="kube-rbac-proxy" Jul 07 15:15:49.763093 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.762855 2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2e65842-a1c8-4e50-801e-b949cc3a410e" containerName="kube-rbac-proxy-web" Jul 07 15:15:49.763093 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.762864 2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2e65842-a1c8-4e50-801e-b949cc3a410e" containerName="prom-label-proxy" Jul 07 15:15:49.763093 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.762874 2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="22d3c83b-a693-4f0a-8ae2-07d83467c620" containerName="registry" Jul 07 15:15:49.767835 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.767816 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:15:49.813784 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.813758 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 07 15:15:49.816273 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.816246 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 07 15:15:49.816273 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.816267 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 07 15:15:49.816446 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.816299 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 07 15:15:49.816446 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.816335 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 07 15:15:49.816688 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.816672 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 07 15:15:49.816872 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.816858 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-xjrrg\"" Jul 07 15:15:49.817438 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.817425 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 07 15:15:49.828803 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.828780 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 07 15:15:49.830308 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.830287 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f4f360be-ae5c-4ef3-a09e-7d7f993cbc42-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f4f360be-ae5c-4ef3-a09e-7d7f993cbc42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:15:49.830400 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.830328 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f4f360be-ae5c-4ef3-a09e-7d7f993cbc42-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f4f360be-ae5c-4ef3-a09e-7d7f993cbc42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:15:49.830400 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.830362 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f4f360be-ae5c-4ef3-a09e-7d7f993cbc42-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f4f360be-ae5c-4ef3-a09e-7d7f993cbc42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:15:49.830512 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.830411 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f4f360be-ae5c-4ef3-a09e-7d7f993cbc42-web-config\") pod \"alertmanager-main-0\" (UID: \"f4f360be-ae5c-4ef3-a09e-7d7f993cbc42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:15:49.830512 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.830440 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f4f360be-ae5c-4ef3-a09e-7d7f993cbc42-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f4f360be-ae5c-4ef3-a09e-7d7f993cbc42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:15:49.830512 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.830465 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvhkl\" (UniqueName: \"kubernetes.io/projected/f4f360be-ae5c-4ef3-a09e-7d7f993cbc42-kube-api-access-jvhkl\") pod \"alertmanager-main-0\" (UID: \"f4f360be-ae5c-4ef3-a09e-7d7f993cbc42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:15:49.830512 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.830508 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f4f360be-ae5c-4ef3-a09e-7d7f993cbc42-config-volume\") pod \"alertmanager-main-0\" (UID: \"f4f360be-ae5c-4ef3-a09e-7d7f993cbc42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:15:49.830727 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.830570 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4f360be-ae5c-4ef3-a09e-7d7f993cbc42-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f4f360be-ae5c-4ef3-a09e-7d7f993cbc42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:15:49.830727 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.830655 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f4f360be-ae5c-4ef3-a09e-7d7f993cbc42-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f4f360be-ae5c-4ef3-a09e-7d7f993cbc42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:15:49.830727 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.830683 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f4f360be-ae5c-4ef3-a09e-7d7f993cbc42-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f4f360be-ae5c-4ef3-a09e-7d7f993cbc42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:15:49.830872 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.830786 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f4f360be-ae5c-4ef3-a09e-7d7f993cbc42-config-out\") pod \"alertmanager-main-0\" (UID: \"f4f360be-ae5c-4ef3-a09e-7d7f993cbc42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:15:49.830872 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.830815 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f4f360be-ae5c-4ef3-a09e-7d7f993cbc42-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f4f360be-ae5c-4ef3-a09e-7d7f993cbc42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:15:49.830872 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.830850 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f4f360be-ae5c-4ef3-a09e-7d7f993cbc42-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f4f360be-ae5c-4ef3-a09e-7d7f993cbc42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:15:49.843728 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.843699 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 15:15:49.865195 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.865165 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 07 15:15:49.931888 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.931850 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f4f360be-ae5c-4ef3-a09e-7d7f993cbc42-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f4f360be-ae5c-4ef3-a09e-7d7f993cbc42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:15:49.932094 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.931892 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f4f360be-ae5c-4ef3-a09e-7d7f993cbc42-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f4f360be-ae5c-4ef3-a09e-7d7f993cbc42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:15:49.932094 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.931939 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f4f360be-ae5c-4ef3-a09e-7d7f993cbc42-config-out\") pod \"alertmanager-main-0\" (UID: \"f4f360be-ae5c-4ef3-a09e-7d7f993cbc42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:15:49.932094 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.931964 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f4f360be-ae5c-4ef3-a09e-7d7f993cbc42-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f4f360be-ae5c-4ef3-a09e-7d7f993cbc42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:15:49.932094 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.931991 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f4f360be-ae5c-4ef3-a09e-7d7f993cbc42-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f4f360be-ae5c-4ef3-a09e-7d7f993cbc42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:15:49.932307 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.932148 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f4f360be-ae5c-4ef3-a09e-7d7f993cbc42-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f4f360be-ae5c-4ef3-a09e-7d7f993cbc42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:15:49.932307 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.932208 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f4f360be-ae5c-4ef3-a09e-7d7f993cbc42-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f4f360be-ae5c-4ef3-a09e-7d7f993cbc42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:15:49.932307 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.932252 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f4f360be-ae5c-4ef3-a09e-7d7f993cbc42-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f4f360be-ae5c-4ef3-a09e-7d7f993cbc42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:15:49.932467 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.932302 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f4f360be-ae5c-4ef3-a09e-7d7f993cbc42-web-config\") pod \"alertmanager-main-0\" (UID: \"f4f360be-ae5c-4ef3-a09e-7d7f993cbc42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:15:49.932467 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.932321 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f4f360be-ae5c-4ef3-a09e-7d7f993cbc42-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f4f360be-ae5c-4ef3-a09e-7d7f993cbc42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:15:49.932467 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.932350 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f4f360be-ae5c-4ef3-a09e-7d7f993cbc42-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f4f360be-ae5c-4ef3-a09e-7d7f993cbc42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:15:49.932467 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.932380 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvhkl\" (UniqueName: \"kubernetes.io/projected/f4f360be-ae5c-4ef3-a09e-7d7f993cbc42-kube-api-access-jvhkl\") pod \"alertmanager-main-0\" (UID: \"f4f360be-ae5c-4ef3-a09e-7d7f993cbc42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:15:49.932467 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.932434 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f4f360be-ae5c-4ef3-a09e-7d7f993cbc42-config-volume\") pod \"alertmanager-main-0\" (UID: \"f4f360be-ae5c-4ef3-a09e-7d7f993cbc42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:15:49.932765 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.932470 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4f360be-ae5c-4ef3-a09e-7d7f993cbc42-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f4f360be-ae5c-4ef3-a09e-7d7f993cbc42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:15:49.933067 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.932962 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f4f360be-ae5c-4ef3-a09e-7d7f993cbc42-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f4f360be-ae5c-4ef3-a09e-7d7f993cbc42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:15:49.933499 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.933454 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4f360be-ae5c-4ef3-a09e-7d7f993cbc42-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f4f360be-ae5c-4ef3-a09e-7d7f993cbc42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:15:49.934932 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.934850 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f4f360be-ae5c-4ef3-a09e-7d7f993cbc42-config-out\") pod \"alertmanager-main-0\" (UID: \"f4f360be-ae5c-4ef3-a09e-7d7f993cbc42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:15:49.935522 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.935143 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f4f360be-ae5c-4ef3-a09e-7d7f993cbc42-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f4f360be-ae5c-4ef3-a09e-7d7f993cbc42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:15:49.935522 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.935262 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f4f360be-ae5c-4ef3-a09e-7d7f993cbc42-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f4f360be-ae5c-4ef3-a09e-7d7f993cbc42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:15:49.935522 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.935312 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f4f360be-ae5c-4ef3-a09e-7d7f993cbc42-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f4f360be-ae5c-4ef3-a09e-7d7f993cbc42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:15:49.935522 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.935466 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f4f360be-ae5c-4ef3-a09e-7d7f993cbc42-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f4f360be-ae5c-4ef3-a09e-7d7f993cbc42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:15:49.935821 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.935567 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f4f360be-ae5c-4ef3-a09e-7d7f993cbc42-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f4f360be-ae5c-4ef3-a09e-7d7f993cbc42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:15:49.935821 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.935757 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f4f360be-ae5c-4ef3-a09e-7d7f993cbc42-web-config\") pod \"alertmanager-main-0\" (UID: \"f4f360be-ae5c-4ef3-a09e-7d7f993cbc42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:15:49.936366 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.936345 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f4f360be-ae5c-4ef3-a09e-7d7f993cbc42-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f4f360be-ae5c-4ef3-a09e-7d7f993cbc42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:15:49.936509 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.936492 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f4f360be-ae5c-4ef3-a09e-7d7f993cbc42-config-volume\") pod \"alertmanager-main-0\" (UID: \"f4f360be-ae5c-4ef3-a09e-7d7f993cbc42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:15:49.970740 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:49.970675 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvhkl\" (UniqueName: \"kubernetes.io/projected/f4f360be-ae5c-4ef3-a09e-7d7f993cbc42-kube-api-access-jvhkl\") pod \"alertmanager-main-0\" (UID: \"f4f360be-ae5c-4ef3-a09e-7d7f993cbc42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:15:50.076390 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:50.076346 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 15:15:50.231345 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:50.231266 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 15:15:50.234682 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:15:50.234653 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4f360be_ae5c_4ef3_a09e_7d7f993cbc42.slice/crio-9698db2e3571dfe433ae98014f8e0649da5efa8e238f1f30ec241882d8ddd748 WatchSource:0}: Error finding container 9698db2e3571dfe433ae98014f8e0649da5efa8e238f1f30ec241882d8ddd748: Status 404 returned error can't find the container with id 9698db2e3571dfe433ae98014f8e0649da5efa8e238f1f30ec241882d8ddd748 Jul 07 15:15:50.350707 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:15:50.350676 2549 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4f360be_ae5c_4ef3_a09e_7d7f993cbc42.slice/crio-01bf01c136d2d6a574177a68ced5f854584945068c6a65b671c3eb40d64dc042.scope\": RecentStats: unable to find data in memory cache]" Jul 07 15:15:50.387564 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:50.387528 2549 generic.go:356] "Generic (PLEG): container finished" podID="f4f360be-ae5c-4ef3-a09e-7d7f993cbc42" containerID="01bf01c136d2d6a574177a68ced5f854584945068c6a65b671c3eb40d64dc042" exitCode=0 Jul 07 15:15:50.387728 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:50.387618 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f4f360be-ae5c-4ef3-a09e-7d7f993cbc42","Type":"ContainerDied","Data":"01bf01c136d2d6a574177a68ced5f854584945068c6a65b671c3eb40d64dc042"} Jul 07 15:15:50.387728 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:50.387688 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f4f360be-ae5c-4ef3-a09e-7d7f993cbc42","Type":"ContainerStarted","Data":"9698db2e3571dfe433ae98014f8e0649da5efa8e238f1f30ec241882d8ddd748"} Jul 07 15:15:50.534205 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:50.534179 2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2e65842-a1c8-4e50-801e-b949cc3a410e" path="/var/lib/kubelet/pods/b2e65842-a1c8-4e50-801e-b949cc3a410e/volumes" Jul 07 15:15:51.397069 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:51.396986 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f4f360be-ae5c-4ef3-a09e-7d7f993cbc42","Type":"ContainerStarted","Data":"a9eceab30f896c1f40270779435f22f3a850e3b858160ca3f68677f38dd7883c"} Jul 07 15:15:51.397069 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:51.397052 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f4f360be-ae5c-4ef3-a09e-7d7f993cbc42","Type":"ContainerStarted","Data":"2ffb26d6677876fb9477030569df75d0a1703837e6aea6c8e94400fd5490c5b2"} Jul 07 15:15:51.397473 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:51.397077 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f4f360be-ae5c-4ef3-a09e-7d7f993cbc42","Type":"ContainerStarted","Data":"81f63ab6f46ced47abde34bd9d4e6d9ac748dffd2de133a61e192ae261630f7b"} Jul 07 15:15:51.397473 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:51.397101 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f4f360be-ae5c-4ef3-a09e-7d7f993cbc42","Type":"ContainerStarted","Data":"974d6a68408c5e36f6edfd3b18ba6c02f349ef84300bb20f85c16fc88ed23840"} Jul 07 15:15:51.397473 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:51.397125 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f4f360be-ae5c-4ef3-a09e-7d7f993cbc42","Type":"ContainerStarted","Data":"adeb5e8547ae59407d42ef8a43c86157e6e3607c4ab218924bb4c1b322c59cd0"} Jul 07 15:15:51.397473 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:51.397147 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f4f360be-ae5c-4ef3-a09e-7d7f993cbc42","Type":"ContainerStarted","Data":"734dfad3f377909d1e88e2878ba301f7eaa672bc8db473f950e6ca68e18d1780"} Jul 07 15:15:51.434403 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:51.434346 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.417270016 podStartE2EDuration="2.434330066s" totalImagesPullingTime="0s" totalInitContainerRuntime="17.06005ms" isStatefulPod=false podCreationTimestamp="2026-07-07 15:15:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:15:51.432935446 +0000 UTC m=+169.440412920" watchObservedRunningTime="2026-07-07 15:15:51.434330066 +0000 UTC m=+169.441807516" Jul 07 15:15:52.055813 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:52.051595 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5675fd947d-fm6jm"] Jul 07 15:15:52.057770 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:52.057742 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5675fd947d-fm6jm" Jul 07 15:15:52.061899 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:52.061876 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 07 15:15:52.062152 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:52.062136 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 07 15:15:52.062232 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:52.062147 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 07 15:15:52.062232 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:52.062136 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-xbk4c\"" Jul 07 15:15:52.062232 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:52.062179 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 07 15:15:52.062761 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:52.062744 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 07 15:15:52.068563 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:52.068545 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 07 15:15:52.073996 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:52.073978 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5675fd947d-fm6jm"] Jul 07 15:15:52.153787 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:52.153753 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e04181b-70b5-421d-8635-0b4abe99c717-serving-certs-ca-bundle\") pod \"telemeter-client-5675fd947d-fm6jm\" (UID: \"4e04181b-70b5-421d-8635-0b4abe99c717\") " pod="openshift-monitoring/telemeter-client-5675fd947d-fm6jm" Jul 07 15:15:52.153787 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:52.153791 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/4e04181b-70b5-421d-8635-0b4abe99c717-federate-client-tls\") pod \"telemeter-client-5675fd947d-fm6jm\" (UID: \"4e04181b-70b5-421d-8635-0b4abe99c717\") " pod="openshift-monitoring/telemeter-client-5675fd947d-fm6jm" Jul 07 15:15:52.154015 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:52.153814 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/4e04181b-70b5-421d-8635-0b4abe99c717-secret-telemeter-client\") pod \"telemeter-client-5675fd947d-fm6jm\" (UID: \"4e04181b-70b5-421d-8635-0b4abe99c717\") " pod="openshift-monitoring/telemeter-client-5675fd947d-fm6jm" Jul 07 15:15:52.154015 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:52.153834 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e04181b-70b5-421d-8635-0b4abe99c717-metrics-client-ca\") pod \"telemeter-client-5675fd947d-fm6jm\" (UID: \"4e04181b-70b5-421d-8635-0b4abe99c717\") " pod="openshift-monitoring/telemeter-client-5675fd947d-fm6jm" Jul 07 15:15:52.154015 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:52.153923 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4e04181b-70b5-421d-8635-0b4abe99c717-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5675fd947d-fm6jm\" (UID: \"4e04181b-70b5-421d-8635-0b4abe99c717\") " pod="openshift-monitoring/telemeter-client-5675fd947d-fm6jm" Jul 07 15:15:52.154015 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:52.153995 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77cmh\" (UniqueName: \"kubernetes.io/projected/4e04181b-70b5-421d-8635-0b4abe99c717-kube-api-access-77cmh\") pod \"telemeter-client-5675fd947d-fm6jm\" (UID: \"4e04181b-70b5-421d-8635-0b4abe99c717\") " pod="openshift-monitoring/telemeter-client-5675fd947d-fm6jm" Jul 07 15:15:52.154162 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:52.154024 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e04181b-70b5-421d-8635-0b4abe99c717-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5675fd947d-fm6jm\" (UID: \"4e04181b-70b5-421d-8635-0b4abe99c717\") " pod="openshift-monitoring/telemeter-client-5675fd947d-fm6jm" Jul 07 15:15:52.154162 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:52.154046 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/4e04181b-70b5-421d-8635-0b4abe99c717-telemeter-client-tls\") pod \"telemeter-client-5675fd947d-fm6jm\" (UID: \"4e04181b-70b5-421d-8635-0b4abe99c717\") " pod="openshift-monitoring/telemeter-client-5675fd947d-fm6jm" Jul 07 15:15:52.255443 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:52.255407 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77cmh\" (UniqueName: \"kubernetes.io/projected/4e04181b-70b5-421d-8635-0b4abe99c717-kube-api-access-77cmh\") pod \"telemeter-client-5675fd947d-fm6jm\" (UID: \"4e04181b-70b5-421d-8635-0b4abe99c717\") " pod="openshift-monitoring/telemeter-client-5675fd947d-fm6jm" Jul 07 15:15:52.255443 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:52.255450 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e04181b-70b5-421d-8635-0b4abe99c717-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5675fd947d-fm6jm\" (UID: \"4e04181b-70b5-421d-8635-0b4abe99c717\") " pod="openshift-monitoring/telemeter-client-5675fd947d-fm6jm" Jul 07 15:15:52.255727 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:52.255472 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/4e04181b-70b5-421d-8635-0b4abe99c717-telemeter-client-tls\") pod \"telemeter-client-5675fd947d-fm6jm\" (UID: \"4e04181b-70b5-421d-8635-0b4abe99c717\") " pod="openshift-monitoring/telemeter-client-5675fd947d-fm6jm" Jul 07 15:15:52.255727 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:52.255518 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e04181b-70b5-421d-8635-0b4abe99c717-serving-certs-ca-bundle\") pod \"telemeter-client-5675fd947d-fm6jm\" (UID: \"4e04181b-70b5-421d-8635-0b4abe99c717\") " pod="openshift-monitoring/telemeter-client-5675fd947d-fm6jm" Jul 07 15:15:52.255727 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:52.255535 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/4e04181b-70b5-421d-8635-0b4abe99c717-federate-client-tls\") pod \"telemeter-client-5675fd947d-fm6jm\" (UID: \"4e04181b-70b5-421d-8635-0b4abe99c717\") " pod="openshift-monitoring/telemeter-client-5675fd947d-fm6jm" Jul 07 15:15:52.255727 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:52.255559 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/4e04181b-70b5-421d-8635-0b4abe99c717-secret-telemeter-client\") pod \"telemeter-client-5675fd947d-fm6jm\" (UID: \"4e04181b-70b5-421d-8635-0b4abe99c717\") " pod="openshift-monitoring/telemeter-client-5675fd947d-fm6jm" Jul 07 15:15:52.255727 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:52.255676 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e04181b-70b5-421d-8635-0b4abe99c717-metrics-client-ca\") pod \"telemeter-client-5675fd947d-fm6jm\" (UID: \"4e04181b-70b5-421d-8635-0b4abe99c717\") " pod="openshift-monitoring/telemeter-client-5675fd947d-fm6jm" Jul 07 15:15:52.255969 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:52.255733 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4e04181b-70b5-421d-8635-0b4abe99c717-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5675fd947d-fm6jm\" (UID: \"4e04181b-70b5-421d-8635-0b4abe99c717\") " pod="openshift-monitoring/telemeter-client-5675fd947d-fm6jm" Jul 07 15:15:52.256445 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:52.256419 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e04181b-70b5-421d-8635-0b4abe99c717-metrics-client-ca\") pod \"telemeter-client-5675fd947d-fm6jm\" (UID: \"4e04181b-70b5-421d-8635-0b4abe99c717\") " pod="openshift-monitoring/telemeter-client-5675fd947d-fm6jm" Jul 07 15:15:52.256445 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:52.256433 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e04181b-70b5-421d-8635-0b4abe99c717-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5675fd947d-fm6jm\" (UID: \"4e04181b-70b5-421d-8635-0b4abe99c717\") " pod="openshift-monitoring/telemeter-client-5675fd947d-fm6jm" Jul 07 15:15:52.256646 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:52.256498 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e04181b-70b5-421d-8635-0b4abe99c717-serving-certs-ca-bundle\") pod \"telemeter-client-5675fd947d-fm6jm\" (UID: \"4e04181b-70b5-421d-8635-0b4abe99c717\") " pod="openshift-monitoring/telemeter-client-5675fd947d-fm6jm" Jul 07 15:15:52.258271 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:52.258245 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/4e04181b-70b5-421d-8635-0b4abe99c717-federate-client-tls\") pod \"telemeter-client-5675fd947d-fm6jm\" (UID: \"4e04181b-70b5-421d-8635-0b4abe99c717\") " pod="openshift-monitoring/telemeter-client-5675fd947d-fm6jm" Jul 07 15:15:52.258340 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:52.258253 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/4e04181b-70b5-421d-8635-0b4abe99c717-telemeter-client-tls\") pod \"telemeter-client-5675fd947d-fm6jm\" (UID: \"4e04181b-70b5-421d-8635-0b4abe99c717\") " pod="openshift-monitoring/telemeter-client-5675fd947d-fm6jm" Jul 07 15:15:52.258520 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:52.258504 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/4e04181b-70b5-421d-8635-0b4abe99c717-secret-telemeter-client\") pod \"telemeter-client-5675fd947d-fm6jm\" (UID: \"4e04181b-70b5-421d-8635-0b4abe99c717\") " pod="openshift-monitoring/telemeter-client-5675fd947d-fm6jm" Jul 07 15:15:52.258649 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:52.258629 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4e04181b-70b5-421d-8635-0b4abe99c717-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5675fd947d-fm6jm\" (UID: \"4e04181b-70b5-421d-8635-0b4abe99c717\") " pod="openshift-monitoring/telemeter-client-5675fd947d-fm6jm" Jul 07 15:15:52.264648 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:52.264629 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-77cmh\" (UniqueName: \"kubernetes.io/projected/4e04181b-70b5-421d-8635-0b4abe99c717-kube-api-access-77cmh\") pod \"telemeter-client-5675fd947d-fm6jm\" (UID: \"4e04181b-70b5-421d-8635-0b4abe99c717\") " pod="openshift-monitoring/telemeter-client-5675fd947d-fm6jm" Jul 07 15:15:52.368007 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:52.367973 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5675fd947d-fm6jm" Jul 07 15:15:52.518287 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:52.518254 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5675fd947d-fm6jm"] Jul 07 15:15:52.522623 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:15:52.522593 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e04181b_70b5_421d_8635_0b4abe99c717.slice/crio-1ea25821b4c4e9c6729a3f990ccab0e47a7c6ca396d8b6897b78537049903cd9 WatchSource:0}: Error finding container 1ea25821b4c4e9c6729a3f990ccab0e47a7c6ca396d8b6897b78537049903cd9: Status 404 returned error can't find the container with id 1ea25821b4c4e9c6729a3f990ccab0e47a7c6ca396d8b6897b78537049903cd9 Jul 07 15:15:53.408035 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:53.408000 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5675fd947d-fm6jm" event={"ID":"4e04181b-70b5-421d-8635-0b4abe99c717","Type":"ContainerStarted","Data":"1ea25821b4c4e9c6729a3f990ccab0e47a7c6ca396d8b6897b78537049903cd9"} Jul 07 15:15:55.425915 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:55.425881 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5675fd947d-fm6jm" event={"ID":"4e04181b-70b5-421d-8635-0b4abe99c717","Type":"ContainerStarted","Data":"2954e54594a560a60623274ccc5d83a472328db70c0ae7aa260211b4a986116e"} Jul 07 15:15:55.426307 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:55.425972 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5675fd947d-fm6jm" event={"ID":"4e04181b-70b5-421d-8635-0b4abe99c717","Type":"ContainerStarted","Data":"4336897b984cfbf136ade4691fa49118fff14b7e482d28dcb68a165c3cec82ca"} Jul 07 15:15:55.426307 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:55.425994 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5675fd947d-fm6jm" event={"ID":"4e04181b-70b5-421d-8635-0b4abe99c717","Type":"ContainerStarted","Data":"115a98ca9b12a819ac937eb6dc125a589e14fa6d1d9bc584f7d8b8547f026c04"} Jul 07 15:15:55.455371 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:55.455318 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5675fd947d-fm6jm" podStartSLOduration=1.52130465 podStartE2EDuration="3.455302656s" totalImagesPullingTime="1.933998006s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:15:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:15:55.454715549 +0000 UTC m=+173.462193002" watchObservedRunningTime="2026-07-07 15:15:55.455302656 +0000 UTC m=+173.462780107" Jul 07 15:15:56.126707 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:56.126674 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5c5c9b69c-spw8w"] Jul 07 15:15:56.129480 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:56.129452 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c5c9b69c-spw8w" Jul 07 15:15:56.146182 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:56.146152 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c5c9b69c-spw8w"] Jul 07 15:15:56.193399 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:56.193360 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2415c1e4-3d58-4218-8d06-b7c4d05d6385-oauth-serving-cert\") pod \"console-5c5c9b69c-spw8w\" (UID: \"2415c1e4-3d58-4218-8d06-b7c4d05d6385\") " pod="openshift-console/console-5c5c9b69c-spw8w" Jul 07 15:15:56.193562 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:56.193473 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhnz5\" (UniqueName: \"kubernetes.io/projected/2415c1e4-3d58-4218-8d06-b7c4d05d6385-kube-api-access-fhnz5\") pod \"console-5c5c9b69c-spw8w\" (UID: \"2415c1e4-3d58-4218-8d06-b7c4d05d6385\") " pod="openshift-console/console-5c5c9b69c-spw8w" Jul 07 15:15:56.193562 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:56.193524 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2415c1e4-3d58-4218-8d06-b7c4d05d6385-trusted-ca-bundle\") pod \"console-5c5c9b69c-spw8w\" (UID: \"2415c1e4-3d58-4218-8d06-b7c4d05d6385\") " pod="openshift-console/console-5c5c9b69c-spw8w" Jul 07 15:15:56.193562 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:56.193547 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2415c1e4-3d58-4218-8d06-b7c4d05d6385-service-ca\") pod \"console-5c5c9b69c-spw8w\" (UID: \"2415c1e4-3d58-4218-8d06-b7c4d05d6385\") " pod="openshift-console/console-5c5c9b69c-spw8w" Jul 07 15:15:56.193723 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:56.193689 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2415c1e4-3d58-4218-8d06-b7c4d05d6385-console-config\") pod \"console-5c5c9b69c-spw8w\" (UID: \"2415c1e4-3d58-4218-8d06-b7c4d05d6385\") " pod="openshift-console/console-5c5c9b69c-spw8w" Jul 07 15:15:56.193761 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:56.193739 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2415c1e4-3d58-4218-8d06-b7c4d05d6385-console-serving-cert\") pod \"console-5c5c9b69c-spw8w\" (UID: \"2415c1e4-3d58-4218-8d06-b7c4d05d6385\") " pod="openshift-console/console-5c5c9b69c-spw8w" Jul 07 15:15:56.193761 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:56.193754 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2415c1e4-3d58-4218-8d06-b7c4d05d6385-console-oauth-config\") pod \"console-5c5c9b69c-spw8w\" (UID: \"2415c1e4-3d58-4218-8d06-b7c4d05d6385\") " pod="openshift-console/console-5c5c9b69c-spw8w" Jul 07 15:15:56.294425 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:56.294383 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2415c1e4-3d58-4218-8d06-b7c4d05d6385-trusted-ca-bundle\") pod \"console-5c5c9b69c-spw8w\" (UID: \"2415c1e4-3d58-4218-8d06-b7c4d05d6385\") " pod="openshift-console/console-5c5c9b69c-spw8w" Jul 07 15:15:56.294425 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:56.294429 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2415c1e4-3d58-4218-8d06-b7c4d05d6385-service-ca\") pod \"console-5c5c9b69c-spw8w\" (UID: \"2415c1e4-3d58-4218-8d06-b7c4d05d6385\") " pod="openshift-console/console-5c5c9b69c-spw8w" Jul 07 15:15:56.294753 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:56.294478 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2415c1e4-3d58-4218-8d06-b7c4d05d6385-console-config\") pod \"console-5c5c9b69c-spw8w\" (UID: \"2415c1e4-3d58-4218-8d06-b7c4d05d6385\") " pod="openshift-console/console-5c5c9b69c-spw8w" Jul 07 15:15:56.294753 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:56.294522 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2415c1e4-3d58-4218-8d06-b7c4d05d6385-console-serving-cert\") pod \"console-5c5c9b69c-spw8w\" (UID: \"2415c1e4-3d58-4218-8d06-b7c4d05d6385\") " pod="openshift-console/console-5c5c9b69c-spw8w" Jul 07 15:15:56.294753 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:56.294546 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2415c1e4-3d58-4218-8d06-b7c4d05d6385-console-oauth-config\") pod \"console-5c5c9b69c-spw8w\" (UID: \"2415c1e4-3d58-4218-8d06-b7c4d05d6385\") " pod="openshift-console/console-5c5c9b69c-spw8w" Jul 07 15:15:56.294753 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:56.294712 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2415c1e4-3d58-4218-8d06-b7c4d05d6385-oauth-serving-cert\") pod \"console-5c5c9b69c-spw8w\" (UID: \"2415c1e4-3d58-4218-8d06-b7c4d05d6385\") " pod="openshift-console/console-5c5c9b69c-spw8w" Jul 07 15:15:56.294956 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:56.294777 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhnz5\" (UniqueName: \"kubernetes.io/projected/2415c1e4-3d58-4218-8d06-b7c4d05d6385-kube-api-access-fhnz5\") pod \"console-5c5c9b69c-spw8w\" (UID: \"2415c1e4-3d58-4218-8d06-b7c4d05d6385\") " pod="openshift-console/console-5c5c9b69c-spw8w" Jul 07 15:15:56.295371 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:56.295350 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2415c1e4-3d58-4218-8d06-b7c4d05d6385-console-config\") pod \"console-5c5c9b69c-spw8w\" (UID: \"2415c1e4-3d58-4218-8d06-b7c4d05d6385\") " pod="openshift-console/console-5c5c9b69c-spw8w" Jul 07 15:15:56.295482 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:56.295350 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2415c1e4-3d58-4218-8d06-b7c4d05d6385-service-ca\") pod \"console-5c5c9b69c-spw8w\" (UID: \"2415c1e4-3d58-4218-8d06-b7c4d05d6385\") " pod="openshift-console/console-5c5c9b69c-spw8w" Jul 07 15:15:56.295482 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:56.295410 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2415c1e4-3d58-4218-8d06-b7c4d05d6385-trusted-ca-bundle\") pod \"console-5c5c9b69c-spw8w\" (UID: \"2415c1e4-3d58-4218-8d06-b7c4d05d6385\") " pod="openshift-console/console-5c5c9b69c-spw8w" Jul 07 15:15:56.295482 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:56.295453 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2415c1e4-3d58-4218-8d06-b7c4d05d6385-oauth-serving-cert\") pod \"console-5c5c9b69c-spw8w\" (UID: \"2415c1e4-3d58-4218-8d06-b7c4d05d6385\") " pod="openshift-console/console-5c5c9b69c-spw8w" Jul 07 15:15:56.297073 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:56.297053 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2415c1e4-3d58-4218-8d06-b7c4d05d6385-console-oauth-config\") pod \"console-5c5c9b69c-spw8w\" (UID: \"2415c1e4-3d58-4218-8d06-b7c4d05d6385\") " pod="openshift-console/console-5c5c9b69c-spw8w" Jul 07 15:15:56.297176 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:56.297158 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2415c1e4-3d58-4218-8d06-b7c4d05d6385-console-serving-cert\") pod \"console-5c5c9b69c-spw8w\" (UID: \"2415c1e4-3d58-4218-8d06-b7c4d05d6385\") " pod="openshift-console/console-5c5c9b69c-spw8w" Jul 07 15:15:56.304546 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:56.304525 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhnz5\" (UniqueName: \"kubernetes.io/projected/2415c1e4-3d58-4218-8d06-b7c4d05d6385-kube-api-access-fhnz5\") pod \"console-5c5c9b69c-spw8w\" (UID: \"2415c1e4-3d58-4218-8d06-b7c4d05d6385\") " pod="openshift-console/console-5c5c9b69c-spw8w" Jul 07 15:15:56.439772 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:56.439684 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c5c9b69c-spw8w" Jul 07 15:15:56.575215 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:56.575191 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c5c9b69c-spw8w"] Jul 07 15:15:56.577677 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:15:56.577654 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2415c1e4_3d58_4218_8d06_b7c4d05d6385.slice/crio-ce2f6990a01c040f8bdda34689d148f98b62b200dc40cdb65ceabdef3d37dc0b WatchSource:0}: Error finding container ce2f6990a01c040f8bdda34689d148f98b62b200dc40cdb65ceabdef3d37dc0b: Status 404 returned error can't find the container with id ce2f6990a01c040f8bdda34689d148f98b62b200dc40cdb65ceabdef3d37dc0b Jul 07 15:15:57.436674 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:57.436636 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c5c9b69c-spw8w" event={"ID":"2415c1e4-3d58-4218-8d06-b7c4d05d6385","Type":"ContainerStarted","Data":"6db658e653fa2d830d4c3668c7d09b8a48d2da11ed9997bf9181ee8114b80dca"} Jul 07 15:15:57.436849 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:57.436699 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c5c9b69c-spw8w" event={"ID":"2415c1e4-3d58-4218-8d06-b7c4d05d6385","Type":"ContainerStarted","Data":"ce2f6990a01c040f8bdda34689d148f98b62b200dc40cdb65ceabdef3d37dc0b"} Jul 07 15:15:57.461905 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:15:57.461845 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5c5c9b69c-spw8w" podStartSLOduration=1.461828755 podStartE2EDuration="1.461828755s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:15:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:15:57.460823557 +0000 UTC m=+175.468301010" watchObservedRunningTime="2026-07-07 15:15:57.461828755 +0000 UTC m=+175.469306245" Jul 07 15:16:06.440737 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:16:06.440697 2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5c5c9b69c-spw8w" Jul 07 15:16:06.441192 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:16:06.440756 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5c5c9b69c-spw8w" Jul 07 15:16:06.445920 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:16:06.445894 2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5c5c9b69c-spw8w" Jul 07 15:16:06.479328 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:16:06.479298 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5c5c9b69c-spw8w" Jul 07 15:16:06.535250 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:16:06.535222 2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57b986b797-tf2fq"] Jul 07 15:16:31.560782 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:16:31.560707 2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-57b986b797-tf2fq" podUID="2bc6442d-60c1-4296-ba74-aeb9a087c67b" containerName="console" containerID="cri-o://4c842d163561414225a97479f53a5e6b1387e5a9e9b26e73971f67b280b506d2" gracePeriod=15 Jul 07 15:16:31.810640 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:16:31.810615 2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57b986b797-tf2fq" Jul 07 15:16:31.882858 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:16:31.882827 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2bc6442d-60c1-4296-ba74-aeb9a087c67b-oauth-serving-cert\") pod \"2bc6442d-60c1-4296-ba74-aeb9a087c67b\" (UID: \"2bc6442d-60c1-4296-ba74-aeb9a087c67b\") " Jul 07 15:16:31.883056 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:16:31.882869 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2bc6442d-60c1-4296-ba74-aeb9a087c67b-console-serving-cert\") pod \"2bc6442d-60c1-4296-ba74-aeb9a087c67b\" (UID: \"2bc6442d-60c1-4296-ba74-aeb9a087c67b\") " Jul 07 15:16:31.883056 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:16:31.882900 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2bc6442d-60c1-4296-ba74-aeb9a087c67b-console-config\") pod \"2bc6442d-60c1-4296-ba74-aeb9a087c67b\" (UID: \"2bc6442d-60c1-4296-ba74-aeb9a087c67b\") " Jul 07 15:16:31.883056 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:16:31.882947 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jxbj6\" (UniqueName: \"kubernetes.io/projected/2bc6442d-60c1-4296-ba74-aeb9a087c67b-kube-api-access-jxbj6\") pod \"2bc6442d-60c1-4296-ba74-aeb9a087c67b\" (UID: \"2bc6442d-60c1-4296-ba74-aeb9a087c67b\") " Jul 07 15:16:31.883056 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:16:31.882983 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2bc6442d-60c1-4296-ba74-aeb9a087c67b-service-ca\") pod \"2bc6442d-60c1-4296-ba74-aeb9a087c67b\" (UID: \"2bc6442d-60c1-4296-ba74-aeb9a087c67b\") " Jul 07 15:16:31.883056 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:16:31.883015 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bc6442d-60c1-4296-ba74-aeb9a087c67b-trusted-ca-bundle\") pod \"2bc6442d-60c1-4296-ba74-aeb9a087c67b\" (UID: \"2bc6442d-60c1-4296-ba74-aeb9a087c67b\") " Jul 07 15:16:31.883056 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:16:31.883055 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2bc6442d-60c1-4296-ba74-aeb9a087c67b-console-oauth-config\") pod \"2bc6442d-60c1-4296-ba74-aeb9a087c67b\" (UID: \"2bc6442d-60c1-4296-ba74-aeb9a087c67b\") " Jul 07 15:16:31.883374 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:16:31.883340 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2bc6442d-60c1-4296-ba74-aeb9a087c67b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "2bc6442d-60c1-4296-ba74-aeb9a087c67b" (UID: "2bc6442d-60c1-4296-ba74-aeb9a087c67b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:16:31.883438 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:16:31.883355 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2bc6442d-60c1-4296-ba74-aeb9a087c67b-console-config" (OuterVolumeSpecName: "console-config") pod "2bc6442d-60c1-4296-ba74-aeb9a087c67b" (UID: "2bc6442d-60c1-4296-ba74-aeb9a087c67b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:16:31.883496 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:16:31.883436 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2bc6442d-60c1-4296-ba74-aeb9a087c67b-service-ca" (OuterVolumeSpecName: "service-ca") pod "2bc6442d-60c1-4296-ba74-aeb9a087c67b" (UID: "2bc6442d-60c1-4296-ba74-aeb9a087c67b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:16:31.883496 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:16:31.883461 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2bc6442d-60c1-4296-ba74-aeb9a087c67b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "2bc6442d-60c1-4296-ba74-aeb9a087c67b" (UID: "2bc6442d-60c1-4296-ba74-aeb9a087c67b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:16:31.885328 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:16:31.885299 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2bc6442d-60c1-4296-ba74-aeb9a087c67b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "2bc6442d-60c1-4296-ba74-aeb9a087c67b" (UID: "2bc6442d-60c1-4296-ba74-aeb9a087c67b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:16:31.885328 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:16:31.885317 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2bc6442d-60c1-4296-ba74-aeb9a087c67b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "2bc6442d-60c1-4296-ba74-aeb9a087c67b" (UID: "2bc6442d-60c1-4296-ba74-aeb9a087c67b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:16:31.885328 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:16:31.885324 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2bc6442d-60c1-4296-ba74-aeb9a087c67b-kube-api-access-jxbj6" (OuterVolumeSpecName: "kube-api-access-jxbj6") pod "2bc6442d-60c1-4296-ba74-aeb9a087c67b" (UID: "2bc6442d-60c1-4296-ba74-aeb9a087c67b"). InnerVolumeSpecName "kube-api-access-jxbj6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:16:31.983710 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:16:31.983669 2549 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2bc6442d-60c1-4296-ba74-aeb9a087c67b-service-ca\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:16:31.983710 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:16:31.983700 2549 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bc6442d-60c1-4296-ba74-aeb9a087c67b-trusted-ca-bundle\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:16:31.983710 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:16:31.983710 2549 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2bc6442d-60c1-4296-ba74-aeb9a087c67b-console-oauth-config\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:16:31.983710 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:16:31.983719 2549 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2bc6442d-60c1-4296-ba74-aeb9a087c67b-oauth-serving-cert\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:16:31.984017 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:16:31.983728 2549 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2bc6442d-60c1-4296-ba74-aeb9a087c67b-console-serving-cert\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:16:31.984017 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:16:31.983738 2549 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2bc6442d-60c1-4296-ba74-aeb9a087c67b-console-config\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:16:31.984017 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:16:31.983746 2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jxbj6\" (UniqueName: \"kubernetes.io/projected/2bc6442d-60c1-4296-ba74-aeb9a087c67b-kube-api-access-jxbj6\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:16:32.593997 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:16:32.593966 2549 generic.go:356] "Generic (PLEG): container finished" podID="2bc6442d-60c1-4296-ba74-aeb9a087c67b" containerID="4c842d163561414225a97479f53a5e6b1387e5a9e9b26e73971f67b280b506d2" exitCode=0 Jul 07 15:16:32.594421 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:16:32.594005 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57b986b797-tf2fq" event={"ID":"2bc6442d-60c1-4296-ba74-aeb9a087c67b","Type":"ContainerDied","Data":"4c842d163561414225a97479f53a5e6b1387e5a9e9b26e73971f67b280b506d2"} Jul 07 15:16:32.594421 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:16:32.594011 2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57b986b797-tf2fq" Jul 07 15:16:32.594421 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:16:32.594044 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57b986b797-tf2fq" event={"ID":"2bc6442d-60c1-4296-ba74-aeb9a087c67b","Type":"ContainerDied","Data":"d170ca05b73b84282d770ebf16fbb92e0d4c7e70f8b7ddd374030b9d1a140e94"} Jul 07 15:16:32.594421 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:16:32.594063 2549 scope.go:117] "RemoveContainer" containerID="4c842d163561414225a97479f53a5e6b1387e5a9e9b26e73971f67b280b506d2" Jul 07 15:16:32.602675 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:16:32.602657 2549 scope.go:117] "RemoveContainer" containerID="4c842d163561414225a97479f53a5e6b1387e5a9e9b26e73971f67b280b506d2" Jul 07 15:16:32.602928 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:16:32.602909 2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c842d163561414225a97479f53a5e6b1387e5a9e9b26e73971f67b280b506d2\": container with ID starting with 4c842d163561414225a97479f53a5e6b1387e5a9e9b26e73971f67b280b506d2 not found: ID does not exist" containerID="4c842d163561414225a97479f53a5e6b1387e5a9e9b26e73971f67b280b506d2" Jul 07 15:16:32.602996 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:16:32.602939 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c842d163561414225a97479f53a5e6b1387e5a9e9b26e73971f67b280b506d2"} err="failed to get container status \"4c842d163561414225a97479f53a5e6b1387e5a9e9b26e73971f67b280b506d2\": rpc error: code = NotFound desc = could not find container \"4c842d163561414225a97479f53a5e6b1387e5a9e9b26e73971f67b280b506d2\": container with ID starting with 4c842d163561414225a97479f53a5e6b1387e5a9e9b26e73971f67b280b506d2 not found: ID does not exist" Jul 07 15:16:32.628457 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:16:32.628427 2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57b986b797-tf2fq"] Jul 07 15:16:32.636989 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:16:32.636966 2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-57b986b797-tf2fq"] Jul 07 15:16:34.534203 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:16:34.534162 2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2bc6442d-60c1-4296-ba74-aeb9a087c67b" path="/var/lib/kubelet/pods/2bc6442d-60c1-4296-ba74-aeb9a087c67b/volumes" Jul 07 15:17:29.158699 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:17:29.158562 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-754c6f66b8-jdbdh"] Jul 07 15:17:29.159178 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:17:29.159029 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bc6442d-60c1-4296-ba74-aeb9a087c67b" containerName="console" Jul 07 15:17:29.159178 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:17:29.159064 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bc6442d-60c1-4296-ba74-aeb9a087c67b" containerName="console" Jul 07 15:17:29.159282 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:17:29.159191 2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bc6442d-60c1-4296-ba74-aeb9a087c67b" containerName="console" Jul 07 15:17:29.161391 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:17:29.161373 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-754c6f66b8-jdbdh" Jul 07 15:17:29.173391 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:17:29.173359 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d287bac1-b960-4585-b80a-0b3a93f04782-console-oauth-config\") pod \"console-754c6f66b8-jdbdh\" (UID: \"d287bac1-b960-4585-b80a-0b3a93f04782\") " pod="openshift-console/console-754c6f66b8-jdbdh" Jul 07 15:17:29.173520 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:17:29.173422 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d287bac1-b960-4585-b80a-0b3a93f04782-oauth-serving-cert\") pod \"console-754c6f66b8-jdbdh\" (UID: \"d287bac1-b960-4585-b80a-0b3a93f04782\") " pod="openshift-console/console-754c6f66b8-jdbdh" Jul 07 15:17:29.173520 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:17:29.173481 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d287bac1-b960-4585-b80a-0b3a93f04782-console-serving-cert\") pod \"console-754c6f66b8-jdbdh\" (UID: \"d287bac1-b960-4585-b80a-0b3a93f04782\") " pod="openshift-console/console-754c6f66b8-jdbdh" Jul 07 15:17:29.173675 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:17:29.173533 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d287bac1-b960-4585-b80a-0b3a93f04782-trusted-ca-bundle\") pod \"console-754c6f66b8-jdbdh\" (UID: \"d287bac1-b960-4585-b80a-0b3a93f04782\") " pod="openshift-console/console-754c6f66b8-jdbdh" Jul 07 15:17:29.173732 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:17:29.173665 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-947hl\" (UniqueName: \"kubernetes.io/projected/d287bac1-b960-4585-b80a-0b3a93f04782-kube-api-access-947hl\") pod \"console-754c6f66b8-jdbdh\" (UID: \"d287bac1-b960-4585-b80a-0b3a93f04782\") " pod="openshift-console/console-754c6f66b8-jdbdh" Jul 07 15:17:29.173732 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:17:29.173714 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d287bac1-b960-4585-b80a-0b3a93f04782-console-config\") pod \"console-754c6f66b8-jdbdh\" (UID: \"d287bac1-b960-4585-b80a-0b3a93f04782\") " pod="openshift-console/console-754c6f66b8-jdbdh" Jul 07 15:17:29.173878 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:17:29.173743 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d287bac1-b960-4585-b80a-0b3a93f04782-service-ca\") pod \"console-754c6f66b8-jdbdh\" (UID: \"d287bac1-b960-4585-b80a-0b3a93f04782\") " pod="openshift-console/console-754c6f66b8-jdbdh" Jul 07 15:17:29.174244 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:17:29.174221 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-754c6f66b8-jdbdh"] Jul 07 15:17:29.274609 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:17:29.274544 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d287bac1-b960-4585-b80a-0b3a93f04782-console-config\") pod \"console-754c6f66b8-jdbdh\" (UID: \"d287bac1-b960-4585-b80a-0b3a93f04782\") " pod="openshift-console/console-754c6f66b8-jdbdh" Jul 07 15:17:29.274609 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:17:29.274614 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d287bac1-b960-4585-b80a-0b3a93f04782-service-ca\") pod \"console-754c6f66b8-jdbdh\" (UID: \"d287bac1-b960-4585-b80a-0b3a93f04782\") " pod="openshift-console/console-754c6f66b8-jdbdh" Jul 07 15:17:29.274869 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:17:29.274666 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d287bac1-b960-4585-b80a-0b3a93f04782-console-oauth-config\") pod \"console-754c6f66b8-jdbdh\" (UID: \"d287bac1-b960-4585-b80a-0b3a93f04782\") " pod="openshift-console/console-754c6f66b8-jdbdh" Jul 07 15:17:29.274869 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:17:29.274683 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d287bac1-b960-4585-b80a-0b3a93f04782-oauth-serving-cert\") pod \"console-754c6f66b8-jdbdh\" (UID: \"d287bac1-b960-4585-b80a-0b3a93f04782\") " pod="openshift-console/console-754c6f66b8-jdbdh" Jul 07 15:17:29.274869 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:17:29.274703 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d287bac1-b960-4585-b80a-0b3a93f04782-console-serving-cert\") pod \"console-754c6f66b8-jdbdh\" (UID: \"d287bac1-b960-4585-b80a-0b3a93f04782\") " pod="openshift-console/console-754c6f66b8-jdbdh" Jul 07 15:17:29.274869 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:17:29.274741 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d287bac1-b960-4585-b80a-0b3a93f04782-trusted-ca-bundle\") pod \"console-754c6f66b8-jdbdh\" (UID: \"d287bac1-b960-4585-b80a-0b3a93f04782\") " pod="openshift-console/console-754c6f66b8-jdbdh" Jul 07 15:17:29.274869 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:17:29.274820 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-947hl\" (UniqueName: \"kubernetes.io/projected/d287bac1-b960-4585-b80a-0b3a93f04782-kube-api-access-947hl\") pod \"console-754c6f66b8-jdbdh\" (UID: \"d287bac1-b960-4585-b80a-0b3a93f04782\") " pod="openshift-console/console-754c6f66b8-jdbdh" Jul 07 15:17:29.275398 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:17:29.275372 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d287bac1-b960-4585-b80a-0b3a93f04782-service-ca\") pod \"console-754c6f66b8-jdbdh\" (UID: \"d287bac1-b960-4585-b80a-0b3a93f04782\") " pod="openshift-console/console-754c6f66b8-jdbdh" Jul 07 15:17:29.275513 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:17:29.275374 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d287bac1-b960-4585-b80a-0b3a93f04782-console-config\") pod \"console-754c6f66b8-jdbdh\" (UID: \"d287bac1-b960-4585-b80a-0b3a93f04782\") " pod="openshift-console/console-754c6f66b8-jdbdh" Jul 07 15:17:29.275513 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:17:29.275445 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d287bac1-b960-4585-b80a-0b3a93f04782-oauth-serving-cert\") pod \"console-754c6f66b8-jdbdh\" (UID: \"d287bac1-b960-4585-b80a-0b3a93f04782\") " pod="openshift-console/console-754c6f66b8-jdbdh" Jul 07 15:17:29.275704 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:17:29.275683 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d287bac1-b960-4585-b80a-0b3a93f04782-trusted-ca-bundle\") pod \"console-754c6f66b8-jdbdh\" (UID: \"d287bac1-b960-4585-b80a-0b3a93f04782\") " pod="openshift-console/console-754c6f66b8-jdbdh" Jul 07 15:17:29.277645 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:17:29.277619 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d287bac1-b960-4585-b80a-0b3a93f04782-console-oauth-config\") pod \"console-754c6f66b8-jdbdh\" (UID: \"d287bac1-b960-4585-b80a-0b3a93f04782\") " pod="openshift-console/console-754c6f66b8-jdbdh" Jul 07 15:17:29.277836 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:17:29.277814 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d287bac1-b960-4585-b80a-0b3a93f04782-console-serving-cert\") pod \"console-754c6f66b8-jdbdh\" (UID: \"d287bac1-b960-4585-b80a-0b3a93f04782\") " pod="openshift-console/console-754c6f66b8-jdbdh" Jul 07 15:17:29.285323 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:17:29.285292 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-947hl\" (UniqueName: \"kubernetes.io/projected/d287bac1-b960-4585-b80a-0b3a93f04782-kube-api-access-947hl\") pod \"console-754c6f66b8-jdbdh\" (UID: \"d287bac1-b960-4585-b80a-0b3a93f04782\") " pod="openshift-console/console-754c6f66b8-jdbdh" Jul 07 15:17:29.471057 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:17:29.470969 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-754c6f66b8-jdbdh" Jul 07 15:17:29.601777 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:17:29.601734 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-754c6f66b8-jdbdh"] Jul 07 15:17:29.605029 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:17:29.604994 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd287bac1_b960_4585_b80a_0b3a93f04782.slice/crio-d21b5353c0f8a3da0bde970131c0f3ee9100b10fbb5d07f75027f069045ac854 WatchSource:0}: Error finding container d21b5353c0f8a3da0bde970131c0f3ee9100b10fbb5d07f75027f069045ac854: Status 404 returned error can't find the container with id d21b5353c0f8a3da0bde970131c0f3ee9100b10fbb5d07f75027f069045ac854 Jul 07 15:17:29.852425 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:17:29.852384 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-754c6f66b8-jdbdh" event={"ID":"d287bac1-b960-4585-b80a-0b3a93f04782","Type":"ContainerStarted","Data":"4623568c843a6cccf4922fed2448cf62ac8607905b659d82cd410bfacb514fd2"} Jul 07 15:17:29.852662 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:17:29.852439 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-754c6f66b8-jdbdh" event={"ID":"d287bac1-b960-4585-b80a-0b3a93f04782","Type":"ContainerStarted","Data":"d21b5353c0f8a3da0bde970131c0f3ee9100b10fbb5d07f75027f069045ac854"} Jul 07 15:17:29.872390 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:17:29.872338 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-754c6f66b8-jdbdh" podStartSLOduration=0.872318277 podStartE2EDuration="872.318277ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:17:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:17:29.871555208 +0000 UTC m=+267.879032658" watchObservedRunningTime="2026-07-07 15:17:29.872318277 +0000 UTC m=+267.879795730" Jul 07 15:17:39.471404 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:17:39.471367 2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-754c6f66b8-jdbdh" Jul 07 15:17:39.471919 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:17:39.471430 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-754c6f66b8-jdbdh" Jul 07 15:17:39.476406 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:17:39.476383 2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-754c6f66b8-jdbdh" Jul 07 15:17:39.900994 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:17:39.900967 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-754c6f66b8-jdbdh" Jul 07 15:17:39.972403 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:17:39.972377 2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c5c9b69c-spw8w"] Jul 07 15:18:02.361873 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:02.361832 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kb4ml_3301f6ef-6671-4755-9047-16959836232c/console-operator/2.log" Jul 07 15:18:02.368993 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:02.368969 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kb4ml_3301f6ef-6671-4755-9047-16959836232c/console-operator/2.log" Jul 07 15:18:02.380814 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:02.380788 2549 kubelet.go:1648] "Image garbage collection succeeded" Jul 07 15:18:05.001096 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:05.001028 2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5c5c9b69c-spw8w" podUID="2415c1e4-3d58-4218-8d06-b7c4d05d6385" containerName="console" containerID="cri-o://6db658e653fa2d830d4c3668c7d09b8a48d2da11ed9997bf9181ee8114b80dca" gracePeriod=15 Jul 07 15:18:05.241333 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:05.241306 2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c5c9b69c-spw8w" Jul 07 15:18:05.384549 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:05.384516 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2415c1e4-3d58-4218-8d06-b7c4d05d6385-oauth-serving-cert\") pod \"2415c1e4-3d58-4218-8d06-b7c4d05d6385\" (UID: \"2415c1e4-3d58-4218-8d06-b7c4d05d6385\") " Jul 07 15:18:05.384753 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:05.384572 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2415c1e4-3d58-4218-8d06-b7c4d05d6385-console-config\") pod \"2415c1e4-3d58-4218-8d06-b7c4d05d6385\" (UID: \"2415c1e4-3d58-4218-8d06-b7c4d05d6385\") " Jul 07 15:18:05.384753 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:05.384661 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2415c1e4-3d58-4218-8d06-b7c4d05d6385-service-ca\") pod \"2415c1e4-3d58-4218-8d06-b7c4d05d6385\" (UID: \"2415c1e4-3d58-4218-8d06-b7c4d05d6385\") " Jul 07 15:18:05.384753 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:05.384677 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2415c1e4-3d58-4218-8d06-b7c4d05d6385-console-serving-cert\") pod \"2415c1e4-3d58-4218-8d06-b7c4d05d6385\" (UID: \"2415c1e4-3d58-4218-8d06-b7c4d05d6385\") " Jul 07 15:18:05.384753 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:05.384719 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2415c1e4-3d58-4218-8d06-b7c4d05d6385-trusted-ca-bundle\") pod \"2415c1e4-3d58-4218-8d06-b7c4d05d6385\" (UID: \"2415c1e4-3d58-4218-8d06-b7c4d05d6385\") " Jul 07 15:18:05.384753 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:05.384736 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fhnz5\" (UniqueName: \"kubernetes.io/projected/2415c1e4-3d58-4218-8d06-b7c4d05d6385-kube-api-access-fhnz5\") pod \"2415c1e4-3d58-4218-8d06-b7c4d05d6385\" (UID: \"2415c1e4-3d58-4218-8d06-b7c4d05d6385\") " Jul 07 15:18:05.385003 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:05.384772 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2415c1e4-3d58-4218-8d06-b7c4d05d6385-console-oauth-config\") pod \"2415c1e4-3d58-4218-8d06-b7c4d05d6385\" (UID: \"2415c1e4-3d58-4218-8d06-b7c4d05d6385\") " Jul 07 15:18:05.385003 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:05.384961 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2415c1e4-3d58-4218-8d06-b7c4d05d6385-console-config" (OuterVolumeSpecName: "console-config") pod "2415c1e4-3d58-4218-8d06-b7c4d05d6385" (UID: "2415c1e4-3d58-4218-8d06-b7c4d05d6385"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:18:05.385129 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:05.385098 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2415c1e4-3d58-4218-8d06-b7c4d05d6385-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "2415c1e4-3d58-4218-8d06-b7c4d05d6385" (UID: "2415c1e4-3d58-4218-8d06-b7c4d05d6385"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:18:05.385190 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:05.385103 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2415c1e4-3d58-4218-8d06-b7c4d05d6385-service-ca" (OuterVolumeSpecName: "service-ca") pod "2415c1e4-3d58-4218-8d06-b7c4d05d6385" (UID: "2415c1e4-3d58-4218-8d06-b7c4d05d6385"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:18:05.385251 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:05.385229 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2415c1e4-3d58-4218-8d06-b7c4d05d6385-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "2415c1e4-3d58-4218-8d06-b7c4d05d6385" (UID: "2415c1e4-3d58-4218-8d06-b7c4d05d6385"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:18:05.386990 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:05.386957 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2415c1e4-3d58-4218-8d06-b7c4d05d6385-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "2415c1e4-3d58-4218-8d06-b7c4d05d6385" (UID: "2415c1e4-3d58-4218-8d06-b7c4d05d6385"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:18:05.386990 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:05.386978 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2415c1e4-3d58-4218-8d06-b7c4d05d6385-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "2415c1e4-3d58-4218-8d06-b7c4d05d6385" (UID: "2415c1e4-3d58-4218-8d06-b7c4d05d6385"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:18:05.387168 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:05.387073 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2415c1e4-3d58-4218-8d06-b7c4d05d6385-kube-api-access-fhnz5" (OuterVolumeSpecName: "kube-api-access-fhnz5") pod "2415c1e4-3d58-4218-8d06-b7c4d05d6385" (UID: "2415c1e4-3d58-4218-8d06-b7c4d05d6385"). InnerVolumeSpecName "kube-api-access-fhnz5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:18:05.485723 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:05.485669 2549 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2415c1e4-3d58-4218-8d06-b7c4d05d6385-service-ca\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:18:05.485723 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:05.485716 2549 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2415c1e4-3d58-4218-8d06-b7c4d05d6385-console-serving-cert\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:18:05.485723 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:05.485731 2549 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2415c1e4-3d58-4218-8d06-b7c4d05d6385-trusted-ca-bundle\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:18:05.485977 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:05.485744 2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fhnz5\" (UniqueName: \"kubernetes.io/projected/2415c1e4-3d58-4218-8d06-b7c4d05d6385-kube-api-access-fhnz5\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:18:05.485977 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:05.485756 2549 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2415c1e4-3d58-4218-8d06-b7c4d05d6385-console-oauth-config\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:18:05.485977 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:05.485784 2549 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2415c1e4-3d58-4218-8d06-b7c4d05d6385-oauth-serving-cert\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:18:05.485977 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:05.485795 2549 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2415c1e4-3d58-4218-8d06-b7c4d05d6385-console-config\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:18:06.015780 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:06.015744 2549 generic.go:356] "Generic (PLEG): container finished" podID="2415c1e4-3d58-4218-8d06-b7c4d05d6385" containerID="6db658e653fa2d830d4c3668c7d09b8a48d2da11ed9997bf9181ee8114b80dca" exitCode=0 Jul 07 15:18:06.016208 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:06.015787 2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c5c9b69c-spw8w" Jul 07 15:18:06.016208 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:06.015798 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c5c9b69c-spw8w" event={"ID":"2415c1e4-3d58-4218-8d06-b7c4d05d6385","Type":"ContainerDied","Data":"6db658e653fa2d830d4c3668c7d09b8a48d2da11ed9997bf9181ee8114b80dca"} Jul 07 15:18:06.016208 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:06.015846 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c5c9b69c-spw8w" event={"ID":"2415c1e4-3d58-4218-8d06-b7c4d05d6385","Type":"ContainerDied","Data":"ce2f6990a01c040f8bdda34689d148f98b62b200dc40cdb65ceabdef3d37dc0b"} Jul 07 15:18:06.016208 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:06.015868 2549 scope.go:117] "RemoveContainer" containerID="6db658e653fa2d830d4c3668c7d09b8a48d2da11ed9997bf9181ee8114b80dca" Jul 07 15:18:06.025082 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:06.025061 2549 scope.go:117] "RemoveContainer" containerID="6db658e653fa2d830d4c3668c7d09b8a48d2da11ed9997bf9181ee8114b80dca" Jul 07 15:18:06.025328 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:18:06.025310 2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6db658e653fa2d830d4c3668c7d09b8a48d2da11ed9997bf9181ee8114b80dca\": container with ID starting with 6db658e653fa2d830d4c3668c7d09b8a48d2da11ed9997bf9181ee8114b80dca not found: ID does not exist" containerID="6db658e653fa2d830d4c3668c7d09b8a48d2da11ed9997bf9181ee8114b80dca" Jul 07 15:18:06.025368 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:06.025338 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6db658e653fa2d830d4c3668c7d09b8a48d2da11ed9997bf9181ee8114b80dca"} err="failed to get container status \"6db658e653fa2d830d4c3668c7d09b8a48d2da11ed9997bf9181ee8114b80dca\": rpc error: code = NotFound desc = could not find container \"6db658e653fa2d830d4c3668c7d09b8a48d2da11ed9997bf9181ee8114b80dca\": container with ID starting with 6db658e653fa2d830d4c3668c7d09b8a48d2da11ed9997bf9181ee8114b80dca not found: ID does not exist" Jul 07 15:18:06.037361 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:06.037337 2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c5c9b69c-spw8w"] Jul 07 15:18:06.042131 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:06.042112 2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5c5c9b69c-spw8w"] Jul 07 15:18:06.533815 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:06.533783 2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2415c1e4-3d58-4218-8d06-b7c4d05d6385" path="/var/lib/kubelet/pods/2415c1e4-3d58-4218-8d06-b7c4d05d6385/volumes" Jul 07 15:18:31.106720 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:31.106681 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-br7sz"] Jul 07 15:18:31.107203 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:31.107049 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2415c1e4-3d58-4218-8d06-b7c4d05d6385" containerName="console" Jul 07 15:18:31.107203 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:31.107061 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="2415c1e4-3d58-4218-8d06-b7c4d05d6385" containerName="console" Jul 07 15:18:31.107203 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:31.107147 2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="2415c1e4-3d58-4218-8d06-b7c4d05d6385" containerName="console" Jul 07 15:18:31.109132 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:31.109117 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-br7sz" Jul 07 15:18:31.112146 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:31.112120 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 07 15:18:31.112457 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:31.112439 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-mq2n7\"" Jul 07 15:18:31.120002 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:31.119983 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 07 15:18:31.126610 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:31.126570 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-br7sz"] Jul 07 15:18:31.208860 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:31.208827 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nb6h6\" (UniqueName: \"kubernetes.io/projected/198db941-daa3-4783-98c3-3b02bd25b8b7-kube-api-access-nb6h6\") pod \"cert-manager-operator-controller-manager-55c444bf86-br7sz\" (UID: \"198db941-daa3-4783-98c3-3b02bd25b8b7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-br7sz" Jul 07 15:18:31.209046 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:31.208876 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/198db941-daa3-4783-98c3-3b02bd25b8b7-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-br7sz\" (UID: \"198db941-daa3-4783-98c3-3b02bd25b8b7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-br7sz" Jul 07 15:18:31.309820 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:31.309788 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/198db941-daa3-4783-98c3-3b02bd25b8b7-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-br7sz\" (UID: \"198db941-daa3-4783-98c3-3b02bd25b8b7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-br7sz" Jul 07 15:18:31.309992 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:31.309867 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nb6h6\" (UniqueName: \"kubernetes.io/projected/198db941-daa3-4783-98c3-3b02bd25b8b7-kube-api-access-nb6h6\") pod \"cert-manager-operator-controller-manager-55c444bf86-br7sz\" (UID: \"198db941-daa3-4783-98c3-3b02bd25b8b7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-br7sz" Jul 07 15:18:31.310228 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:31.310209 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/198db941-daa3-4783-98c3-3b02bd25b8b7-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-br7sz\" (UID: \"198db941-daa3-4783-98c3-3b02bd25b8b7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-br7sz" Jul 07 15:18:31.318081 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:31.318060 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nb6h6\" (UniqueName: \"kubernetes.io/projected/198db941-daa3-4783-98c3-3b02bd25b8b7-kube-api-access-nb6h6\") pod \"cert-manager-operator-controller-manager-55c444bf86-br7sz\" (UID: \"198db941-daa3-4783-98c3-3b02bd25b8b7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-br7sz" Jul 07 15:18:31.418152 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:31.418058 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-br7sz" Jul 07 15:18:31.554874 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:31.554830 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-br7sz"] Jul 07 15:18:31.557524 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:18:31.557500 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod198db941_daa3_4783_98c3_3b02bd25b8b7.slice/crio-bc9027990d67ffc5dec366560193a17123b9c54d0309e4cb1d4b30f73c622a7d WatchSource:0}: Error finding container bc9027990d67ffc5dec366560193a17123b9c54d0309e4cb1d4b30f73c622a7d: Status 404 returned error can't find the container with id bc9027990d67ffc5dec366560193a17123b9c54d0309e4cb1d4b30f73c622a7d Jul 07 15:18:31.560831 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:31.560814 2549 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 15:18:32.137937 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:32.137902 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-br7sz" event={"ID":"198db941-daa3-4783-98c3-3b02bd25b8b7","Type":"ContainerStarted","Data":"bc9027990d67ffc5dec366560193a17123b9c54d0309e4cb1d4b30f73c622a7d"} Jul 07 15:18:35.154739 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:35.154699 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-br7sz" event={"ID":"198db941-daa3-4783-98c3-3b02bd25b8b7","Type":"ContainerStarted","Data":"70020cf3dc6ca498ac4fdc1d23bf7f0869aa162e93de125f0ee1724ed7f32cc6"} Jul 07 15:18:35.179462 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:35.179414 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-br7sz" podStartSLOduration=1.61521954 podStartE2EDuration="4.179393615s" totalImagesPullingTime="2.564174075s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:18:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:18:35.178576046 +0000 UTC m=+333.186053510" watchObservedRunningTime="2026-07-07 15:18:35.179393615 +0000 UTC m=+333.186871107" Jul 07 15:18:35.801151 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:35.801121 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5hhq8"] Jul 07 15:18:35.803990 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:35.803968 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5hhq8" Jul 07 15:18:35.808627 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:35.808605 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-5kvd6\"" Jul 07 15:18:35.808730 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:35.808691 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 07 15:18:35.808730 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:35.808691 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 07 15:18:35.818282 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:35.818258 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5hhq8"] Jul 07 15:18:35.852010 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:35.851978 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/50960239-91db-4e67-aac4-f8ed1d5ca609-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5hhq8\" (UID: \"50960239-91db-4e67-aac4-f8ed1d5ca609\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5hhq8" Jul 07 15:18:35.852188 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:35.852035 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/50960239-91db-4e67-aac4-f8ed1d5ca609-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5hhq8\" (UID: \"50960239-91db-4e67-aac4-f8ed1d5ca609\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5hhq8" Jul 07 15:18:35.852188 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:35.852098 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jwzg\" (UniqueName: \"kubernetes.io/projected/50960239-91db-4e67-aac4-f8ed1d5ca609-kube-api-access-2jwzg\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5hhq8\" (UID: \"50960239-91db-4e67-aac4-f8ed1d5ca609\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5hhq8" Jul 07 15:18:35.952747 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:35.952710 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/50960239-91db-4e67-aac4-f8ed1d5ca609-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5hhq8\" (UID: \"50960239-91db-4e67-aac4-f8ed1d5ca609\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5hhq8" Jul 07 15:18:35.952920 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:35.952768 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/50960239-91db-4e67-aac4-f8ed1d5ca609-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5hhq8\" (UID: \"50960239-91db-4e67-aac4-f8ed1d5ca609\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5hhq8" Jul 07 15:18:35.952920 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:35.952793 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jwzg\" (UniqueName: \"kubernetes.io/projected/50960239-91db-4e67-aac4-f8ed1d5ca609-kube-api-access-2jwzg\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5hhq8\" (UID: \"50960239-91db-4e67-aac4-f8ed1d5ca609\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5hhq8" Jul 07 15:18:35.953106 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:35.953085 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/50960239-91db-4e67-aac4-f8ed1d5ca609-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5hhq8\" (UID: \"50960239-91db-4e67-aac4-f8ed1d5ca609\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5hhq8" Jul 07 15:18:35.953147 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:35.953129 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/50960239-91db-4e67-aac4-f8ed1d5ca609-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5hhq8\" (UID: \"50960239-91db-4e67-aac4-f8ed1d5ca609\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5hhq8" Jul 07 15:18:35.961806 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:35.961774 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jwzg\" (UniqueName: \"kubernetes.io/projected/50960239-91db-4e67-aac4-f8ed1d5ca609-kube-api-access-2jwzg\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5hhq8\" (UID: \"50960239-91db-4e67-aac4-f8ed1d5ca609\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5hhq8" Jul 07 15:18:36.114130 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:36.114096 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5hhq8" Jul 07 15:18:36.253916 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:36.253893 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5hhq8"] Jul 07 15:18:36.255652 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:18:36.255615 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50960239_91db_4e67_aac4_f8ed1d5ca609.slice/crio-fd59291a35e8f77165ec425fdb1a8a946c71803dd4f31b994bf36aa468ed5e39 WatchSource:0}: Error finding container fd59291a35e8f77165ec425fdb1a8a946c71803dd4f31b994bf36aa468ed5e39: Status 404 returned error can't find the container with id fd59291a35e8f77165ec425fdb1a8a946c71803dd4f31b994bf36aa468ed5e39 Jul 07 15:18:37.166196 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:37.166156 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5hhq8" event={"ID":"50960239-91db-4e67-aac4-f8ed1d5ca609","Type":"ContainerStarted","Data":"fd59291a35e8f77165ec425fdb1a8a946c71803dd4f31b994bf36aa468ed5e39"} Jul 07 15:18:42.668144 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:42.668104 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-c6lwj"] Jul 07 15:18:42.672391 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:42.672368 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-c6lwj" Jul 07 15:18:42.675806 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:42.675782 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-v2pvb\"" Jul 07 15:18:42.676424 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:42.676402 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 07 15:18:42.676972 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:42.676949 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 07 15:18:42.684281 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:42.684257 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-c6lwj"] Jul 07 15:18:42.711508 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:42.711469 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d0c02b43-5b20-40ff-8850-03b609d2ce50-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-c6lwj\" (UID: \"d0c02b43-5b20-40ff-8850-03b609d2ce50\") " pod="cert-manager/cert-manager-webhook-6486ffd997-c6lwj" Jul 07 15:18:42.711692 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:42.711564 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qpnl\" (UniqueName: \"kubernetes.io/projected/d0c02b43-5b20-40ff-8850-03b609d2ce50-kube-api-access-7qpnl\") pod \"cert-manager-webhook-6486ffd997-c6lwj\" (UID: \"d0c02b43-5b20-40ff-8850-03b609d2ce50\") " pod="cert-manager/cert-manager-webhook-6486ffd997-c6lwj" Jul 07 15:18:42.812304 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:42.812270 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d0c02b43-5b20-40ff-8850-03b609d2ce50-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-c6lwj\" (UID: \"d0c02b43-5b20-40ff-8850-03b609d2ce50\") " pod="cert-manager/cert-manager-webhook-6486ffd997-c6lwj" Jul 07 15:18:42.812488 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:42.812335 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qpnl\" (UniqueName: \"kubernetes.io/projected/d0c02b43-5b20-40ff-8850-03b609d2ce50-kube-api-access-7qpnl\") pod \"cert-manager-webhook-6486ffd997-c6lwj\" (UID: \"d0c02b43-5b20-40ff-8850-03b609d2ce50\") " pod="cert-manager/cert-manager-webhook-6486ffd997-c6lwj" Jul 07 15:18:42.821618 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:42.821555 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d0c02b43-5b20-40ff-8850-03b609d2ce50-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-c6lwj\" (UID: \"d0c02b43-5b20-40ff-8850-03b609d2ce50\") " pod="cert-manager/cert-manager-webhook-6486ffd997-c6lwj" Jul 07 15:18:42.821782 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:42.821748 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qpnl\" (UniqueName: \"kubernetes.io/projected/d0c02b43-5b20-40ff-8850-03b609d2ce50-kube-api-access-7qpnl\") pod \"cert-manager-webhook-6486ffd997-c6lwj\" (UID: \"d0c02b43-5b20-40ff-8850-03b609d2ce50\") " pod="cert-manager/cert-manager-webhook-6486ffd997-c6lwj" Jul 07 15:18:42.993722 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:42.993644 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-c6lwj" Jul 07 15:18:43.500081 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:43.500057 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-c6lwj"] Jul 07 15:18:43.502325 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:18:43.502300 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd0c02b43_5b20_40ff_8850_03b609d2ce50.slice/crio-8d66aead9fa87bd5c91bdc74f4b0f729d5b0e9d3ba8fdea3973d6bd7221a2bc0 WatchSource:0}: Error finding container 8d66aead9fa87bd5c91bdc74f4b0f729d5b0e9d3ba8fdea3973d6bd7221a2bc0: Status 404 returned error can't find the container with id 8d66aead9fa87bd5c91bdc74f4b0f729d5b0e9d3ba8fdea3973d6bd7221a2bc0 Jul 07 15:18:44.201452 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:44.201361 2549 generic.go:356] "Generic (PLEG): container finished" podID="50960239-91db-4e67-aac4-f8ed1d5ca609" containerID="a018ac63b022dc37b6cc1e12cb3a817f0ff569e2a0a15b6ca68bb7dd70477ee0" exitCode=0 Jul 07 15:18:44.201893 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:44.201449 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5hhq8" event={"ID":"50960239-91db-4e67-aac4-f8ed1d5ca609","Type":"ContainerDied","Data":"a018ac63b022dc37b6cc1e12cb3a817f0ff569e2a0a15b6ca68bb7dd70477ee0"} Jul 07 15:18:44.202795 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:44.202761 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-c6lwj" event={"ID":"d0c02b43-5b20-40ff-8850-03b609d2ce50","Type":"ContainerStarted","Data":"8d66aead9fa87bd5c91bdc74f4b0f729d5b0e9d3ba8fdea3973d6bd7221a2bc0"} Jul 07 15:18:48.226568 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:48.226535 2549 generic.go:356] "Generic (PLEG): container finished" podID="50960239-91db-4e67-aac4-f8ed1d5ca609" containerID="84208846f067741669ebb5d5c7d1bedf0a7f63d2123e122cc5611f41949cc542" exitCode=0 Jul 07 15:18:48.227021 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:48.226621 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5hhq8" event={"ID":"50960239-91db-4e67-aac4-f8ed1d5ca609","Type":"ContainerDied","Data":"84208846f067741669ebb5d5c7d1bedf0a7f63d2123e122cc5611f41949cc542"} Jul 07 15:18:48.228189 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:48.228169 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-c6lwj" event={"ID":"d0c02b43-5b20-40ff-8850-03b609d2ce50","Type":"ContainerStarted","Data":"ee361d9166601855d72db4c4455020120e6ec8315afee23e09eac42bcf99f724"} Jul 07 15:18:48.228297 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:48.228283 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-c6lwj" Jul 07 15:18:48.260336 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:48.260192 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-c6lwj" podStartSLOduration=2.63771918 podStartE2EDuration="6.260174341s" totalImagesPullingTime="3.622455161s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:18:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:18:48.259825237 +0000 UTC m=+346.267302700" watchObservedRunningTime="2026-07-07 15:18:48.260174341 +0000 UTC m=+346.267651796" Jul 07 15:18:54.235601 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:54.235491 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-c6lwj" Jul 07 15:18:57.276072 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:57.275986 2549 generic.go:356] "Generic (PLEG): container finished" podID="50960239-91db-4e67-aac4-f8ed1d5ca609" containerID="5d97ab81d2f0583b32ac61f1856f059b3768140b236a3e1e4b628dc434b9c3ca" exitCode=0 Jul 07 15:18:57.276072 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:57.276052 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5hhq8" event={"ID":"50960239-91db-4e67-aac4-f8ed1d5ca609","Type":"ContainerDied","Data":"5d97ab81d2f0583b32ac61f1856f059b3768140b236a3e1e4b628dc434b9c3ca"} Jul 07 15:18:58.466095 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:58.466058 2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5hhq8" Jul 07 15:18:58.558862 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:58.558785 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/50960239-91db-4e67-aac4-f8ed1d5ca609-util\") pod \"50960239-91db-4e67-aac4-f8ed1d5ca609\" (UID: \"50960239-91db-4e67-aac4-f8ed1d5ca609\") " Jul 07 15:18:58.558862 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:58.558859 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2jwzg\" (UniqueName: \"kubernetes.io/projected/50960239-91db-4e67-aac4-f8ed1d5ca609-kube-api-access-2jwzg\") pod \"50960239-91db-4e67-aac4-f8ed1d5ca609\" (UID: \"50960239-91db-4e67-aac4-f8ed1d5ca609\") " Jul 07 15:18:58.559066 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:58.558892 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/50960239-91db-4e67-aac4-f8ed1d5ca609-bundle\") pod \"50960239-91db-4e67-aac4-f8ed1d5ca609\" (UID: \"50960239-91db-4e67-aac4-f8ed1d5ca609\") " Jul 07 15:18:58.559287 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:58.559264 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50960239-91db-4e67-aac4-f8ed1d5ca609-bundle" (OuterVolumeSpecName: "bundle") pod "50960239-91db-4e67-aac4-f8ed1d5ca609" (UID: "50960239-91db-4e67-aac4-f8ed1d5ca609"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:18:58.561058 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:58.561036 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50960239-91db-4e67-aac4-f8ed1d5ca609-kube-api-access-2jwzg" (OuterVolumeSpecName: "kube-api-access-2jwzg") pod "50960239-91db-4e67-aac4-f8ed1d5ca609" (UID: "50960239-91db-4e67-aac4-f8ed1d5ca609"). InnerVolumeSpecName "kube-api-access-2jwzg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:18:58.562927 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:58.562908 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50960239-91db-4e67-aac4-f8ed1d5ca609-util" (OuterVolumeSpecName: "util") pod "50960239-91db-4e67-aac4-f8ed1d5ca609" (UID: "50960239-91db-4e67-aac4-f8ed1d5ca609"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:18:58.660026 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:58.659983 2549 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/50960239-91db-4e67-aac4-f8ed1d5ca609-util\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:18:58.660026 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:58.660021 2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2jwzg\" (UniqueName: \"kubernetes.io/projected/50960239-91db-4e67-aac4-f8ed1d5ca609-kube-api-access-2jwzg\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:18:58.660026 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:58.660036 2549 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/50960239-91db-4e67-aac4-f8ed1d5ca609-bundle\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:18:59.286704 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:59.286676 2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5hhq8" Jul 07 15:18:59.286704 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:59.286692 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5hhq8" event={"ID":"50960239-91db-4e67-aac4-f8ed1d5ca609","Type":"ContainerDied","Data":"fd59291a35e8f77165ec425fdb1a8a946c71803dd4f31b994bf36aa468ed5e39"} Jul 07 15:18:59.286920 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:18:59.286738 2549 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fd59291a35e8f77165ec425fdb1a8a946c71803dd4f31b994bf36aa468ed5e39" Jul 07 15:19:03.513087 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:03.513050 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rkz2v"] Jul 07 15:19:03.513758 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:03.513432 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50960239-91db-4e67-aac4-f8ed1d5ca609" containerName="util" Jul 07 15:19:03.513758 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:03.513443 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="50960239-91db-4e67-aac4-f8ed1d5ca609" containerName="util" Jul 07 15:19:03.513758 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:03.513475 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50960239-91db-4e67-aac4-f8ed1d5ca609" containerName="pull" Jul 07 15:19:03.513758 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:03.513481 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="50960239-91db-4e67-aac4-f8ed1d5ca609" containerName="pull" Jul 07 15:19:03.513758 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:03.513487 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50960239-91db-4e67-aac4-f8ed1d5ca609" containerName="extract" Jul 07 15:19:03.513758 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:03.513493 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="50960239-91db-4e67-aac4-f8ed1d5ca609" containerName="extract" Jul 07 15:19:03.513758 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:03.513555 2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="50960239-91db-4e67-aac4-f8ed1d5ca609" containerName="extract" Jul 07 15:19:03.546304 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:03.546266 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rkz2v"] Jul 07 15:19:03.546470 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:03.546329 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rkz2v" Jul 07 15:19:03.548990 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:03.548969 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 07 15:19:03.549109 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:03.548994 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 07 15:19:03.549821 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:03.549803 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-8pcx2\"" Jul 07 15:19:03.704573 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:03.704532 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1d9ffb2d-b0a5-4619-80cc-89bbb669887d-tmp\") pod \"openshift-lws-operator-fd8ccff4c-rkz2v\" (UID: \"1d9ffb2d-b0a5-4619-80cc-89bbb669887d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rkz2v" Jul 07 15:19:03.704798 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:03.704621 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rzvx\" (UniqueName: \"kubernetes.io/projected/1d9ffb2d-b0a5-4619-80cc-89bbb669887d-kube-api-access-8rzvx\") pod \"openshift-lws-operator-fd8ccff4c-rkz2v\" (UID: \"1d9ffb2d-b0a5-4619-80cc-89bbb669887d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rkz2v" Jul 07 15:19:03.806223 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:03.806135 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1d9ffb2d-b0a5-4619-80cc-89bbb669887d-tmp\") pod \"openshift-lws-operator-fd8ccff4c-rkz2v\" (UID: \"1d9ffb2d-b0a5-4619-80cc-89bbb669887d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rkz2v" Jul 07 15:19:03.806223 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:03.806183 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rzvx\" (UniqueName: \"kubernetes.io/projected/1d9ffb2d-b0a5-4619-80cc-89bbb669887d-kube-api-access-8rzvx\") pod \"openshift-lws-operator-fd8ccff4c-rkz2v\" (UID: \"1d9ffb2d-b0a5-4619-80cc-89bbb669887d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rkz2v" Jul 07 15:19:03.806566 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:03.806543 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1d9ffb2d-b0a5-4619-80cc-89bbb669887d-tmp\") pod \"openshift-lws-operator-fd8ccff4c-rkz2v\" (UID: \"1d9ffb2d-b0a5-4619-80cc-89bbb669887d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rkz2v" Jul 07 15:19:03.814847 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:03.814820 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rzvx\" (UniqueName: \"kubernetes.io/projected/1d9ffb2d-b0a5-4619-80cc-89bbb669887d-kube-api-access-8rzvx\") pod \"openshift-lws-operator-fd8ccff4c-rkz2v\" (UID: \"1d9ffb2d-b0a5-4619-80cc-89bbb669887d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rkz2v" Jul 07 15:19:03.855914 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:03.855881 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rkz2v" Jul 07 15:19:03.997314 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:03.997280 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rkz2v"] Jul 07 15:19:03.998110 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:19:03.998085 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1d9ffb2d_b0a5_4619_80cc_89bbb669887d.slice/crio-015ed9d6a36b7e360773c68bec139450faa0c531145625fa657434051ec1df1b WatchSource:0}: Error finding container 015ed9d6a36b7e360773c68bec139450faa0c531145625fa657434051ec1df1b: Status 404 returned error can't find the container with id 015ed9d6a36b7e360773c68bec139450faa0c531145625fa657434051ec1df1b Jul 07 15:19:04.313200 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:04.313160 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rkz2v" event={"ID":"1d9ffb2d-b0a5-4619-80cc-89bbb669887d","Type":"ContainerStarted","Data":"015ed9d6a36b7e360773c68bec139450faa0c531145625fa657434051ec1df1b"} Jul 07 15:19:06.326965 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:06.326920 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rkz2v" event={"ID":"1d9ffb2d-b0a5-4619-80cc-89bbb669887d","Type":"ContainerStarted","Data":"064b430e689eb87159bbb2b77d58d6a9eeebb16a22e84c7009a8bfe2e878859a"} Jul 07 15:19:06.373682 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:06.373621 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rkz2v" podStartSLOduration=1.616620922 podStartE2EDuration="3.373605487s" totalImagesPullingTime="1.756984565s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:19:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:19:06.371249802 +0000 UTC m=+364.378727253" watchObservedRunningTime="2026-07-07 15:19:06.373605487 +0000 UTC m=+364.381082937" Jul 07 15:19:09.245884 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:09.245854 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk2tl4"] Jul 07 15:19:09.248680 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:09.248664 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk2tl4" Jul 07 15:19:09.251039 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:09.251015 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 07 15:19:09.251199 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:09.251017 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 07 15:19:09.251199 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:09.251017 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-5kvd6\"" Jul 07 15:19:09.259703 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:09.259679 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk2tl4"] Jul 07 15:19:09.359247 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:09.359216 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/878954fe-d5f0-41c1-a1c0-f6c950c90d75-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk2tl4\" (UID: \"878954fe-d5f0-41c1-a1c0-f6c950c90d75\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk2tl4" Jul 07 15:19:09.359436 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:09.359289 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqwxt\" (UniqueName: \"kubernetes.io/projected/878954fe-d5f0-41c1-a1c0-f6c950c90d75-kube-api-access-lqwxt\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk2tl4\" (UID: \"878954fe-d5f0-41c1-a1c0-f6c950c90d75\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk2tl4" Jul 07 15:19:09.359436 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:09.359334 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/878954fe-d5f0-41c1-a1c0-f6c950c90d75-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk2tl4\" (UID: \"878954fe-d5f0-41c1-a1c0-f6c950c90d75\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk2tl4" Jul 07 15:19:09.460561 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:09.460523 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/878954fe-d5f0-41c1-a1c0-f6c950c90d75-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk2tl4\" (UID: \"878954fe-d5f0-41c1-a1c0-f6c950c90d75\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk2tl4" Jul 07 15:19:09.460787 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:09.460654 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqwxt\" (UniqueName: \"kubernetes.io/projected/878954fe-d5f0-41c1-a1c0-f6c950c90d75-kube-api-access-lqwxt\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk2tl4\" (UID: \"878954fe-d5f0-41c1-a1c0-f6c950c90d75\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk2tl4" Jul 07 15:19:09.460787 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:09.460692 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/878954fe-d5f0-41c1-a1c0-f6c950c90d75-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk2tl4\" (UID: \"878954fe-d5f0-41c1-a1c0-f6c950c90d75\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk2tl4" Jul 07 15:19:09.461002 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:09.460975 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/878954fe-d5f0-41c1-a1c0-f6c950c90d75-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk2tl4\" (UID: \"878954fe-d5f0-41c1-a1c0-f6c950c90d75\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk2tl4" Jul 07 15:19:09.461083 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:09.461023 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/878954fe-d5f0-41c1-a1c0-f6c950c90d75-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk2tl4\" (UID: \"878954fe-d5f0-41c1-a1c0-f6c950c90d75\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk2tl4" Jul 07 15:19:09.473620 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:09.473577 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqwxt\" (UniqueName: \"kubernetes.io/projected/878954fe-d5f0-41c1-a1c0-f6c950c90d75-kube-api-access-lqwxt\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk2tl4\" (UID: \"878954fe-d5f0-41c1-a1c0-f6c950c90d75\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk2tl4" Jul 07 15:19:09.557909 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:09.557820 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk2tl4" Jul 07 15:19:09.693781 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:09.693752 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk2tl4"] Jul 07 15:19:09.696281 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:19:09.696251 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod878954fe_d5f0_41c1_a1c0_f6c950c90d75.slice/crio-731b62a32a1346e5647d906b42bbcd1d1c15b85e4a76cc9eeb5ba34c02d9d0d1 WatchSource:0}: Error finding container 731b62a32a1346e5647d906b42bbcd1d1c15b85e4a76cc9eeb5ba34c02d9d0d1: Status 404 returned error can't find the container with id 731b62a32a1346e5647d906b42bbcd1d1c15b85e4a76cc9eeb5ba34c02d9d0d1 Jul 07 15:19:10.348921 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:10.348887 2549 generic.go:356] "Generic (PLEG): container finished" podID="878954fe-d5f0-41c1-a1c0-f6c950c90d75" containerID="a3adce8e99ec0dba7944484d23ceee88070684b088a92f727685e4aa6e24d5a3" exitCode=0 Jul 07 15:19:10.349306 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:10.348946 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk2tl4" event={"ID":"878954fe-d5f0-41c1-a1c0-f6c950c90d75","Type":"ContainerDied","Data":"a3adce8e99ec0dba7944484d23ceee88070684b088a92f727685e4aa6e24d5a3"} Jul 07 15:19:10.349306 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:10.348985 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk2tl4" event={"ID":"878954fe-d5f0-41c1-a1c0-f6c950c90d75","Type":"ContainerStarted","Data":"731b62a32a1346e5647d906b42bbcd1d1c15b85e4a76cc9eeb5ba34c02d9d0d1"} Jul 07 15:19:11.356009 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:11.355968 2549 generic.go:356] "Generic (PLEG): container finished" podID="878954fe-d5f0-41c1-a1c0-f6c950c90d75" containerID="f00c5b1e2e886ef3ba42c0ff83fcd6fff1896ff0d8cc20d014e95ed8759e0158" exitCode=0 Jul 07 15:19:11.356401 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:11.356039 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk2tl4" event={"ID":"878954fe-d5f0-41c1-a1c0-f6c950c90d75","Type":"ContainerDied","Data":"f00c5b1e2e886ef3ba42c0ff83fcd6fff1896ff0d8cc20d014e95ed8759e0158"} Jul 07 15:19:12.363121 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:12.363077 2549 generic.go:356] "Generic (PLEG): container finished" podID="878954fe-d5f0-41c1-a1c0-f6c950c90d75" containerID="9ea15338e5c9feeb23b252bac1d6dd7097ac2073ef0e94c9cf804f3df7f98670" exitCode=0 Jul 07 15:19:12.363500 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:12.363159 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk2tl4" event={"ID":"878954fe-d5f0-41c1-a1c0-f6c950c90d75","Type":"ContainerDied","Data":"9ea15338e5c9feeb23b252bac1d6dd7097ac2073ef0e94c9cf804f3df7f98670"} Jul 07 15:19:13.503685 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:13.503655 2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk2tl4" Jul 07 15:19:13.599805 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:13.599767 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/878954fe-d5f0-41c1-a1c0-f6c950c90d75-util\") pod \"878954fe-d5f0-41c1-a1c0-f6c950c90d75\" (UID: \"878954fe-d5f0-41c1-a1c0-f6c950c90d75\") " Jul 07 15:19:13.599982 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:13.599838 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/878954fe-d5f0-41c1-a1c0-f6c950c90d75-bundle\") pod \"878954fe-d5f0-41c1-a1c0-f6c950c90d75\" (UID: \"878954fe-d5f0-41c1-a1c0-f6c950c90d75\") " Jul 07 15:19:13.599982 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:13.599905 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lqwxt\" (UniqueName: \"kubernetes.io/projected/878954fe-d5f0-41c1-a1c0-f6c950c90d75-kube-api-access-lqwxt\") pod \"878954fe-d5f0-41c1-a1c0-f6c950c90d75\" (UID: \"878954fe-d5f0-41c1-a1c0-f6c950c90d75\") " Jul 07 15:19:13.600527 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:13.600497 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/878954fe-d5f0-41c1-a1c0-f6c950c90d75-bundle" (OuterVolumeSpecName: "bundle") pod "878954fe-d5f0-41c1-a1c0-f6c950c90d75" (UID: "878954fe-d5f0-41c1-a1c0-f6c950c90d75"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:19:13.601992 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:13.601962 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/878954fe-d5f0-41c1-a1c0-f6c950c90d75-kube-api-access-lqwxt" (OuterVolumeSpecName: "kube-api-access-lqwxt") pod "878954fe-d5f0-41c1-a1c0-f6c950c90d75" (UID: "878954fe-d5f0-41c1-a1c0-f6c950c90d75"). InnerVolumeSpecName "kube-api-access-lqwxt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:19:13.607733 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:13.607685 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/878954fe-d5f0-41c1-a1c0-f6c950c90d75-util" (OuterVolumeSpecName: "util") pod "878954fe-d5f0-41c1-a1c0-f6c950c90d75" (UID: "878954fe-d5f0-41c1-a1c0-f6c950c90d75"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:19:13.700857 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:13.700761 2549 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/878954fe-d5f0-41c1-a1c0-f6c950c90d75-bundle\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:19:13.700857 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:13.700796 2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lqwxt\" (UniqueName: \"kubernetes.io/projected/878954fe-d5f0-41c1-a1c0-f6c950c90d75-kube-api-access-lqwxt\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:19:13.700857 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:13.700807 2549 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/878954fe-d5f0-41c1-a1c0-f6c950c90d75-util\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:19:14.374957 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:14.374923 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk2tl4" event={"ID":"878954fe-d5f0-41c1-a1c0-f6c950c90d75","Type":"ContainerDied","Data":"731b62a32a1346e5647d906b42bbcd1d1c15b85e4a76cc9eeb5ba34c02d9d0d1"} Jul 07 15:19:14.375147 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:14.374969 2549 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="731b62a32a1346e5647d906b42bbcd1d1c15b85e4a76cc9eeb5ba34c02d9d0d1" Jul 07 15:19:14.375147 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:14.374971 2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afk2tl4" Jul 07 15:19:37.236532 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:37.236497 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-dcbf69668-wldx7"] Jul 07 15:19:37.237074 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:37.236856 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="878954fe-d5f0-41c1-a1c0-f6c950c90d75" containerName="pull" Jul 07 15:19:37.237074 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:37.236867 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="878954fe-d5f0-41c1-a1c0-f6c950c90d75" containerName="pull" Jul 07 15:19:37.237074 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:37.236876 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="878954fe-d5f0-41c1-a1c0-f6c950c90d75" containerName="extract" Jul 07 15:19:37.237074 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:37.236881 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="878954fe-d5f0-41c1-a1c0-f6c950c90d75" containerName="extract" Jul 07 15:19:37.237074 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:37.236903 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="878954fe-d5f0-41c1-a1c0-f6c950c90d75" containerName="util" Jul 07 15:19:37.237074 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:37.236909 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="878954fe-d5f0-41c1-a1c0-f6c950c90d75" containerName="util" Jul 07 15:19:37.237074 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:37.236964 2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="878954fe-d5f0-41c1-a1c0-f6c950c90d75" containerName="extract" Jul 07 15:19:37.239321 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:37.239306 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-dcbf69668-wldx7" Jul 07 15:19:37.242895 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:37.242874 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 07 15:19:37.243033 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:37.242898 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 07 15:19:37.243033 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:37.242918 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-rcsvm\"" Jul 07 15:19:37.243180 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:37.243048 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 07 15:19:37.243740 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:37.243724 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 07 15:19:37.264970 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:37.264944 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-dcbf69668-wldx7"] Jul 07 15:19:37.305467 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:37.305431 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8779b525-5eb5-43f7-8168-ab732a67c10a-webhook-cert\") pod \"opendatahub-operator-controller-manager-dcbf69668-wldx7\" (UID: \"8779b525-5eb5-43f7-8168-ab732a67c10a\") " pod="opendatahub/opendatahub-operator-controller-manager-dcbf69668-wldx7" Jul 07 15:19:37.305467 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:37.305472 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clwsh\" (UniqueName: \"kubernetes.io/projected/8779b525-5eb5-43f7-8168-ab732a67c10a-kube-api-access-clwsh\") pod \"opendatahub-operator-controller-manager-dcbf69668-wldx7\" (UID: \"8779b525-5eb5-43f7-8168-ab732a67c10a\") " pod="opendatahub/opendatahub-operator-controller-manager-dcbf69668-wldx7" Jul 07 15:19:37.305693 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:37.305551 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8779b525-5eb5-43f7-8168-ab732a67c10a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-dcbf69668-wldx7\" (UID: \"8779b525-5eb5-43f7-8168-ab732a67c10a\") " pod="opendatahub/opendatahub-operator-controller-manager-dcbf69668-wldx7" Jul 07 15:19:37.305693 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:37.305666 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8779b525-5eb5-43f7-8168-ab732a67c10a-tmp\") pod \"opendatahub-operator-controller-manager-dcbf69668-wldx7\" (UID: \"8779b525-5eb5-43f7-8168-ab732a67c10a\") " pod="opendatahub/opendatahub-operator-controller-manager-dcbf69668-wldx7" Jul 07 15:19:37.305805 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:37.305701 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8779b525-5eb5-43f7-8168-ab732a67c10a-manifests\") pod \"opendatahub-operator-controller-manager-dcbf69668-wldx7\" (UID: \"8779b525-5eb5-43f7-8168-ab732a67c10a\") " pod="opendatahub/opendatahub-operator-controller-manager-dcbf69668-wldx7" Jul 07 15:19:37.406889 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:37.406851 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8779b525-5eb5-43f7-8168-ab732a67c10a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-dcbf69668-wldx7\" (UID: \"8779b525-5eb5-43f7-8168-ab732a67c10a\") " pod="opendatahub/opendatahub-operator-controller-manager-dcbf69668-wldx7" Jul 07 15:19:37.407089 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:37.406900 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8779b525-5eb5-43f7-8168-ab732a67c10a-tmp\") pod \"opendatahub-operator-controller-manager-dcbf69668-wldx7\" (UID: \"8779b525-5eb5-43f7-8168-ab732a67c10a\") " pod="opendatahub/opendatahub-operator-controller-manager-dcbf69668-wldx7" Jul 07 15:19:37.407089 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:37.406927 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8779b525-5eb5-43f7-8168-ab732a67c10a-manifests\") pod \"opendatahub-operator-controller-manager-dcbf69668-wldx7\" (UID: \"8779b525-5eb5-43f7-8168-ab732a67c10a\") " pod="opendatahub/opendatahub-operator-controller-manager-dcbf69668-wldx7" Jul 07 15:19:37.407089 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:37.406964 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8779b525-5eb5-43f7-8168-ab732a67c10a-webhook-cert\") pod \"opendatahub-operator-controller-manager-dcbf69668-wldx7\" (UID: \"8779b525-5eb5-43f7-8168-ab732a67c10a\") " pod="opendatahub/opendatahub-operator-controller-manager-dcbf69668-wldx7" Jul 07 15:19:37.407089 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:37.406986 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clwsh\" (UniqueName: \"kubernetes.io/projected/8779b525-5eb5-43f7-8168-ab732a67c10a-kube-api-access-clwsh\") pod \"opendatahub-operator-controller-manager-dcbf69668-wldx7\" (UID: \"8779b525-5eb5-43f7-8168-ab732a67c10a\") " pod="opendatahub/opendatahub-operator-controller-manager-dcbf69668-wldx7" Jul 07 15:19:37.407424 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:37.407400 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8779b525-5eb5-43f7-8168-ab732a67c10a-manifests\") pod \"opendatahub-operator-controller-manager-dcbf69668-wldx7\" (UID: \"8779b525-5eb5-43f7-8168-ab732a67c10a\") " pod="opendatahub/opendatahub-operator-controller-manager-dcbf69668-wldx7" Jul 07 15:19:37.409142 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:37.409119 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8779b525-5eb5-43f7-8168-ab732a67c10a-tmp\") pod \"opendatahub-operator-controller-manager-dcbf69668-wldx7\" (UID: \"8779b525-5eb5-43f7-8168-ab732a67c10a\") " pod="opendatahub/opendatahub-operator-controller-manager-dcbf69668-wldx7" Jul 07 15:19:37.409484 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:37.409460 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8779b525-5eb5-43f7-8168-ab732a67c10a-webhook-cert\") pod \"opendatahub-operator-controller-manager-dcbf69668-wldx7\" (UID: \"8779b525-5eb5-43f7-8168-ab732a67c10a\") " pod="opendatahub/opendatahub-operator-controller-manager-dcbf69668-wldx7" Jul 07 15:19:37.409526 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:37.409478 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8779b525-5eb5-43f7-8168-ab732a67c10a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-dcbf69668-wldx7\" (UID: \"8779b525-5eb5-43f7-8168-ab732a67c10a\") " pod="opendatahub/opendatahub-operator-controller-manager-dcbf69668-wldx7" Jul 07 15:19:37.421637 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:37.421612 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-clwsh\" (UniqueName: \"kubernetes.io/projected/8779b525-5eb5-43f7-8168-ab732a67c10a-kube-api-access-clwsh\") pod \"opendatahub-operator-controller-manager-dcbf69668-wldx7\" (UID: \"8779b525-5eb5-43f7-8168-ab732a67c10a\") " pod="opendatahub/opendatahub-operator-controller-manager-dcbf69668-wldx7" Jul 07 15:19:37.549907 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:37.549807 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-dcbf69668-wldx7" Jul 07 15:19:37.690503 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:37.690476 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-dcbf69668-wldx7"] Jul 07 15:19:37.693427 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:19:37.693403 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8779b525_5eb5_43f7_8168_ab732a67c10a.slice/crio-e95841fb4f752d09ef5d1ba051528ebf9af316e609cec8efad35584a4f78c6ef WatchSource:0}: Error finding container e95841fb4f752d09ef5d1ba051528ebf9af316e609cec8efad35584a4f78c6ef: Status 404 returned error can't find the container with id e95841fb4f752d09ef5d1ba051528ebf9af316e609cec8efad35584a4f78c6ef Jul 07 15:19:38.494848 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:38.494806 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dcbf69668-wldx7" event={"ID":"8779b525-5eb5-43f7-8168-ab732a67c10a","Type":"ContainerStarted","Data":"e95841fb4f752d09ef5d1ba051528ebf9af316e609cec8efad35584a4f78c6ef"} Jul 07 15:19:41.513394 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:41.513358 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dcbf69668-wldx7" event={"ID":"8779b525-5eb5-43f7-8168-ab732a67c10a","Type":"ContainerStarted","Data":"afeb1205c2e793e554346030359395333c722d638ee86713ef2c4315b64db059"} Jul 07 15:19:42.519197 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:42.519110 2549 generic.go:356] "Generic (PLEG): container finished" podID="8779b525-5eb5-43f7-8168-ab732a67c10a" containerID="afeb1205c2e793e554346030359395333c722d638ee86713ef2c4315b64db059" exitCode=0 Jul 07 15:19:42.519593 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:42.519198 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dcbf69668-wldx7" event={"ID":"8779b525-5eb5-43f7-8168-ab732a67c10a","Type":"ContainerDied","Data":"afeb1205c2e793e554346030359395333c722d638ee86713ef2c4315b64db059"} Jul 07 15:19:43.527510 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:43.527474 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dcbf69668-wldx7" event={"ID":"8779b525-5eb5-43f7-8168-ab732a67c10a","Type":"ContainerStarted","Data":"c8a7c815193d6c91792352b32e27d793dcf53506fae926038469d16d754e18fc"} Jul 07 15:19:43.528015 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:43.527572 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-dcbf69668-wldx7" Jul 07 15:19:43.595105 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:43.595029 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-dcbf69668-wldx7" podStartSLOduration=1.904435782 podStartE2EDuration="6.595004479s" totalImagesPullingTime="3.679000159s" totalInitContainerRuntime="1.011568538s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:19:37 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:19:43.593456619 +0000 UTC m=+401.600934067" watchObservedRunningTime="2026-07-07 15:19:43.595004479 +0000 UTC m=+401.602481935" Jul 07 15:19:54.535915 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:54.535886 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-dcbf69668-wldx7" Jul 07 15:19:58.729233 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:58.729197 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5c6c759bc-5c7v6"] Jul 07 15:19:58.732907 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:58.732886 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5c6c759bc-5c7v6" Jul 07 15:19:58.735359 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:19:58.735329 2549 status_manager.go:1018] "Failed to get status for pod" err="pods \"kube-auth-proxy-5c6c759bc-5c7v6\" is forbidden: User \"system:node:ip-10-0-138-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-138-11.ec2.internal' and this object" podUID="01d82b99-af4a-4369-9164-360e15d583ac" pod="openshift-ingress/kube-auth-proxy-5c6c759bc-5c7v6" Jul 07 15:19:58.735461 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:19:58.735368 2549 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"kube-auth-proxy-tls\" is forbidden: User \"system:node:ip-10-0-138-11.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-138-11.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" type="*v1.Secret" Jul 07 15:19:58.735658 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:19:58.735634 2549 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"kube-auth-proxy-creds\" is forbidden: User \"system:node:ip-10-0-138-11.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-138-11.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" type="*v1.Secret" Jul 07 15:19:58.739019 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:58.739001 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-mzmfk\"" Jul 07 15:19:58.761806 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:58.761782 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5c6c759bc-5c7v6"] Jul 07 15:19:58.909120 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:58.909085 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/01d82b99-af4a-4369-9164-360e15d583ac-tmp\") pod \"kube-auth-proxy-5c6c759bc-5c7v6\" (UID: \"01d82b99-af4a-4369-9164-360e15d583ac\") " pod="openshift-ingress/kube-auth-proxy-5c6c759bc-5c7v6" Jul 07 15:19:58.909305 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:58.909197 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01d82b99-af4a-4369-9164-360e15d583ac-tls-certs\") pod \"kube-auth-proxy-5c6c759bc-5c7v6\" (UID: \"01d82b99-af4a-4369-9164-360e15d583ac\") " pod="openshift-ingress/kube-auth-proxy-5c6c759bc-5c7v6" Jul 07 15:19:58.909305 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:58.909291 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p56zn\" (UniqueName: \"kubernetes.io/projected/01d82b99-af4a-4369-9164-360e15d583ac-kube-api-access-p56zn\") pod \"kube-auth-proxy-5c6c759bc-5c7v6\" (UID: \"01d82b99-af4a-4369-9164-360e15d583ac\") " pod="openshift-ingress/kube-auth-proxy-5c6c759bc-5c7v6" Jul 07 15:19:58.963204 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:58.963175 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-7fshj"] Jul 07 15:19:58.966537 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:58.966519 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-7fshj" Jul 07 15:19:58.969390 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:58.969373 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-b4j6h\"" Jul 07 15:19:58.969794 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:58.969780 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 07 15:19:58.999335 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:58.999259 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-7fshj"] Jul 07 15:19:59.010341 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:59.010304 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01d82b99-af4a-4369-9164-360e15d583ac-tls-certs\") pod \"kube-auth-proxy-5c6c759bc-5c7v6\" (UID: \"01d82b99-af4a-4369-9164-360e15d583ac\") " pod="openshift-ingress/kube-auth-proxy-5c6c759bc-5c7v6" Jul 07 15:19:59.010518 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:59.010381 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p56zn\" (UniqueName: \"kubernetes.io/projected/01d82b99-af4a-4369-9164-360e15d583ac-kube-api-access-p56zn\") pod \"kube-auth-proxy-5c6c759bc-5c7v6\" (UID: \"01d82b99-af4a-4369-9164-360e15d583ac\") " pod="openshift-ingress/kube-auth-proxy-5c6c759bc-5c7v6" Jul 07 15:19:59.010518 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:59.010427 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/01d82b99-af4a-4369-9164-360e15d583ac-tmp\") pod \"kube-auth-proxy-5c6c759bc-5c7v6\" (UID: \"01d82b99-af4a-4369-9164-360e15d583ac\") " pod="openshift-ingress/kube-auth-proxy-5c6c759bc-5c7v6" Jul 07 15:19:59.012827 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:59.012796 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/01d82b99-af4a-4369-9164-360e15d583ac-tmp\") pod \"kube-auth-proxy-5c6c759bc-5c7v6\" (UID: \"01d82b99-af4a-4369-9164-360e15d583ac\") " pod="openshift-ingress/kube-auth-proxy-5c6c759bc-5c7v6" Jul 07 15:19:59.034000 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:59.033968 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p56zn\" (UniqueName: \"kubernetes.io/projected/01d82b99-af4a-4369-9164-360e15d583ac-kube-api-access-p56zn\") pod \"kube-auth-proxy-5c6c759bc-5c7v6\" (UID: \"01d82b99-af4a-4369-9164-360e15d583ac\") " pod="openshift-ingress/kube-auth-proxy-5c6c759bc-5c7v6" Jul 07 15:19:59.111524 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:59.111490 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e74b6418-6103-41d5-9828-988395d57e82-cert\") pod \"odh-model-controller-7998c67dc4-7fshj\" (UID: \"e74b6418-6103-41d5-9828-988395d57e82\") " pod="opendatahub/odh-model-controller-7998c67dc4-7fshj" Jul 07 15:19:59.111726 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:59.111608 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l89jx\" (UniqueName: \"kubernetes.io/projected/e74b6418-6103-41d5-9828-988395d57e82-kube-api-access-l89jx\") pod \"odh-model-controller-7998c67dc4-7fshj\" (UID: \"e74b6418-6103-41d5-9828-988395d57e82\") " pod="opendatahub/odh-model-controller-7998c67dc4-7fshj" Jul 07 15:19:59.212794 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:59.212758 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e74b6418-6103-41d5-9828-988395d57e82-cert\") pod \"odh-model-controller-7998c67dc4-7fshj\" (UID: \"e74b6418-6103-41d5-9828-988395d57e82\") " pod="opendatahub/odh-model-controller-7998c67dc4-7fshj" Jul 07 15:19:59.212988 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:59.212869 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l89jx\" (UniqueName: \"kubernetes.io/projected/e74b6418-6103-41d5-9828-988395d57e82-kube-api-access-l89jx\") pod \"odh-model-controller-7998c67dc4-7fshj\" (UID: \"e74b6418-6103-41d5-9828-988395d57e82\") " pod="opendatahub/odh-model-controller-7998c67dc4-7fshj" Jul 07 15:19:59.212988 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:19:59.212933 2549 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 07 15:19:59.213111 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:19:59.213010 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e74b6418-6103-41d5-9828-988395d57e82-cert podName:e74b6418-6103-41d5-9828-988395d57e82 nodeName:}" failed. No retries permitted until 2026-07-07 15:19:59.712989766 +0000 UTC m=+417.720467196 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/e74b6418-6103-41d5-9828-988395d57e82-cert") pod "odh-model-controller-7998c67dc4-7fshj" (UID: "e74b6418-6103-41d5-9828-988395d57e82") : secret "odh-model-controller-webhook-cert" not found Jul 07 15:19:59.229639 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:59.229614 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l89jx\" (UniqueName: \"kubernetes.io/projected/e74b6418-6103-41d5-9828-988395d57e82-kube-api-access-l89jx\") pod \"odh-model-controller-7998c67dc4-7fshj\" (UID: \"e74b6418-6103-41d5-9828-988395d57e82\") " pod="opendatahub/odh-model-controller-7998c67dc4-7fshj" Jul 07 15:19:59.565608 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:59.565567 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 07 15:19:59.573011 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:59.572978 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01d82b99-af4a-4369-9164-360e15d583ac-tls-certs\") pod \"kube-auth-proxy-5c6c759bc-5c7v6\" (UID: \"01d82b99-af4a-4369-9164-360e15d583ac\") " pod="openshift-ingress/kube-auth-proxy-5c6c759bc-5c7v6" Jul 07 15:19:59.643058 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:59.643025 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5c6c759bc-5c7v6" Jul 07 15:19:59.719269 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:59.718175 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e74b6418-6103-41d5-9828-988395d57e82-cert\") pod \"odh-model-controller-7998c67dc4-7fshj\" (UID: \"e74b6418-6103-41d5-9828-988395d57e82\") " pod="opendatahub/odh-model-controller-7998c67dc4-7fshj" Jul 07 15:19:59.721957 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:59.721858 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e74b6418-6103-41d5-9828-988395d57e82-cert\") pod \"odh-model-controller-7998c67dc4-7fshj\" (UID: \"e74b6418-6103-41d5-9828-988395d57e82\") " pod="opendatahub/odh-model-controller-7998c67dc4-7fshj" Jul 07 15:19:59.807151 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:59.807116 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5c6c759bc-5c7v6"] Jul 07 15:19:59.809374 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:19:59.809347 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01d82b99_af4a_4369_9164_360e15d583ac.slice/crio-49723d8146fcf6f5ef892470c4454f19c9ff3d5f3c52dd5bf3ef09b289afa819 WatchSource:0}: Error finding container 49723d8146fcf6f5ef892470c4454f19c9ff3d5f3c52dd5bf3ef09b289afa819: Status 404 returned error can't find the container with id 49723d8146fcf6f5ef892470c4454f19c9ff3d5f3c52dd5bf3ef09b289afa819 Jul 07 15:19:59.878486 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:59.878386 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-7fshj" Jul 07 15:19:59.957362 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:19:59.957337 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 07 15:20:00.040537 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:00.040511 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-7fshj"] Jul 07 15:20:00.042696 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:20:00.042669 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode74b6418_6103_41d5_9828_988395d57e82.slice/crio-b9a0b58a1535cfe88ad686d3a4c569077754aaa0d1c0735cccf68000e9a40ae6 WatchSource:0}: Error finding container b9a0b58a1535cfe88ad686d3a4c569077754aaa0d1c0735cccf68000e9a40ae6: Status 404 returned error can't find the container with id b9a0b58a1535cfe88ad686d3a4c569077754aaa0d1c0735cccf68000e9a40ae6 Jul 07 15:20:00.090961 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:00.090918 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-dc8f7"] Jul 07 15:20:00.095879 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:00.095854 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-dc8f7" Jul 07 15:20:00.099450 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:00.099426 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 07 15:20:00.099567 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:00.099428 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-bvvj7\"" Jul 07 15:20:00.107501 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:00.107466 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-dc8f7"] Jul 07 15:20:00.198352 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:00.198267 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-xgf6x"] Jul 07 15:20:00.201742 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:00.201721 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-xgf6x" Jul 07 15:20:00.204597 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:00.204553 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 07 15:20:00.204738 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:00.204591 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 07 15:20:00.204738 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:00.204596 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 07 15:20:00.204894 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:00.204879 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 07 15:20:00.204894 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:00.204888 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-z58x7\"" Jul 07 15:20:00.214681 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:00.214657 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-xgf6x"] Jul 07 15:20:00.222847 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:00.222823 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvfpx\" (UniqueName: \"kubernetes.io/projected/9b60c5f2-1af3-4b36-adcb-f1ade7775ca5-kube-api-access-zvfpx\") pod \"model-serving-api-54d89bd5fc-dc8f7\" (UID: \"9b60c5f2-1af3-4b36-adcb-f1ade7775ca5\") " pod="opendatahub/model-serving-api-54d89bd5fc-dc8f7" Jul 07 15:20:00.222982 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:00.222929 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b60c5f2-1af3-4b36-adcb-f1ade7775ca5-tls-certs\") pod \"model-serving-api-54d89bd5fc-dc8f7\" (UID: \"9b60c5f2-1af3-4b36-adcb-f1ade7775ca5\") " pod="opendatahub/model-serving-api-54d89bd5fc-dc8f7" Jul 07 15:20:00.323774 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:00.323735 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-xgf6x\" (UID: \"6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xgf6x" Jul 07 15:20:00.323953 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:00.323850 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d-local-certs\") pod \"istiod-openshift-gateway-7574459656-xgf6x\" (UID: \"6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xgf6x" Jul 07 15:20:00.323953 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:00.323904 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvfpx\" (UniqueName: \"kubernetes.io/projected/9b60c5f2-1af3-4b36-adcb-f1ade7775ca5-kube-api-access-zvfpx\") pod \"model-serving-api-54d89bd5fc-dc8f7\" (UID: \"9b60c5f2-1af3-4b36-adcb-f1ade7775ca5\") " pod="opendatahub/model-serving-api-54d89bd5fc-dc8f7" Jul 07 15:20:00.323953 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:00.323937 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d-istio-token\") pod \"istiod-openshift-gateway-7574459656-xgf6x\" (UID: \"6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xgf6x" Jul 07 15:20:00.324166 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:00.323960 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f58mq\" (UniqueName: \"kubernetes.io/projected/6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d-kube-api-access-f58mq\") pod \"istiod-openshift-gateway-7574459656-xgf6x\" (UID: \"6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xgf6x" Jul 07 15:20:00.324166 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:00.323998 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-xgf6x\" (UID: \"6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xgf6x" Jul 07 15:20:00.324166 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:00.324074 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-xgf6x\" (UID: \"6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xgf6x" Jul 07 15:20:00.324166 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:00.324104 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d-cacerts\") pod \"istiod-openshift-gateway-7574459656-xgf6x\" (UID: \"6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xgf6x" Jul 07 15:20:00.324166 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:00.324138 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b60c5f2-1af3-4b36-adcb-f1ade7775ca5-tls-certs\") pod \"model-serving-api-54d89bd5fc-dc8f7\" (UID: \"9b60c5f2-1af3-4b36-adcb-f1ade7775ca5\") " pod="opendatahub/model-serving-api-54d89bd5fc-dc8f7" Jul 07 15:20:00.326873 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:00.326848 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b60c5f2-1af3-4b36-adcb-f1ade7775ca5-tls-certs\") pod \"model-serving-api-54d89bd5fc-dc8f7\" (UID: \"9b60c5f2-1af3-4b36-adcb-f1ade7775ca5\") " pod="opendatahub/model-serving-api-54d89bd5fc-dc8f7" Jul 07 15:20:00.332575 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:00.332549 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvfpx\" (UniqueName: \"kubernetes.io/projected/9b60c5f2-1af3-4b36-adcb-f1ade7775ca5-kube-api-access-zvfpx\") pod \"model-serving-api-54d89bd5fc-dc8f7\" (UID: \"9b60c5f2-1af3-4b36-adcb-f1ade7775ca5\") " pod="opendatahub/model-serving-api-54d89bd5fc-dc8f7" Jul 07 15:20:00.408638 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:00.408603 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-dc8f7" Jul 07 15:20:00.424948 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:00.424917 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-xgf6x\" (UID: \"6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xgf6x" Jul 07 15:20:00.425094 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:00.424964 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d-cacerts\") pod \"istiod-openshift-gateway-7574459656-xgf6x\" (UID: \"6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xgf6x" Jul 07 15:20:00.425167 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:00.425093 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-xgf6x\" (UID: \"6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xgf6x" Jul 07 15:20:00.425227 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:00.425207 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d-local-certs\") pod \"istiod-openshift-gateway-7574459656-xgf6x\" (UID: \"6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xgf6x" Jul 07 15:20:00.425296 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:00.425273 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d-istio-token\") pod \"istiod-openshift-gateway-7574459656-xgf6x\" (UID: \"6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xgf6x" Jul 07 15:20:00.425354 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:00.425313 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f58mq\" (UniqueName: \"kubernetes.io/projected/6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d-kube-api-access-f58mq\") pod \"istiod-openshift-gateway-7574459656-xgf6x\" (UID: \"6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xgf6x" Jul 07 15:20:00.425354 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:00.425344 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-xgf6x\" (UID: \"6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xgf6x" Jul 07 15:20:00.425630 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:00.425605 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-xgf6x\" (UID: \"6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xgf6x" Jul 07 15:20:00.428057 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:00.428018 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d-local-certs\") pod \"istiod-openshift-gateway-7574459656-xgf6x\" (UID: \"6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xgf6x" Jul 07 15:20:00.428801 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:00.428776 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d-cacerts\") pod \"istiod-openshift-gateway-7574459656-xgf6x\" (UID: \"6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xgf6x" Jul 07 15:20:00.429002 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:00.428980 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-xgf6x\" (UID: \"6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xgf6x" Jul 07 15:20:00.429068 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:00.429037 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-xgf6x\" (UID: \"6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xgf6x" Jul 07 15:20:00.434862 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:00.434842 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d-istio-token\") pod \"istiod-openshift-gateway-7574459656-xgf6x\" (UID: \"6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xgf6x" Jul 07 15:20:00.435466 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:00.435447 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f58mq\" (UniqueName: \"kubernetes.io/projected/6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d-kube-api-access-f58mq\") pod \"istiod-openshift-gateway-7574459656-xgf6x\" (UID: \"6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xgf6x" Jul 07 15:20:00.517659 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:00.511622 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-xgf6x" Jul 07 15:20:00.586928 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:00.586894 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-dc8f7"] Jul 07 15:20:00.594727 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:20:00.594696 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b60c5f2_1af3_4b36_adcb_f1ade7775ca5.slice/crio-8c25d21d69a63f5525e50663f9d727d348f16be3360816dfce4eb1ca0cdd0dc8 WatchSource:0}: Error finding container 8c25d21d69a63f5525e50663f9d727d348f16be3360816dfce4eb1ca0cdd0dc8: Status 404 returned error can't find the container with id 8c25d21d69a63f5525e50663f9d727d348f16be3360816dfce4eb1ca0cdd0dc8 Jul 07 15:20:00.618856 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:00.618815 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-7fshj" event={"ID":"e74b6418-6103-41d5-9828-988395d57e82","Type":"ContainerStarted","Data":"b9a0b58a1535cfe88ad686d3a4c569077754aaa0d1c0735cccf68000e9a40ae6"} Jul 07 15:20:00.620533 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:00.620499 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-dc8f7" event={"ID":"9b60c5f2-1af3-4b36-adcb-f1ade7775ca5","Type":"ContainerStarted","Data":"8c25d21d69a63f5525e50663f9d727d348f16be3360816dfce4eb1ca0cdd0dc8"} Jul 07 15:20:00.622222 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:00.622197 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5c6c759bc-5c7v6" event={"ID":"01d82b99-af4a-4369-9164-360e15d583ac","Type":"ContainerStarted","Data":"49723d8146fcf6f5ef892470c4454f19c9ff3d5f3c52dd5bf3ef09b289afa819"} Jul 07 15:20:00.691615 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:00.691561 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-xgf6x"] Jul 07 15:20:00.695707 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:20:00.695676 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e235b41_5ffb_4f5a_b1c2_87daf25a3e4d.slice/crio-b0fff63baa68c041eeb3a20125fed111f7fd4f49563ef78595f4b79e09e67e73 WatchSource:0}: Error finding container b0fff63baa68c041eeb3a20125fed111f7fd4f49563ef78595f4b79e09e67e73: Status 404 returned error can't find the container with id b0fff63baa68c041eeb3a20125fed111f7fd4f49563ef78595f4b79e09e67e73 Jul 07 15:20:01.631456 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:01.631409 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-xgf6x" event={"ID":"6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d","Type":"ContainerStarted","Data":"b0fff63baa68c041eeb3a20125fed111f7fd4f49563ef78595f4b79e09e67e73"} Jul 07 15:20:06.049703 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:06.049680 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 07 15:20:06.099300 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:06.099261 2549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 15:20:06.099410 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:06.099346 2549 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 15:20:06.680002 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:06.679961 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5c6c759bc-5c7v6" event={"ID":"01d82b99-af4a-4369-9164-360e15d583ac","Type":"ContainerStarted","Data":"a800d685aa941b52758ab17ebfcfe4250a8cd5b29d6c644d3455ec8e1ca312a4"} Jul 07 15:20:06.681793 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:06.681758 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-7fshj" event={"ID":"e74b6418-6103-41d5-9828-988395d57e82","Type":"ContainerStarted","Data":"244d80e8b935b482b62308e6ca69a5633ff0406434a7d5225b441da43e322de6"} Jul 07 15:20:06.681929 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:06.681885 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-7fshj" Jul 07 15:20:06.683944 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:06.683917 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-xgf6x" event={"ID":"6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d","Type":"ContainerStarted","Data":"6ac6ce8599420ffc52f312cc14fa42e399875b0085f1ff48df6fb199ac3ac2ff"} Jul 07 15:20:06.684086 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:06.684049 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-xgf6x" Jul 07 15:20:06.685686 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:06.685660 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-dc8f7" event={"ID":"9b60c5f2-1af3-4b36-adcb-f1ade7775ca5","Type":"ContainerStarted","Data":"dc7954c171fe5974a74c32fdb36af74cfc3f1686c9b49acd606529ad4045e38d"} Jul 07 15:20:06.685835 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:06.685816 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-dc8f7" Jul 07 15:20:06.719977 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:06.719929 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5c6c759bc-5c7v6" podStartSLOduration=2.484004381 podStartE2EDuration="8.719914421s" totalImagesPullingTime="6.23591004s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:19:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:20:06.71557051 +0000 UTC m=+424.723047960" watchObservedRunningTime="2026-07-07 15:20:06.719914421 +0000 UTC m=+424.727391871" Jul 07 15:20:06.765773 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:06.765720 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-7fshj" podStartSLOduration=2.769893188 podStartE2EDuration="8.765704191s" totalImagesPullingTime="5.995811003s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:19:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:20:06.764426068 +0000 UTC m=+424.771903544" watchObservedRunningTime="2026-07-07 15:20:06.765704191 +0000 UTC m=+424.773181668" Jul 07 15:20:06.813375 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:06.813288 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-xgf6x" podStartSLOduration=1.41219699 podStartE2EDuration="6.813265909s" totalImagesPullingTime="5.401068919s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:20:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:20:06.809037024 +0000 UTC m=+424.816514474" watchObservedRunningTime="2026-07-07 15:20:06.813265909 +0000 UTC m=+424.820743364" Jul 07 15:20:06.841905 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:06.841855 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-dc8f7" podStartSLOduration=1.3980327350000001 podStartE2EDuration="6.841838254s" totalImagesPullingTime="5.443805519s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:20:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:20:06.839920548 +0000 UTC m=+424.847397999" watchObservedRunningTime="2026-07-07 15:20:06.841838254 +0000 UTC m=+424.849315705" Jul 07 15:20:07.698625 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:07.698574 2549 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-xgf6x container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 07 15:20:07.699000 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:07.698653 2549 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-xgf6x" podUID="6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 15:20:10.698046 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:10.698020 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-xgf6x" Jul 07 15:20:12.443169 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:12.443138 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-t97r8"] Jul 07 15:20:12.448296 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:12.448277 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-t97r8" Jul 07 15:20:12.455925 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:12.455903 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-n2bcn\"" Jul 07 15:20:12.456284 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:12.456265 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 07 15:20:12.460121 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:12.460101 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-9f7tp"] Jul 07 15:20:12.463797 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:12.463780 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9f7tp" Jul 07 15:20:12.465895 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:12.465867 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 07 15:20:12.466134 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:12.466117 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-sjnr7\"" Jul 07 15:20:12.478037 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:12.478011 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-t97r8"] Jul 07 15:20:12.480017 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:12.479997 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-9f7tp"] Jul 07 15:20:12.542405 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:12.542373 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/27ffe7dc-0f8f-4d32-882b-6b0829ab71ad-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-9f7tp\" (UID: \"27ffe7dc-0f8f-4d32-882b-6b0829ab71ad\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9f7tp" Jul 07 15:20:12.542568 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:12.542419 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgwhb\" (UniqueName: \"kubernetes.io/projected/27ffe7dc-0f8f-4d32-882b-6b0829ab71ad-kube-api-access-kgwhb\") pod \"llmisvc-controller-manager-777c6bd7d5-9f7tp\" (UID: \"27ffe7dc-0f8f-4d32-882b-6b0829ab71ad\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9f7tp" Jul 07 15:20:12.542568 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:12.542460 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dvjn\" (UniqueName: \"kubernetes.io/projected/ff832223-8f66-4299-94e1-35b79a4ccc22-kube-api-access-7dvjn\") pod \"kserve-controller-manager-d6f4b65b8-t97r8\" (UID: \"ff832223-8f66-4299-94e1-35b79a4ccc22\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-t97r8" Jul 07 15:20:12.542568 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:12.542486 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ff832223-8f66-4299-94e1-35b79a4ccc22-cert\") pod \"kserve-controller-manager-d6f4b65b8-t97r8\" (UID: \"ff832223-8f66-4299-94e1-35b79a4ccc22\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-t97r8" Jul 07 15:20:12.643599 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:12.643560 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgwhb\" (UniqueName: \"kubernetes.io/projected/27ffe7dc-0f8f-4d32-882b-6b0829ab71ad-kube-api-access-kgwhb\") pod \"llmisvc-controller-manager-777c6bd7d5-9f7tp\" (UID: \"27ffe7dc-0f8f-4d32-882b-6b0829ab71ad\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9f7tp" Jul 07 15:20:12.643773 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:12.643650 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dvjn\" (UniqueName: \"kubernetes.io/projected/ff832223-8f66-4299-94e1-35b79a4ccc22-kube-api-access-7dvjn\") pod \"kserve-controller-manager-d6f4b65b8-t97r8\" (UID: \"ff832223-8f66-4299-94e1-35b79a4ccc22\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-t97r8" Jul 07 15:20:12.643773 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:12.643673 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ff832223-8f66-4299-94e1-35b79a4ccc22-cert\") pod \"kserve-controller-manager-d6f4b65b8-t97r8\" (UID: \"ff832223-8f66-4299-94e1-35b79a4ccc22\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-t97r8" Jul 07 15:20:12.643773 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:12.643753 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/27ffe7dc-0f8f-4d32-882b-6b0829ab71ad-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-9f7tp\" (UID: \"27ffe7dc-0f8f-4d32-882b-6b0829ab71ad\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9f7tp" Jul 07 15:20:12.643902 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:20:12.643870 2549 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 07 15:20:12.643948 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:20:12.643932 2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff832223-8f66-4299-94e1-35b79a4ccc22-cert podName:ff832223-8f66-4299-94e1-35b79a4ccc22 nodeName:}" failed. No retries permitted until 2026-07-07 15:20:13.143914274 +0000 UTC m=+431.151391714 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ff832223-8f66-4299-94e1-35b79a4ccc22-cert") pod "kserve-controller-manager-d6f4b65b8-t97r8" (UID: "ff832223-8f66-4299-94e1-35b79a4ccc22") : secret "kserve-webhook-server-cert" not found Jul 07 15:20:12.646290 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:12.646261 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/27ffe7dc-0f8f-4d32-882b-6b0829ab71ad-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-9f7tp\" (UID: \"27ffe7dc-0f8f-4d32-882b-6b0829ab71ad\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9f7tp" Jul 07 15:20:12.653872 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:12.653814 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgwhb\" (UniqueName: \"kubernetes.io/projected/27ffe7dc-0f8f-4d32-882b-6b0829ab71ad-kube-api-access-kgwhb\") pod \"llmisvc-controller-manager-777c6bd7d5-9f7tp\" (UID: \"27ffe7dc-0f8f-4d32-882b-6b0829ab71ad\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9f7tp" Jul 07 15:20:12.654965 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:12.654946 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dvjn\" (UniqueName: \"kubernetes.io/projected/ff832223-8f66-4299-94e1-35b79a4ccc22-kube-api-access-7dvjn\") pod \"kserve-controller-manager-d6f4b65b8-t97r8\" (UID: \"ff832223-8f66-4299-94e1-35b79a4ccc22\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-t97r8" Jul 07 15:20:12.774461 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:12.774373 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9f7tp" Jul 07 15:20:12.947075 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:20:12.947043 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod27ffe7dc_0f8f_4d32_882b_6b0829ab71ad.slice/crio-d878990837ed95d93db45213e444a3a5cbe4ea47084f7ab7036d56f4bf1f1858 WatchSource:0}: Error finding container d878990837ed95d93db45213e444a3a5cbe4ea47084f7ab7036d56f4bf1f1858: Status 404 returned error can't find the container with id d878990837ed95d93db45213e444a3a5cbe4ea47084f7ab7036d56f4bf1f1858 Jul 07 15:20:12.949242 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:12.949219 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-9f7tp"] Jul 07 15:20:13.148916 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:13.148881 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ff832223-8f66-4299-94e1-35b79a4ccc22-cert\") pod \"kserve-controller-manager-d6f4b65b8-t97r8\" (UID: \"ff832223-8f66-4299-94e1-35b79a4ccc22\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-t97r8" Jul 07 15:20:13.151337 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:13.151312 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ff832223-8f66-4299-94e1-35b79a4ccc22-cert\") pod \"kserve-controller-manager-d6f4b65b8-t97r8\" (UID: \"ff832223-8f66-4299-94e1-35b79a4ccc22\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-t97r8" Jul 07 15:20:13.358968 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:13.358931 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-t97r8" Jul 07 15:20:13.495318 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:13.495291 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-t97r8"] Jul 07 15:20:13.497813 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:20:13.497787 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff832223_8f66_4299_94e1_35b79a4ccc22.slice/crio-45dd04bba1b5152c1aabbf5ac413c0ca7e4e9ca8e103bc084172434760d75ea1 WatchSource:0}: Error finding container 45dd04bba1b5152c1aabbf5ac413c0ca7e4e9ca8e103bc084172434760d75ea1: Status 404 returned error can't find the container with id 45dd04bba1b5152c1aabbf5ac413c0ca7e4e9ca8e103bc084172434760d75ea1 Jul 07 15:20:13.729570 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:13.729483 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-t97r8" event={"ID":"ff832223-8f66-4299-94e1-35b79a4ccc22","Type":"ContainerStarted","Data":"45dd04bba1b5152c1aabbf5ac413c0ca7e4e9ca8e103bc084172434760d75ea1"} Jul 07 15:20:13.730603 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:13.730567 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9f7tp" event={"ID":"27ffe7dc-0f8f-4d32-882b-6b0829ab71ad","Type":"ContainerStarted","Data":"d878990837ed95d93db45213e444a3a5cbe4ea47084f7ab7036d56f4bf1f1858"} Jul 07 15:20:14.194000 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:14.193965 2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-xgf6x"] Jul 07 15:20:14.194177 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:14.194156 2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-xgf6x" podUID="6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d" containerName="discovery" containerID="cri-o://6ac6ce8599420ffc52f312cc14fa42e399875b0085f1ff48df6fb199ac3ac2ff" gracePeriod=30 Jul 07 15:20:14.572960 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:14.572925 2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-xgf6x" Jul 07 15:20:14.664025 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:14.663990 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d-istio-token\") pod \"6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d\" (UID: \"6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d\") " Jul 07 15:20:14.664195 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:14.664040 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d-istio-csr-dns-cert\") pod \"6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d\" (UID: \"6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d\") " Jul 07 15:20:14.664195 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:14.664127 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d-istio-csr-ca-configmap\") pod \"6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d\" (UID: \"6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d\") " Jul 07 15:20:14.664195 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:14.664162 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d-local-certs\") pod \"6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d\" (UID: \"6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d\") " Jul 07 15:20:14.664195 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:14.664192 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d-cacerts\") pod \"6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d\" (UID: \"6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d\") " Jul 07 15:20:14.664426 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:14.664230 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f58mq\" (UniqueName: \"kubernetes.io/projected/6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d-kube-api-access-f58mq\") pod \"6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d\" (UID: \"6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d\") " Jul 07 15:20:14.664426 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:14.664319 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d-istio-kubeconfig\") pod \"6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d\" (UID: \"6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d\") " Jul 07 15:20:14.664677 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:14.664567 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d" (UID: "6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:20:14.667734 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:14.667685 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d" (UID: "6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:20:14.669628 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:14.669522 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d-local-certs" (OuterVolumeSpecName: "local-certs") pod "6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d" (UID: "6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:20:14.669628 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:14.669525 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d-istio-token" (OuterVolumeSpecName: "istio-token") pod "6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d" (UID: "6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:20:14.669628 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:14.669624 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d-kube-api-access-f58mq" (OuterVolumeSpecName: "kube-api-access-f58mq") pod "6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d" (UID: "6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d"). InnerVolumeSpecName "kube-api-access-f58mq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:20:14.669862 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:14.669626 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d" (UID: "6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:20:14.669862 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:14.669668 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d-cacerts" (OuterVolumeSpecName: "cacerts") pod "6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d" (UID: "6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:20:14.740663 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:14.740628 2549 generic.go:356] "Generic (PLEG): container finished" podID="6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d" containerID="6ac6ce8599420ffc52f312cc14fa42e399875b0085f1ff48df6fb199ac3ac2ff" exitCode=0 Jul 07 15:20:14.740844 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:14.740671 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-xgf6x" event={"ID":"6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d","Type":"ContainerDied","Data":"6ac6ce8599420ffc52f312cc14fa42e399875b0085f1ff48df6fb199ac3ac2ff"} Jul 07 15:20:14.740844 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:14.740694 2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-xgf6x" Jul 07 15:20:14.740844 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:14.740773 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-xgf6x" event={"ID":"6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d","Type":"ContainerDied","Data":"b0fff63baa68c041eeb3a20125fed111f7fd4f49563ef78595f4b79e09e67e73"} Jul 07 15:20:14.740844 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:14.740818 2549 scope.go:117] "RemoveContainer" containerID="6ac6ce8599420ffc52f312cc14fa42e399875b0085f1ff48df6fb199ac3ac2ff" Jul 07 15:20:14.756748 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:14.756720 2549 scope.go:117] "RemoveContainer" containerID="6ac6ce8599420ffc52f312cc14fa42e399875b0085f1ff48df6fb199ac3ac2ff" Jul 07 15:20:14.757218 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:20:14.757192 2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ac6ce8599420ffc52f312cc14fa42e399875b0085f1ff48df6fb199ac3ac2ff\": container with ID starting with 6ac6ce8599420ffc52f312cc14fa42e399875b0085f1ff48df6fb199ac3ac2ff not found: ID does not exist" containerID="6ac6ce8599420ffc52f312cc14fa42e399875b0085f1ff48df6fb199ac3ac2ff" Jul 07 15:20:14.757316 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:14.757232 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ac6ce8599420ffc52f312cc14fa42e399875b0085f1ff48df6fb199ac3ac2ff"} err="failed to get container status \"6ac6ce8599420ffc52f312cc14fa42e399875b0085f1ff48df6fb199ac3ac2ff\": rpc error: code = NotFound desc = could not find container \"6ac6ce8599420ffc52f312cc14fa42e399875b0085f1ff48df6fb199ac3ac2ff\": container with ID starting with 6ac6ce8599420ffc52f312cc14fa42e399875b0085f1ff48df6fb199ac3ac2ff not found: ID does not exist" Jul 07 15:20:14.765363 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:14.765325 2549 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d-istio-csr-ca-configmap\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:20:14.765543 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:14.765370 2549 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d-local-certs\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:20:14.765543 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:14.765387 2549 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d-cacerts\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:20:14.765543 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:14.765398 2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f58mq\" (UniqueName: \"kubernetes.io/projected/6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d-kube-api-access-f58mq\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:20:14.765543 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:14.765408 2549 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d-istio-kubeconfig\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:20:14.765543 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:14.765421 2549 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d-istio-token\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:20:14.765543 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:14.765435 2549 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d-istio-csr-dns-cert\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:20:14.768503 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:14.768471 2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-xgf6x"] Jul 07 15:20:14.771283 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:14.771249 2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-xgf6x"] Jul 07 15:20:16.536931 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:16.536900 2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d" path="/var/lib/kubelet/pods/6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d/volumes" Jul 07 15:20:17.699307 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:17.699273 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-7fshj" Jul 07 15:20:17.702544 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:17.702523 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-dc8f7" Jul 07 15:20:17.763899 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:17.763849 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-t97r8" event={"ID":"ff832223-8f66-4299-94e1-35b79a4ccc22","Type":"ContainerStarted","Data":"65113a6cc99e044426816ee644c05aa3e556298bc5fd0292b00e708cf3bf5e79"} Jul 07 15:20:17.764080 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:17.763982 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-t97r8" Jul 07 15:20:17.765804 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:17.765770 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9f7tp" event={"ID":"27ffe7dc-0f8f-4d32-882b-6b0829ab71ad","Type":"ContainerStarted","Data":"0332c3c18e8a850a9e00ae6ae4978418012ff99c6b1469867c2c88b1ac3aa631"} Jul 07 15:20:17.765944 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:17.765876 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9f7tp" Jul 07 15:20:17.791195 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:17.791140 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-t97r8" podStartSLOduration=2.299395065 podStartE2EDuration="5.79112019s" totalImagesPullingTime="3.491725125s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:20:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:20:17.789993034 +0000 UTC m=+435.797470483" watchObservedRunningTime="2026-07-07 15:20:17.79112019 +0000 UTC m=+435.798597641" Jul 07 15:20:17.815359 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:17.815304 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9f7tp" podStartSLOduration=1.773320486 podStartE2EDuration="5.815283832s" totalImagesPullingTime="4.041963346s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:20:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:20:17.81209827 +0000 UTC m=+435.819575731" watchObservedRunningTime="2026-07-07 15:20:17.815283832 +0000 UTC m=+435.822761283" Jul 07 15:20:48.773145 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:48.773116 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9f7tp" Jul 07 15:20:48.776076 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:48.776060 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-t97r8" Jul 07 15:20:59.351067 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:59.351028 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-frls8"] Jul 07 15:20:59.351636 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:59.351618 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d" containerName="discovery" Jul 07 15:20:59.351717 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:59.351639 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d" containerName="discovery" Jul 07 15:20:59.351788 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:59.351766 2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e235b41-5ffb-4f5a-b1c2-87daf25a3e4d" containerName="discovery" Jul 07 15:20:59.353977 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:59.353956 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-frls8" Jul 07 15:20:59.356469 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:59.356446 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 07 15:20:59.356604 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:59.356517 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-gzxg5\"" Jul 07 15:20:59.357476 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:59.357458 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 07 15:20:59.376909 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:59.376884 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-frls8"] Jul 07 15:20:59.467394 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:59.467363 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkplr\" (UniqueName: \"kubernetes.io/projected/2a9d6fb5-95d0-4ca4-b5a4-b5e7a122016b-kube-api-access-tkplr\") pod \"kuadrant-operator-catalog-frls8\" (UID: \"2a9d6fb5-95d0-4ca4-b5a4-b5e7a122016b\") " pod="kuadrant-system/kuadrant-operator-catalog-frls8" Jul 07 15:20:59.568642 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:59.568573 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkplr\" (UniqueName: \"kubernetes.io/projected/2a9d6fb5-95d0-4ca4-b5a4-b5e7a122016b-kube-api-access-tkplr\") pod \"kuadrant-operator-catalog-frls8\" (UID: \"2a9d6fb5-95d0-4ca4-b5a4-b5e7a122016b\") " pod="kuadrant-system/kuadrant-operator-catalog-frls8" Jul 07 15:20:59.578064 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:59.578037 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkplr\" (UniqueName: \"kubernetes.io/projected/2a9d6fb5-95d0-4ca4-b5a4-b5e7a122016b-kube-api-access-tkplr\") pod \"kuadrant-operator-catalog-frls8\" (UID: \"2a9d6fb5-95d0-4ca4-b5a4-b5e7a122016b\") " pod="kuadrant-system/kuadrant-operator-catalog-frls8" Jul 07 15:20:59.664875 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:59.664776 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-frls8" Jul 07 15:20:59.702371 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:59.702336 2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-frls8"] Jul 07 15:20:59.803197 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:59.803173 2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-frls8"] Jul 07 15:20:59.805251 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:20:59.805221 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a9d6fb5_95d0_4ca4_b5a4_b5e7a122016b.slice/crio-f7152dbad29f0f734a4a57882fdaa64b60f82f9d8f0fa9673652ac4de6c2822c WatchSource:0}: Error finding container f7152dbad29f0f734a4a57882fdaa64b60f82f9d8f0fa9673652ac4de6c2822c: Status 404 returned error can't find the container with id f7152dbad29f0f734a4a57882fdaa64b60f82f9d8f0fa9673652ac4de6c2822c Jul 07 15:20:59.913032 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:59.912996 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qg79z"] Jul 07 15:20:59.916190 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:59.916146 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-qg79z" Jul 07 15:20:59.927796 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:59.927773 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qg79z"] Jul 07 15:20:59.972192 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:59.972163 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2c9j\" (UniqueName: \"kubernetes.io/projected/79f7e333-21f4-430c-98dc-cdcc9880ba4c-kube-api-access-f2c9j\") pod \"kuadrant-operator-catalog-qg79z\" (UID: \"79f7e333-21f4-430c-98dc-cdcc9880ba4c\") " pod="kuadrant-system/kuadrant-operator-catalog-qg79z" Jul 07 15:20:59.989871 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:20:59.989839 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-frls8" event={"ID":"2a9d6fb5-95d0-4ca4-b5a4-b5e7a122016b","Type":"ContainerStarted","Data":"f7152dbad29f0f734a4a57882fdaa64b60f82f9d8f0fa9673652ac4de6c2822c"} Jul 07 15:21:00.073408 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:00.073367 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2c9j\" (UniqueName: \"kubernetes.io/projected/79f7e333-21f4-430c-98dc-cdcc9880ba4c-kube-api-access-f2c9j\") pod \"kuadrant-operator-catalog-qg79z\" (UID: \"79f7e333-21f4-430c-98dc-cdcc9880ba4c\") " pod="kuadrant-system/kuadrant-operator-catalog-qg79z" Jul 07 15:21:00.082232 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:00.082198 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2c9j\" (UniqueName: \"kubernetes.io/projected/79f7e333-21f4-430c-98dc-cdcc9880ba4c-kube-api-access-f2c9j\") pod \"kuadrant-operator-catalog-qg79z\" (UID: \"79f7e333-21f4-430c-98dc-cdcc9880ba4c\") " pod="kuadrant-system/kuadrant-operator-catalog-qg79z" Jul 07 15:21:00.226472 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:00.226384 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-qg79z" Jul 07 15:21:00.367249 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:00.367226 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qg79z"] Jul 07 15:21:00.369714 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:21:00.369552 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79f7e333_21f4_430c_98dc_cdcc9880ba4c.slice/crio-b5fe78ca58dd31b7d2300d90f6bcf270cec317f798ceea25c5dce8009a46f92e WatchSource:0}: Error finding container b5fe78ca58dd31b7d2300d90f6bcf270cec317f798ceea25c5dce8009a46f92e: Status 404 returned error can't find the container with id b5fe78ca58dd31b7d2300d90f6bcf270cec317f798ceea25c5dce8009a46f92e Jul 07 15:21:00.997607 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:00.997564 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-qg79z" event={"ID":"79f7e333-21f4-430c-98dc-cdcc9880ba4c","Type":"ContainerStarted","Data":"b5fe78ca58dd31b7d2300d90f6bcf270cec317f798ceea25c5dce8009a46f92e"} Jul 07 15:21:03.010154 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:03.010115 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-qg79z" event={"ID":"79f7e333-21f4-430c-98dc-cdcc9880ba4c","Type":"ContainerStarted","Data":"725a615974d7c28a6d92f9cb66dab4e56cf3069bc88b1714e8e39bc4ba35fcb7"} Jul 07 15:21:03.011528 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:03.011504 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-frls8" event={"ID":"2a9d6fb5-95d0-4ca4-b5a4-b5e7a122016b","Type":"ContainerStarted","Data":"1a7d16c8494363bde0993bd0c361cbc4c7a31252882c26f85913aaccbfbe0acf"} Jul 07 15:21:03.011671 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:03.011629 2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-frls8" podUID="2a9d6fb5-95d0-4ca4-b5a4-b5e7a122016b" containerName="registry-server" containerID="cri-o://1a7d16c8494363bde0993bd0c361cbc4c7a31252882c26f85913aaccbfbe0acf" gracePeriod=2 Jul 07 15:21:03.029978 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:03.029938 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-qg79z" podStartSLOduration=1.919532287 podStartE2EDuration="4.02992498s" totalImagesPullingTime="2.110392693s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:20:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:21:03.028563461 +0000 UTC m=+481.036040913" watchObservedRunningTime="2026-07-07 15:21:03.02992498 +0000 UTC m=+481.037402431" Jul 07 15:21:03.051338 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:03.051289 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-frls8" podStartSLOduration=1.373054188 podStartE2EDuration="4.051275376s" totalImagesPullingTime="2.678221188s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:20:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:21:03.050339313 +0000 UTC m=+481.057816782" watchObservedRunningTime="2026-07-07 15:21:03.051275376 +0000 UTC m=+481.058752858" Jul 07 15:21:03.258150 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:03.258127 2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-frls8" Jul 07 15:21:03.315116 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:03.315086 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tkplr\" (UniqueName: \"kubernetes.io/projected/2a9d6fb5-95d0-4ca4-b5a4-b5e7a122016b-kube-api-access-tkplr\") pod \"2a9d6fb5-95d0-4ca4-b5a4-b5e7a122016b\" (UID: \"2a9d6fb5-95d0-4ca4-b5a4-b5e7a122016b\") " Jul 07 15:21:03.317176 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:03.317152 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a9d6fb5-95d0-4ca4-b5a4-b5e7a122016b-kube-api-access-tkplr" (OuterVolumeSpecName: "kube-api-access-tkplr") pod "2a9d6fb5-95d0-4ca4-b5a4-b5e7a122016b" (UID: "2a9d6fb5-95d0-4ca4-b5a4-b5e7a122016b"). InnerVolumeSpecName "kube-api-access-tkplr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:21:03.416038 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:03.416003 2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tkplr\" (UniqueName: \"kubernetes.io/projected/2a9d6fb5-95d0-4ca4-b5a4-b5e7a122016b-kube-api-access-tkplr\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:21:04.017889 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:04.017850 2549 generic.go:356] "Generic (PLEG): container finished" podID="2a9d6fb5-95d0-4ca4-b5a4-b5e7a122016b" containerID="1a7d16c8494363bde0993bd0c361cbc4c7a31252882c26f85913aaccbfbe0acf" exitCode=0 Jul 07 15:21:04.018393 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:04.017925 2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-frls8" Jul 07 15:21:04.018393 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:04.017944 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-frls8" event={"ID":"2a9d6fb5-95d0-4ca4-b5a4-b5e7a122016b","Type":"ContainerDied","Data":"1a7d16c8494363bde0993bd0c361cbc4c7a31252882c26f85913aaccbfbe0acf"} Jul 07 15:21:04.018393 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:04.017993 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-frls8" event={"ID":"2a9d6fb5-95d0-4ca4-b5a4-b5e7a122016b","Type":"ContainerDied","Data":"f7152dbad29f0f734a4a57882fdaa64b60f82f9d8f0fa9673652ac4de6c2822c"} Jul 07 15:21:04.018393 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:04.018013 2549 scope.go:117] "RemoveContainer" containerID="1a7d16c8494363bde0993bd0c361cbc4c7a31252882c26f85913aaccbfbe0acf" Jul 07 15:21:04.027803 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:04.027786 2549 scope.go:117] "RemoveContainer" containerID="1a7d16c8494363bde0993bd0c361cbc4c7a31252882c26f85913aaccbfbe0acf" Jul 07 15:21:04.028085 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:21:04.028064 2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a7d16c8494363bde0993bd0c361cbc4c7a31252882c26f85913aaccbfbe0acf\": container with ID starting with 1a7d16c8494363bde0993bd0c361cbc4c7a31252882c26f85913aaccbfbe0acf not found: ID does not exist" containerID="1a7d16c8494363bde0993bd0c361cbc4c7a31252882c26f85913aaccbfbe0acf" Jul 07 15:21:04.028128 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:04.028094 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a7d16c8494363bde0993bd0c361cbc4c7a31252882c26f85913aaccbfbe0acf"} err="failed to get container status \"1a7d16c8494363bde0993bd0c361cbc4c7a31252882c26f85913aaccbfbe0acf\": rpc error: code = NotFound desc = could not find container \"1a7d16c8494363bde0993bd0c361cbc4c7a31252882c26f85913aaccbfbe0acf\": container with ID starting with 1a7d16c8494363bde0993bd0c361cbc4c7a31252882c26f85913aaccbfbe0acf not found: ID does not exist" Jul 07 15:21:04.041296 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:04.041268 2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-frls8"] Jul 07 15:21:04.043348 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:04.043327 2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-frls8"] Jul 07 15:21:04.535576 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:04.535542 2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a9d6fb5-95d0-4ca4-b5a4-b5e7a122016b" path="/var/lib/kubelet/pods/2a9d6fb5-95d0-4ca4-b5a4-b5e7a122016b/volumes" Jul 07 15:21:10.226730 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:10.226691 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-qg79z" Jul 07 15:21:10.227162 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:10.226763 2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-qg79z" Jul 07 15:21:10.251207 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:10.251180 2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-qg79z" Jul 07 15:21:11.078605 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:11.078556 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-qg79z" Jul 07 15:21:14.973062 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:14.973026 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tfcvf"] Jul 07 15:21:14.973485 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:14.973429 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a9d6fb5-95d0-4ca4-b5a4-b5e7a122016b" containerName="registry-server" Jul 07 15:21:14.973485 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:14.973440 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a9d6fb5-95d0-4ca4-b5a4-b5e7a122016b" containerName="registry-server" Jul 07 15:21:14.973559 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:14.973507 2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a9d6fb5-95d0-4ca4-b5a4-b5e7a122016b" containerName="registry-server" Jul 07 15:21:14.978154 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:14.978134 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tfcvf" Jul 07 15:21:14.980554 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:14.980531 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-z8hxh\"" Jul 07 15:21:14.989751 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:14.989725 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tfcvf"] Jul 07 15:21:15.129564 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:15.129524 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncng5\" (UniqueName: \"kubernetes.io/projected/2d88f62c-1bee-4cf7-a1e8-5d009e7920f4-kube-api-access-ncng5\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tfcvf\" (UID: \"2d88f62c-1bee-4cf7-a1e8-5d009e7920f4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tfcvf" Jul 07 15:21:15.129564 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:15.129566 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2d88f62c-1bee-4cf7-a1e8-5d009e7920f4-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tfcvf\" (UID: \"2d88f62c-1bee-4cf7-a1e8-5d009e7920f4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tfcvf" Jul 07 15:21:15.129822 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:15.129636 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2d88f62c-1bee-4cf7-a1e8-5d009e7920f4-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tfcvf\" (UID: \"2d88f62c-1bee-4cf7-a1e8-5d009e7920f4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tfcvf" Jul 07 15:21:15.230673 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:15.230558 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncng5\" (UniqueName: \"kubernetes.io/projected/2d88f62c-1bee-4cf7-a1e8-5d009e7920f4-kube-api-access-ncng5\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tfcvf\" (UID: \"2d88f62c-1bee-4cf7-a1e8-5d009e7920f4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tfcvf" Jul 07 15:21:15.230673 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:15.230625 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2d88f62c-1bee-4cf7-a1e8-5d009e7920f4-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tfcvf\" (UID: \"2d88f62c-1bee-4cf7-a1e8-5d009e7920f4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tfcvf" Jul 07 15:21:15.230673 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:15.230654 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2d88f62c-1bee-4cf7-a1e8-5d009e7920f4-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tfcvf\" (UID: \"2d88f62c-1bee-4cf7-a1e8-5d009e7920f4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tfcvf" Jul 07 15:21:15.230966 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:15.230945 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2d88f62c-1bee-4cf7-a1e8-5d009e7920f4-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tfcvf\" (UID: \"2d88f62c-1bee-4cf7-a1e8-5d009e7920f4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tfcvf" Jul 07 15:21:15.231025 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:15.231007 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2d88f62c-1bee-4cf7-a1e8-5d009e7920f4-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tfcvf\" (UID: \"2d88f62c-1bee-4cf7-a1e8-5d009e7920f4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tfcvf" Jul 07 15:21:15.238718 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:15.238684 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncng5\" (UniqueName: \"kubernetes.io/projected/2d88f62c-1bee-4cf7-a1e8-5d009e7920f4-kube-api-access-ncng5\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tfcvf\" (UID: \"2d88f62c-1bee-4cf7-a1e8-5d009e7920f4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tfcvf" Jul 07 15:21:15.288398 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:15.288360 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tfcvf" Jul 07 15:21:15.355553 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:15.355499 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1flzkq"] Jul 07 15:21:15.368907 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:15.368863 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1flzkq" Jul 07 15:21:15.392475 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:15.392438 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1flzkq"] Jul 07 15:21:15.460887 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:15.460856 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tfcvf"] Jul 07 15:21:15.463191 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:21:15.463165 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d88f62c_1bee_4cf7_a1e8_5d009e7920f4.slice/crio-17e2f43123e8b666d118abed01ee304e37281d58f7e0dbb2e8f3963dede0c6f9 WatchSource:0}: Error finding container 17e2f43123e8b666d118abed01ee304e37281d58f7e0dbb2e8f3963dede0c6f9: Status 404 returned error can't find the container with id 17e2f43123e8b666d118abed01ee304e37281d58f7e0dbb2e8f3963dede0c6f9 Jul 07 15:21:15.534454 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:15.534421 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f56fd4a1-89f3-476f-98dc-19005651962b-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1flzkq\" (UID: \"f56fd4a1-89f3-476f-98dc-19005651962b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1flzkq" Jul 07 15:21:15.534649 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:15.534461 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4r68w\" (UniqueName: \"kubernetes.io/projected/f56fd4a1-89f3-476f-98dc-19005651962b-kube-api-access-4r68w\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1flzkq\" (UID: \"f56fd4a1-89f3-476f-98dc-19005651962b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1flzkq" Jul 07 15:21:15.534649 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:15.534533 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f56fd4a1-89f3-476f-98dc-19005651962b-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1flzkq\" (UID: \"f56fd4a1-89f3-476f-98dc-19005651962b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1flzkq" Jul 07 15:21:15.635884 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:15.635846 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f56fd4a1-89f3-476f-98dc-19005651962b-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1flzkq\" (UID: \"f56fd4a1-89f3-476f-98dc-19005651962b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1flzkq" Jul 07 15:21:15.635884 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:15.635887 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4r68w\" (UniqueName: \"kubernetes.io/projected/f56fd4a1-89f3-476f-98dc-19005651962b-kube-api-access-4r68w\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1flzkq\" (UID: \"f56fd4a1-89f3-476f-98dc-19005651962b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1flzkq" Jul 07 15:21:15.636119 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:15.635942 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f56fd4a1-89f3-476f-98dc-19005651962b-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1flzkq\" (UID: \"f56fd4a1-89f3-476f-98dc-19005651962b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1flzkq" Jul 07 15:21:15.636330 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:15.636304 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f56fd4a1-89f3-476f-98dc-19005651962b-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1flzkq\" (UID: \"f56fd4a1-89f3-476f-98dc-19005651962b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1flzkq" Jul 07 15:21:15.636374 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:15.636334 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f56fd4a1-89f3-476f-98dc-19005651962b-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1flzkq\" (UID: \"f56fd4a1-89f3-476f-98dc-19005651962b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1flzkq" Jul 07 15:21:15.644040 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:15.644011 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4r68w\" (UniqueName: \"kubernetes.io/projected/f56fd4a1-89f3-476f-98dc-19005651962b-kube-api-access-4r68w\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1flzkq\" (UID: \"f56fd4a1-89f3-476f-98dc-19005651962b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1flzkq" Jul 07 15:21:15.708962 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:15.708924 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1flzkq" Jul 07 15:21:15.760286 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:15.759742 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vhnst"] Jul 07 15:21:15.765112 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:15.765082 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vhnst" Jul 07 15:21:15.772652 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:15.772622 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vhnst"] Jul 07 15:21:15.838213 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:15.838179 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmdql\" (UniqueName: \"kubernetes.io/projected/2e0bce97-2257-42da-83d3-e2d9fe891e0e-kube-api-access-bmdql\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vhnst\" (UID: \"2e0bce97-2257-42da-83d3-e2d9fe891e0e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vhnst" Jul 07 15:21:15.838399 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:15.838239 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e0bce97-2257-42da-83d3-e2d9fe891e0e-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vhnst\" (UID: \"2e0bce97-2257-42da-83d3-e2d9fe891e0e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vhnst" Jul 07 15:21:15.838399 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:15.838279 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e0bce97-2257-42da-83d3-e2d9fe891e0e-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vhnst\" (UID: \"2e0bce97-2257-42da-83d3-e2d9fe891e0e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vhnst" Jul 07 15:21:15.854430 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:15.854213 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1flzkq"] Jul 07 15:21:15.856647 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:21:15.856617 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf56fd4a1_89f3_476f_98dc_19005651962b.slice/crio-7e8a4e47594ff9159c4ed8afb118b56892bcf6ad75fe30d4b9c4e78ad1fc1ab9 WatchSource:0}: Error finding container 7e8a4e47594ff9159c4ed8afb118b56892bcf6ad75fe30d4b9c4e78ad1fc1ab9: Status 404 returned error can't find the container with id 7e8a4e47594ff9159c4ed8afb118b56892bcf6ad75fe30d4b9c4e78ad1fc1ab9 Jul 07 15:21:15.939338 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:15.939296 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmdql\" (UniqueName: \"kubernetes.io/projected/2e0bce97-2257-42da-83d3-e2d9fe891e0e-kube-api-access-bmdql\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vhnst\" (UID: \"2e0bce97-2257-42da-83d3-e2d9fe891e0e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vhnst" Jul 07 15:21:15.939528 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:15.939350 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e0bce97-2257-42da-83d3-e2d9fe891e0e-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vhnst\" (UID: \"2e0bce97-2257-42da-83d3-e2d9fe891e0e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vhnst" Jul 07 15:21:15.939647 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:15.939527 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e0bce97-2257-42da-83d3-e2d9fe891e0e-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vhnst\" (UID: \"2e0bce97-2257-42da-83d3-e2d9fe891e0e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vhnst" Jul 07 15:21:15.939781 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:15.939764 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e0bce97-2257-42da-83d3-e2d9fe891e0e-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vhnst\" (UID: \"2e0bce97-2257-42da-83d3-e2d9fe891e0e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vhnst" Jul 07 15:21:15.939918 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:15.939898 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e0bce97-2257-42da-83d3-e2d9fe891e0e-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vhnst\" (UID: \"2e0bce97-2257-42da-83d3-e2d9fe891e0e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vhnst" Jul 07 15:21:15.948751 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:15.948716 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmdql\" (UniqueName: \"kubernetes.io/projected/2e0bce97-2257-42da-83d3-e2d9fe891e0e-kube-api-access-bmdql\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vhnst\" (UID: \"2e0bce97-2257-42da-83d3-e2d9fe891e0e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vhnst" Jul 07 15:21:16.079069 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:16.079028 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vhnst" Jul 07 15:21:16.085687 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:16.085658 2549 generic.go:356] "Generic (PLEG): container finished" podID="f56fd4a1-89f3-476f-98dc-19005651962b" containerID="dc552134777d432353445897790c4c6fa5a75113fdc375d39571e4578728f011" exitCode=0 Jul 07 15:21:16.085811 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:16.085748 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1flzkq" event={"ID":"f56fd4a1-89f3-476f-98dc-19005651962b","Type":"ContainerDied","Data":"dc552134777d432353445897790c4c6fa5a75113fdc375d39571e4578728f011"} Jul 07 15:21:16.085867 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:16.085808 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1flzkq" event={"ID":"f56fd4a1-89f3-476f-98dc-19005651962b","Type":"ContainerStarted","Data":"7e8a4e47594ff9159c4ed8afb118b56892bcf6ad75fe30d4b9c4e78ad1fc1ab9"} Jul 07 15:21:16.087406 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:16.087382 2549 generic.go:356] "Generic (PLEG): container finished" podID="2d88f62c-1bee-4cf7-a1e8-5d009e7920f4" containerID="680a022b517f8f9b3e40b2fadf9c991844e8b0a5db67861e172d2f4c834a88ed" exitCode=0 Jul 07 15:21:16.087507 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:16.087429 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tfcvf" event={"ID":"2d88f62c-1bee-4cf7-a1e8-5d009e7920f4","Type":"ContainerDied","Data":"680a022b517f8f9b3e40b2fadf9c991844e8b0a5db67861e172d2f4c834a88ed"} Jul 07 15:21:16.087507 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:16.087460 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tfcvf" event={"ID":"2d88f62c-1bee-4cf7-a1e8-5d009e7920f4","Type":"ContainerStarted","Data":"17e2f43123e8b666d118abed01ee304e37281d58f7e0dbb2e8f3963dede0c6f9"} Jul 07 15:21:16.174196 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:16.174158 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jmnk5"] Jul 07 15:21:16.181285 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:16.181250 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jmnk5" Jul 07 15:21:16.190025 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:16.189966 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jmnk5"] Jul 07 15:21:16.225538 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:16.225510 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vhnst"] Jul 07 15:21:16.228181 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:21:16.228153 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e0bce97_2257_42da_83d3_e2d9fe891e0e.slice/crio-1e60367b564e15092d43003e964a203ffd5cd0dd33d2c563737c20dece948e99 WatchSource:0}: Error finding container 1e60367b564e15092d43003e964a203ffd5cd0dd33d2c563737c20dece948e99: Status 404 returned error can't find the container with id 1e60367b564e15092d43003e964a203ffd5cd0dd33d2c563737c20dece948e99 Jul 07 15:21:16.344334 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:16.344239 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6b8fcac7-bd9c-48f1-819a-918c59419cce-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jmnk5\" (UID: \"6b8fcac7-bd9c-48f1-819a-918c59419cce\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jmnk5" Jul 07 15:21:16.344474 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:16.344386 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6b8fcac7-bd9c-48f1-819a-918c59419cce-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jmnk5\" (UID: \"6b8fcac7-bd9c-48f1-819a-918c59419cce\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jmnk5" Jul 07 15:21:16.344474 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:16.344417 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mbv9\" (UniqueName: \"kubernetes.io/projected/6b8fcac7-bd9c-48f1-819a-918c59419cce-kube-api-access-9mbv9\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jmnk5\" (UID: \"6b8fcac7-bd9c-48f1-819a-918c59419cce\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jmnk5" Jul 07 15:21:16.445889 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:16.445849 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6b8fcac7-bd9c-48f1-819a-918c59419cce-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jmnk5\" (UID: \"6b8fcac7-bd9c-48f1-819a-918c59419cce\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jmnk5" Jul 07 15:21:16.445889 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:16.445889 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mbv9\" (UniqueName: \"kubernetes.io/projected/6b8fcac7-bd9c-48f1-819a-918c59419cce-kube-api-access-9mbv9\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jmnk5\" (UID: \"6b8fcac7-bd9c-48f1-819a-918c59419cce\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jmnk5" Jul 07 15:21:16.446108 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:16.445944 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6b8fcac7-bd9c-48f1-819a-918c59419cce-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jmnk5\" (UID: \"6b8fcac7-bd9c-48f1-819a-918c59419cce\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jmnk5" Jul 07 15:21:16.446247 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:16.446228 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6b8fcac7-bd9c-48f1-819a-918c59419cce-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jmnk5\" (UID: \"6b8fcac7-bd9c-48f1-819a-918c59419cce\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jmnk5" Jul 07 15:21:16.446281 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:16.446269 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6b8fcac7-bd9c-48f1-819a-918c59419cce-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jmnk5\" (UID: \"6b8fcac7-bd9c-48f1-819a-918c59419cce\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jmnk5" Jul 07 15:21:16.454825 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:16.454795 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mbv9\" (UniqueName: \"kubernetes.io/projected/6b8fcac7-bd9c-48f1-819a-918c59419cce-kube-api-access-9mbv9\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jmnk5\" (UID: \"6b8fcac7-bd9c-48f1-819a-918c59419cce\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jmnk5" Jul 07 15:21:16.493263 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:16.493228 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jmnk5" Jul 07 15:21:16.640081 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:16.640056 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jmnk5"] Jul 07 15:21:16.641758 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:21:16.641732 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b8fcac7_bd9c_48f1_819a_918c59419cce.slice/crio-63d87e6ebe4bef728af14ca1e23b6da928bf16c1153629137731c210034d7ed0 WatchSource:0}: Error finding container 63d87e6ebe4bef728af14ca1e23b6da928bf16c1153629137731c210034d7ed0: Status 404 returned error can't find the container with id 63d87e6ebe4bef728af14ca1e23b6da928bf16c1153629137731c210034d7ed0 Jul 07 15:21:17.094488 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:17.094456 2549 generic.go:356] "Generic (PLEG): container finished" podID="6b8fcac7-bd9c-48f1-819a-918c59419cce" containerID="452a4ee99d7953914a77a9965ee0cc716873e82c347b1364de802452d08e3dbd" exitCode=0 Jul 07 15:21:17.094944 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:17.094541 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jmnk5" event={"ID":"6b8fcac7-bd9c-48f1-819a-918c59419cce","Type":"ContainerDied","Data":"452a4ee99d7953914a77a9965ee0cc716873e82c347b1364de802452d08e3dbd"} Jul 07 15:21:17.094944 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:17.094620 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jmnk5" event={"ID":"6b8fcac7-bd9c-48f1-819a-918c59419cce","Type":"ContainerStarted","Data":"63d87e6ebe4bef728af14ca1e23b6da928bf16c1153629137731c210034d7ed0"} Jul 07 15:21:17.096152 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:17.096125 2549 generic.go:356] "Generic (PLEG): container finished" podID="2e0bce97-2257-42da-83d3-e2d9fe891e0e" containerID="5644640a523213f9393951e39c7008864c7486a972001a4f0988d3b813241016" exitCode=0 Jul 07 15:21:17.096301 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:17.096214 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vhnst" event={"ID":"2e0bce97-2257-42da-83d3-e2d9fe891e0e","Type":"ContainerDied","Data":"5644640a523213f9393951e39c7008864c7486a972001a4f0988d3b813241016"} Jul 07 15:21:17.096301 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:17.096275 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vhnst" event={"ID":"2e0bce97-2257-42da-83d3-e2d9fe891e0e","Type":"ContainerStarted","Data":"1e60367b564e15092d43003e964a203ffd5cd0dd33d2c563737c20dece948e99"} Jul 07 15:21:17.098433 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:17.098394 2549 generic.go:356] "Generic (PLEG): container finished" podID="2d88f62c-1bee-4cf7-a1e8-5d009e7920f4" containerID="512dbceb5fe6afe56875c6716d64567c3bd39623ed5d3201f1a8e46d3a72f593" exitCode=0 Jul 07 15:21:17.098605 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:17.098440 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tfcvf" event={"ID":"2d88f62c-1bee-4cf7-a1e8-5d009e7920f4","Type":"ContainerDied","Data":"512dbceb5fe6afe56875c6716d64567c3bd39623ed5d3201f1a8e46d3a72f593"} Jul 07 15:21:18.108970 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:18.108880 2549 generic.go:356] "Generic (PLEG): container finished" podID="f56fd4a1-89f3-476f-98dc-19005651962b" containerID="65994e960cd6969e1907dcf1469f838a6728c4c1936c27d8ba48131e7c450259" exitCode=0 Jul 07 15:21:18.109409 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:18.108965 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1flzkq" event={"ID":"f56fd4a1-89f3-476f-98dc-19005651962b","Type":"ContainerDied","Data":"65994e960cd6969e1907dcf1469f838a6728c4c1936c27d8ba48131e7c450259"} Jul 07 15:21:18.111044 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:18.111024 2549 generic.go:356] "Generic (PLEG): container finished" podID="6b8fcac7-bd9c-48f1-819a-918c59419cce" containerID="7c3ce7432f20e9abe7dc7079448ba79a7a5f0536f021064df849c6a614764329" exitCode=0 Jul 07 15:21:18.111177 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:18.111111 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jmnk5" event={"ID":"6b8fcac7-bd9c-48f1-819a-918c59419cce","Type":"ContainerDied","Data":"7c3ce7432f20e9abe7dc7079448ba79a7a5f0536f021064df849c6a614764329"} Jul 07 15:21:18.113019 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:18.112987 2549 generic.go:356] "Generic (PLEG): container finished" podID="2e0bce97-2257-42da-83d3-e2d9fe891e0e" containerID="5498a620131cac552e6da6fb9aef0ee420afa1c737d1f944111b9ca6f1ff4be1" exitCode=0 Jul 07 15:21:18.113124 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:18.113064 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vhnst" event={"ID":"2e0bce97-2257-42da-83d3-e2d9fe891e0e","Type":"ContainerDied","Data":"5498a620131cac552e6da6fb9aef0ee420afa1c737d1f944111b9ca6f1ff4be1"} Jul 07 15:21:18.115479 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:18.115456 2549 generic.go:356] "Generic (PLEG): container finished" podID="2d88f62c-1bee-4cf7-a1e8-5d009e7920f4" containerID="25a83d380a1107db95fcee710dcf3da63a3219af1d45c856d7e83c38d360bee9" exitCode=0 Jul 07 15:21:18.115594 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:18.115494 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tfcvf" event={"ID":"2d88f62c-1bee-4cf7-a1e8-5d009e7920f4","Type":"ContainerDied","Data":"25a83d380a1107db95fcee710dcf3da63a3219af1d45c856d7e83c38d360bee9"} Jul 07 15:21:19.123012 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:19.122981 2549 generic.go:356] "Generic (PLEG): container finished" podID="6b8fcac7-bd9c-48f1-819a-918c59419cce" containerID="e0ce67969b989ae7986a406a00a430fa2b951e3dbde05969115a6064758dae4e" exitCode=0 Jul 07 15:21:19.123448 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:19.123060 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jmnk5" event={"ID":"6b8fcac7-bd9c-48f1-819a-918c59419cce","Type":"ContainerDied","Data":"e0ce67969b989ae7986a406a00a430fa2b951e3dbde05969115a6064758dae4e"} Jul 07 15:21:19.125150 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:19.125125 2549 generic.go:356] "Generic (PLEG): container finished" podID="2e0bce97-2257-42da-83d3-e2d9fe891e0e" containerID="0f6102bf8a01f5f58f4966cddb2f43d832304d821c66be9febfc924591b1b29a" exitCode=0 Jul 07 15:21:19.125276 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:19.125200 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vhnst" event={"ID":"2e0bce97-2257-42da-83d3-e2d9fe891e0e","Type":"ContainerDied","Data":"0f6102bf8a01f5f58f4966cddb2f43d832304d821c66be9febfc924591b1b29a"} Jul 07 15:21:19.127020 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:19.126999 2549 generic.go:356] "Generic (PLEG): container finished" podID="f56fd4a1-89f3-476f-98dc-19005651962b" containerID="5c2e4c06afe6b8b1102324c4eeb95553c20d4e67f34209d0d0ddd5730a039ea4" exitCode=0 Jul 07 15:21:19.127115 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:19.127044 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1flzkq" event={"ID":"f56fd4a1-89f3-476f-98dc-19005651962b","Type":"ContainerDied","Data":"5c2e4c06afe6b8b1102324c4eeb95553c20d4e67f34209d0d0ddd5730a039ea4"} Jul 07 15:21:19.265180 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:19.265157 2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tfcvf" Jul 07 15:21:19.370707 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:19.370670 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2d88f62c-1bee-4cf7-a1e8-5d009e7920f4-bundle\") pod \"2d88f62c-1bee-4cf7-a1e8-5d009e7920f4\" (UID: \"2d88f62c-1bee-4cf7-a1e8-5d009e7920f4\") " Jul 07 15:21:19.370904 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:19.370732 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2d88f62c-1bee-4cf7-a1e8-5d009e7920f4-util\") pod \"2d88f62c-1bee-4cf7-a1e8-5d009e7920f4\" (UID: \"2d88f62c-1bee-4cf7-a1e8-5d009e7920f4\") " Jul 07 15:21:19.370904 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:19.370762 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ncng5\" (UniqueName: \"kubernetes.io/projected/2d88f62c-1bee-4cf7-a1e8-5d009e7920f4-kube-api-access-ncng5\") pod \"2d88f62c-1bee-4cf7-a1e8-5d009e7920f4\" (UID: \"2d88f62c-1bee-4cf7-a1e8-5d009e7920f4\") " Jul 07 15:21:19.371176 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:19.371151 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d88f62c-1bee-4cf7-a1e8-5d009e7920f4-bundle" (OuterVolumeSpecName: "bundle") pod "2d88f62c-1bee-4cf7-a1e8-5d009e7920f4" (UID: "2d88f62c-1bee-4cf7-a1e8-5d009e7920f4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:21:19.372906 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:19.372885 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d88f62c-1bee-4cf7-a1e8-5d009e7920f4-kube-api-access-ncng5" (OuterVolumeSpecName: "kube-api-access-ncng5") pod "2d88f62c-1bee-4cf7-a1e8-5d009e7920f4" (UID: "2d88f62c-1bee-4cf7-a1e8-5d009e7920f4"). InnerVolumeSpecName "kube-api-access-ncng5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:21:19.378929 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:19.378853 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d88f62c-1bee-4cf7-a1e8-5d009e7920f4-util" (OuterVolumeSpecName: "util") pod "2d88f62c-1bee-4cf7-a1e8-5d009e7920f4" (UID: "2d88f62c-1bee-4cf7-a1e8-5d009e7920f4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:21:19.472344 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:19.472309 2549 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2d88f62c-1bee-4cf7-a1e8-5d009e7920f4-bundle\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:21:19.472344 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:19.472335 2549 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2d88f62c-1bee-4cf7-a1e8-5d009e7920f4-util\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:21:19.472344 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:19.472345 2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ncng5\" (UniqueName: \"kubernetes.io/projected/2d88f62c-1bee-4cf7-a1e8-5d009e7920f4-kube-api-access-ncng5\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:21:20.135124 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:20.135097 2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tfcvf" Jul 07 15:21:20.135738 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:20.135096 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tfcvf" event={"ID":"2d88f62c-1bee-4cf7-a1e8-5d009e7920f4","Type":"ContainerDied","Data":"17e2f43123e8b666d118abed01ee304e37281d58f7e0dbb2e8f3963dede0c6f9"} Jul 07 15:21:20.135738 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:20.135223 2549 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="17e2f43123e8b666d118abed01ee304e37281d58f7e0dbb2e8f3963dede0c6f9" Jul 07 15:21:20.296920 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:20.296887 2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1flzkq" Jul 07 15:21:20.332658 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:20.332631 2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vhnst" Jul 07 15:21:20.335786 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:20.335764 2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jmnk5" Jul 07 15:21:20.381100 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:20.381068 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f56fd4a1-89f3-476f-98dc-19005651962b-bundle\") pod \"f56fd4a1-89f3-476f-98dc-19005651962b\" (UID: \"f56fd4a1-89f3-476f-98dc-19005651962b\") " Jul 07 15:21:20.381304 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:20.381137 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4r68w\" (UniqueName: \"kubernetes.io/projected/f56fd4a1-89f3-476f-98dc-19005651962b-kube-api-access-4r68w\") pod \"f56fd4a1-89f3-476f-98dc-19005651962b\" (UID: \"f56fd4a1-89f3-476f-98dc-19005651962b\") " Jul 07 15:21:20.381304 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:20.381161 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f56fd4a1-89f3-476f-98dc-19005651962b-util\") pod \"f56fd4a1-89f3-476f-98dc-19005651962b\" (UID: \"f56fd4a1-89f3-476f-98dc-19005651962b\") " Jul 07 15:21:20.381677 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:20.381647 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f56fd4a1-89f3-476f-98dc-19005651962b-bundle" (OuterVolumeSpecName: "bundle") pod "f56fd4a1-89f3-476f-98dc-19005651962b" (UID: "f56fd4a1-89f3-476f-98dc-19005651962b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:21:20.383247 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:20.383224 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f56fd4a1-89f3-476f-98dc-19005651962b-kube-api-access-4r68w" (OuterVolumeSpecName: "kube-api-access-4r68w") pod "f56fd4a1-89f3-476f-98dc-19005651962b" (UID: "f56fd4a1-89f3-476f-98dc-19005651962b"). InnerVolumeSpecName "kube-api-access-4r68w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:21:20.389413 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:20.389362 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f56fd4a1-89f3-476f-98dc-19005651962b-util" (OuterVolumeSpecName: "util") pod "f56fd4a1-89f3-476f-98dc-19005651962b" (UID: "f56fd4a1-89f3-476f-98dc-19005651962b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:21:20.482477 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:20.482447 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e0bce97-2257-42da-83d3-e2d9fe891e0e-bundle\") pod \"2e0bce97-2257-42da-83d3-e2d9fe891e0e\" (UID: \"2e0bce97-2257-42da-83d3-e2d9fe891e0e\") " Jul 07 15:21:20.482687 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:20.482539 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bmdql\" (UniqueName: \"kubernetes.io/projected/2e0bce97-2257-42da-83d3-e2d9fe891e0e-kube-api-access-bmdql\") pod \"2e0bce97-2257-42da-83d3-e2d9fe891e0e\" (UID: \"2e0bce97-2257-42da-83d3-e2d9fe891e0e\") " Jul 07 15:21:20.482687 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:20.482562 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9mbv9\" (UniqueName: \"kubernetes.io/projected/6b8fcac7-bd9c-48f1-819a-918c59419cce-kube-api-access-9mbv9\") pod \"6b8fcac7-bd9c-48f1-819a-918c59419cce\" (UID: \"6b8fcac7-bd9c-48f1-819a-918c59419cce\") " Jul 07 15:21:20.482687 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:20.482607 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6b8fcac7-bd9c-48f1-819a-918c59419cce-util\") pod \"6b8fcac7-bd9c-48f1-819a-918c59419cce\" (UID: \"6b8fcac7-bd9c-48f1-819a-918c59419cce\") " Jul 07 15:21:20.482687 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:20.482640 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6b8fcac7-bd9c-48f1-819a-918c59419cce-bundle\") pod \"6b8fcac7-bd9c-48f1-819a-918c59419cce\" (UID: \"6b8fcac7-bd9c-48f1-819a-918c59419cce\") " Jul 07 15:21:20.482899 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:20.482717 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e0bce97-2257-42da-83d3-e2d9fe891e0e-util\") pod \"2e0bce97-2257-42da-83d3-e2d9fe891e0e\" (UID: \"2e0bce97-2257-42da-83d3-e2d9fe891e0e\") " Jul 07 15:21:20.483044 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:20.483022 2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4r68w\" (UniqueName: \"kubernetes.io/projected/f56fd4a1-89f3-476f-98dc-19005651962b-kube-api-access-4r68w\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:21:20.483044 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:20.483043 2549 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f56fd4a1-89f3-476f-98dc-19005651962b-util\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:21:20.483226 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:20.483056 2549 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f56fd4a1-89f3-476f-98dc-19005651962b-bundle\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:21:20.483315 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:20.483291 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e0bce97-2257-42da-83d3-e2d9fe891e0e-bundle" (OuterVolumeSpecName: "bundle") pod "2e0bce97-2257-42da-83d3-e2d9fe891e0e" (UID: "2e0bce97-2257-42da-83d3-e2d9fe891e0e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:21:20.483543 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:20.483516 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b8fcac7-bd9c-48f1-819a-918c59419cce-bundle" (OuterVolumeSpecName: "bundle") pod "6b8fcac7-bd9c-48f1-819a-918c59419cce" (UID: "6b8fcac7-bd9c-48f1-819a-918c59419cce"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:21:20.484841 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:20.484820 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e0bce97-2257-42da-83d3-e2d9fe891e0e-kube-api-access-bmdql" (OuterVolumeSpecName: "kube-api-access-bmdql") pod "2e0bce97-2257-42da-83d3-e2d9fe891e0e" (UID: "2e0bce97-2257-42da-83d3-e2d9fe891e0e"). InnerVolumeSpecName "kube-api-access-bmdql". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:21:20.485074 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:20.485056 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b8fcac7-bd9c-48f1-819a-918c59419cce-kube-api-access-9mbv9" (OuterVolumeSpecName: "kube-api-access-9mbv9") pod "6b8fcac7-bd9c-48f1-819a-918c59419cce" (UID: "6b8fcac7-bd9c-48f1-819a-918c59419cce"). InnerVolumeSpecName "kube-api-access-9mbv9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:21:20.489212 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:20.489187 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b8fcac7-bd9c-48f1-819a-918c59419cce-util" (OuterVolumeSpecName: "util") pod "6b8fcac7-bd9c-48f1-819a-918c59419cce" (UID: "6b8fcac7-bd9c-48f1-819a-918c59419cce"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:21:20.490398 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:20.490373 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e0bce97-2257-42da-83d3-e2d9fe891e0e-util" (OuterVolumeSpecName: "util") pod "2e0bce97-2257-42da-83d3-e2d9fe891e0e" (UID: "2e0bce97-2257-42da-83d3-e2d9fe891e0e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:21:20.584028 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:20.583994 2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bmdql\" (UniqueName: \"kubernetes.io/projected/2e0bce97-2257-42da-83d3-e2d9fe891e0e-kube-api-access-bmdql\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:21:20.584028 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:20.584029 2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9mbv9\" (UniqueName: \"kubernetes.io/projected/6b8fcac7-bd9c-48f1-819a-918c59419cce-kube-api-access-9mbv9\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:21:20.584240 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:20.584041 2549 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6b8fcac7-bd9c-48f1-819a-918c59419cce-util\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:21:20.584240 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:20.584054 2549 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6b8fcac7-bd9c-48f1-819a-918c59419cce-bundle\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:21:20.584240 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:20.584064 2549 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e0bce97-2257-42da-83d3-e2d9fe891e0e-util\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:21:20.584240 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:20.584074 2549 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e0bce97-2257-42da-83d3-e2d9fe891e0e-bundle\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:21:21.141914 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:21.141877 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1flzkq" event={"ID":"f56fd4a1-89f3-476f-98dc-19005651962b","Type":"ContainerDied","Data":"7e8a4e47594ff9159c4ed8afb118b56892bcf6ad75fe30d4b9c4e78ad1fc1ab9"} Jul 07 15:21:21.142368 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:21.141929 2549 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7e8a4e47594ff9159c4ed8afb118b56892bcf6ad75fe30d4b9c4e78ad1fc1ab9" Jul 07 15:21:21.142368 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:21.141881 2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1flzkq" Jul 07 15:21:21.144063 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:21.144041 2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jmnk5" Jul 07 15:21:21.144063 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:21.144051 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jmnk5" event={"ID":"6b8fcac7-bd9c-48f1-819a-918c59419cce","Type":"ContainerDied","Data":"63d87e6ebe4bef728af14ca1e23b6da928bf16c1153629137731c210034d7ed0"} Jul 07 15:21:21.144263 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:21.144093 2549 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="63d87e6ebe4bef728af14ca1e23b6da928bf16c1153629137731c210034d7ed0" Jul 07 15:21:21.146172 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:21.146146 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vhnst" event={"ID":"2e0bce97-2257-42da-83d3-e2d9fe891e0e","Type":"ContainerDied","Data":"1e60367b564e15092d43003e964a203ffd5cd0dd33d2c563737c20dece948e99"} Jul 07 15:21:21.146346 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:21.146175 2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vhnst" Jul 07 15:21:21.146346 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:21.146187 2549 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1e60367b564e15092d43003e964a203ffd5cd0dd33d2c563737c20dece948e99" Jul 07 15:21:27.426259 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:27.426224 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m95fv"] Jul 07 15:21:27.426674 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:27.426639 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d88f62c-1bee-4cf7-a1e8-5d009e7920f4" containerName="pull" Jul 07 15:21:27.426674 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:27.426650 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d88f62c-1bee-4cf7-a1e8-5d009e7920f4" containerName="pull" Jul 07 15:21:27.426674 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:27.426660 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e0bce97-2257-42da-83d3-e2d9fe891e0e" containerName="util" Jul 07 15:21:27.426674 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:27.426665 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e0bce97-2257-42da-83d3-e2d9fe891e0e" containerName="util" Jul 07 15:21:27.426810 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:27.426678 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f56fd4a1-89f3-476f-98dc-19005651962b" containerName="pull" Jul 07 15:21:27.426810 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:27.426683 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="f56fd4a1-89f3-476f-98dc-19005651962b" containerName="pull" Jul 07 15:21:27.426810 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:27.426689 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f56fd4a1-89f3-476f-98dc-19005651962b" containerName="extract" Jul 07 15:21:27.426810 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:27.426695 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="f56fd4a1-89f3-476f-98dc-19005651962b" containerName="extract" Jul 07 15:21:27.426810 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:27.426704 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f56fd4a1-89f3-476f-98dc-19005651962b" containerName="util" Jul 07 15:21:27.426810 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:27.426709 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="f56fd4a1-89f3-476f-98dc-19005651962b" containerName="util" Jul 07 15:21:27.426810 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:27.426714 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b8fcac7-bd9c-48f1-819a-918c59419cce" containerName="pull" Jul 07 15:21:27.426810 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:27.426718 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b8fcac7-bd9c-48f1-819a-918c59419cce" containerName="pull" Jul 07 15:21:27.426810 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:27.426725 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e0bce97-2257-42da-83d3-e2d9fe891e0e" containerName="extract" Jul 07 15:21:27.426810 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:27.426729 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e0bce97-2257-42da-83d3-e2d9fe891e0e" containerName="extract" Jul 07 15:21:27.426810 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:27.426739 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d88f62c-1bee-4cf7-a1e8-5d009e7920f4" containerName="util" Jul 07 15:21:27.426810 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:27.426744 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d88f62c-1bee-4cf7-a1e8-5d009e7920f4" containerName="util" Jul 07 15:21:27.426810 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:27.426748 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d88f62c-1bee-4cf7-a1e8-5d009e7920f4" containerName="extract" Jul 07 15:21:27.426810 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:27.426753 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d88f62c-1bee-4cf7-a1e8-5d009e7920f4" containerName="extract" Jul 07 15:21:27.426810 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:27.426758 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b8fcac7-bd9c-48f1-819a-918c59419cce" containerName="extract" Jul 07 15:21:27.426810 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:27.426763 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b8fcac7-bd9c-48f1-819a-918c59419cce" containerName="extract" Jul 07 15:21:27.426810 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:27.426771 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b8fcac7-bd9c-48f1-819a-918c59419cce" containerName="util" Jul 07 15:21:27.426810 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:27.426777 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b8fcac7-bd9c-48f1-819a-918c59419cce" containerName="util" Jul 07 15:21:27.426810 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:27.426782 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e0bce97-2257-42da-83d3-e2d9fe891e0e" containerName="pull" Jul 07 15:21:27.426810 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:27.426789 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e0bce97-2257-42da-83d3-e2d9fe891e0e" containerName="pull" Jul 07 15:21:27.427393 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:27.426855 2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d88f62c-1bee-4cf7-a1e8-5d009e7920f4" containerName="extract" Jul 07 15:21:27.427393 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:27.426865 2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e0bce97-2257-42da-83d3-e2d9fe891e0e" containerName="extract" Jul 07 15:21:27.427393 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:27.426872 2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="f56fd4a1-89f3-476f-98dc-19005651962b" containerName="extract" Jul 07 15:21:27.427393 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:27.426881 2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b8fcac7-bd9c-48f1-819a-918c59419cce" containerName="extract" Jul 07 15:21:27.429819 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:27.429804 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m95fv" Jul 07 15:21:27.432310 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:27.432282 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-wpnqz\"" Jul 07 15:21:27.445720 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:27.445695 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m95fv"] Jul 07 15:21:27.543757 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:27.543722 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rknfp\" (UniqueName: \"kubernetes.io/projected/0ec1dd40-981d-4ca3-99c0-68380384f986-kube-api-access-rknfp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m95fv\" (UID: \"0ec1dd40-981d-4ca3-99c0-68380384f986\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m95fv" Jul 07 15:21:27.543911 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:27.543773 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0ec1dd40-981d-4ca3-99c0-68380384f986-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m95fv\" (UID: \"0ec1dd40-981d-4ca3-99c0-68380384f986\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m95fv" Jul 07 15:21:27.644577 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:27.644537 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rknfp\" (UniqueName: \"kubernetes.io/projected/0ec1dd40-981d-4ca3-99c0-68380384f986-kube-api-access-rknfp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m95fv\" (UID: \"0ec1dd40-981d-4ca3-99c0-68380384f986\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m95fv" Jul 07 15:21:27.644788 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:27.644631 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0ec1dd40-981d-4ca3-99c0-68380384f986-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m95fv\" (UID: \"0ec1dd40-981d-4ca3-99c0-68380384f986\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m95fv" Jul 07 15:21:27.645098 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:27.645079 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0ec1dd40-981d-4ca3-99c0-68380384f986-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m95fv\" (UID: \"0ec1dd40-981d-4ca3-99c0-68380384f986\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m95fv" Jul 07 15:21:27.656304 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:27.656278 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rknfp\" (UniqueName: \"kubernetes.io/projected/0ec1dd40-981d-4ca3-99c0-68380384f986-kube-api-access-rknfp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m95fv\" (UID: \"0ec1dd40-981d-4ca3-99c0-68380384f986\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m95fv" Jul 07 15:21:27.739735 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:27.739651 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m95fv" Jul 07 15:21:27.887886 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:27.887851 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m95fv"] Jul 07 15:21:27.890642 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:21:27.890614 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ec1dd40_981d_4ca3_99c0_68380384f986.slice/crio-896b7d4d5828ddbdc2f49842c44b9e1005bf3d796fe969779b6d19a23c1e847c WatchSource:0}: Error finding container 896b7d4d5828ddbdc2f49842c44b9e1005bf3d796fe969779b6d19a23c1e847c: Status 404 returned error can't find the container with id 896b7d4d5828ddbdc2f49842c44b9e1005bf3d796fe969779b6d19a23c1e847c Jul 07 15:21:28.185947 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:28.185913 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m95fv" event={"ID":"0ec1dd40-981d-4ca3-99c0-68380384f986","Type":"ContainerStarted","Data":"896b7d4d5828ddbdc2f49842c44b9e1005bf3d796fe969779b6d19a23c1e847c"} Jul 07 15:21:29.488521 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:29.488478 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-pwpzp"] Jul 07 15:21:29.492105 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:29.492083 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-pwpzp" Jul 07 15:21:29.496798 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:29.496778 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 07 15:21:29.496923 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:29.496875 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-s62lc\"" Jul 07 15:21:29.505973 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:29.505949 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-pwpzp"] Jul 07 15:21:29.561118 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:29.561077 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9c47\" (UniqueName: \"kubernetes.io/projected/372030ff-029f-461b-8199-5c810f934692-kube-api-access-b9c47\") pod \"dns-operator-controller-manager-864dd78c4c-pwpzp\" (UID: \"372030ff-029f-461b-8199-5c810f934692\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-pwpzp" Jul 07 15:21:29.662351 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:29.662311 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9c47\" (UniqueName: \"kubernetes.io/projected/372030ff-029f-461b-8199-5c810f934692-kube-api-access-b9c47\") pod \"dns-operator-controller-manager-864dd78c4c-pwpzp\" (UID: \"372030ff-029f-461b-8199-5c810f934692\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-pwpzp" Jul 07 15:21:29.675035 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:29.675013 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9c47\" (UniqueName: \"kubernetes.io/projected/372030ff-029f-461b-8199-5c810f934692-kube-api-access-b9c47\") pod \"dns-operator-controller-manager-864dd78c4c-pwpzp\" (UID: \"372030ff-029f-461b-8199-5c810f934692\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-pwpzp" Jul 07 15:21:29.802992 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:29.802913 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-pwpzp" Jul 07 15:21:29.965301 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:29.965258 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-pwpzp"] Jul 07 15:21:29.967497 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:21:29.967452 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod372030ff_029f_461b_8199_5c810f934692.slice/crio-a666650fc618497217cc285e4d55fcf329cdbe3af14ff84da041e561154a5cb3 WatchSource:0}: Error finding container a666650fc618497217cc285e4d55fcf329cdbe3af14ff84da041e561154a5cb3: Status 404 returned error can't find the container with id a666650fc618497217cc285e4d55fcf329cdbe3af14ff84da041e561154a5cb3 Jul 07 15:21:30.200314 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:30.200275 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-pwpzp" event={"ID":"372030ff-029f-461b-8199-5c810f934692","Type":"ContainerStarted","Data":"a666650fc618497217cc285e4d55fcf329cdbe3af14ff84da041e561154a5cb3"} Jul 07 15:21:32.666729 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:32.666698 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-bxk7h"] Jul 07 15:21:32.670548 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:32.670522 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-bxk7h" Jul 07 15:21:32.672969 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:32.672947 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-bj2wn\"" Jul 07 15:21:32.688110 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:32.688081 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-bxk7h"] Jul 07 15:21:32.689732 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:32.689706 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkftf\" (UniqueName: \"kubernetes.io/projected/2cc62d89-fbb0-4e39-95f0-ab676d24f36b-kube-api-access-wkftf\") pod \"authorino-operator-75cc8d5cb7-bxk7h\" (UID: \"2cc62d89-fbb0-4e39-95f0-ab676d24f36b\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-bxk7h" Jul 07 15:21:32.791287 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:32.791247 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkftf\" (UniqueName: \"kubernetes.io/projected/2cc62d89-fbb0-4e39-95f0-ab676d24f36b-kube-api-access-wkftf\") pod \"authorino-operator-75cc8d5cb7-bxk7h\" (UID: \"2cc62d89-fbb0-4e39-95f0-ab676d24f36b\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-bxk7h" Jul 07 15:21:32.804133 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:32.804103 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkftf\" (UniqueName: \"kubernetes.io/projected/2cc62d89-fbb0-4e39-95f0-ab676d24f36b-kube-api-access-wkftf\") pod \"authorino-operator-75cc8d5cb7-bxk7h\" (UID: \"2cc62d89-fbb0-4e39-95f0-ab676d24f36b\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-bxk7h" Jul 07 15:21:32.985640 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:32.985550 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-bxk7h" Jul 07 15:21:33.559059 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:33.559032 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-bxk7h"] Jul 07 15:21:33.560709 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:21:33.560685 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cc62d89_fbb0_4e39_95f0_ab676d24f36b.slice/crio-37e94bf40be9ceee16e7bf35312a8156acdc8fb07dab2f1e683a012eba381df8 WatchSource:0}: Error finding container 37e94bf40be9ceee16e7bf35312a8156acdc8fb07dab2f1e683a012eba381df8: Status 404 returned error can't find the container with id 37e94bf40be9ceee16e7bf35312a8156acdc8fb07dab2f1e683a012eba381df8 Jul 07 15:21:34.229311 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:34.229267 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-pwpzp" event={"ID":"372030ff-029f-461b-8199-5c810f934692","Type":"ContainerStarted","Data":"cd2ef3c797b58e4f6b1a2f0398d0e9fb73bb5f185223fc426712412b98288e95"} Jul 07 15:21:34.229802 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:34.229366 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-pwpzp" Jul 07 15:21:34.231107 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:34.231066 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-bxk7h" event={"ID":"2cc62d89-fbb0-4e39-95f0-ab676d24f36b","Type":"ContainerStarted","Data":"37e94bf40be9ceee16e7bf35312a8156acdc8fb07dab2f1e683a012eba381df8"} Jul 07 15:21:34.233444 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:34.233397 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m95fv" event={"ID":"0ec1dd40-981d-4ca3-99c0-68380384f986","Type":"ContainerStarted","Data":"b31431939476c8d319b8be46f5c0333b2cba10c0ff3185171b2e0371af2176c8"} Jul 07 15:21:34.233625 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:34.233571 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m95fv" Jul 07 15:21:34.265661 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:34.265598 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-pwpzp" podStartSLOduration=1.82573409 podStartE2EDuration="5.265556571s" totalImagesPullingTime="3.439822481s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:21:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:21:34.265106059 +0000 UTC m=+512.272583524" watchObservedRunningTime="2026-07-07 15:21:34.265556571 +0000 UTC m=+512.273034020" Jul 07 15:21:34.299095 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:34.299025 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m95fv" podStartSLOduration=1.768283263 podStartE2EDuration="7.299004235s" totalImagesPullingTime="5.530720972s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:21:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:21:34.295871523 +0000 UTC m=+512.303348975" watchObservedRunningTime="2026-07-07 15:21:34.299004235 +0000 UTC m=+512.306481686" Jul 07 15:21:35.254526 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:35.254493 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s786f"] Jul 07 15:21:35.259864 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:35.259835 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s786f" Jul 07 15:21:35.262167 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:35.262146 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-gb7r6\"" Jul 07 15:21:35.268445 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:35.268381 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s786f"] Jul 07 15:21:35.316175 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:35.316141 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9j75\" (UniqueName: \"kubernetes.io/projected/071d986a-437b-4772-94f0-ff5856883e62-kube-api-access-w9j75\") pod \"limitador-operator-controller-manager-859d7b78bd-s786f\" (UID: \"071d986a-437b-4772-94f0-ff5856883e62\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s786f" Jul 07 15:21:35.416939 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:35.416899 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9j75\" (UniqueName: \"kubernetes.io/projected/071d986a-437b-4772-94f0-ff5856883e62-kube-api-access-w9j75\") pod \"limitador-operator-controller-manager-859d7b78bd-s786f\" (UID: \"071d986a-437b-4772-94f0-ff5856883e62\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s786f" Jul 07 15:21:35.426057 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:35.426029 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9j75\" (UniqueName: \"kubernetes.io/projected/071d986a-437b-4772-94f0-ff5856883e62-kube-api-access-w9j75\") pod \"limitador-operator-controller-manager-859d7b78bd-s786f\" (UID: \"071d986a-437b-4772-94f0-ff5856883e62\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s786f" Jul 07 15:21:35.575330 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:35.575282 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s786f" Jul 07 15:21:35.969297 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:35.969274 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s786f"] Jul 07 15:21:35.970536 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:21:35.970511 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod071d986a_437b_4772_94f0_ff5856883e62.slice/crio-94186b195fc523a729810f9f312639661ad241079bd2f8b5d4c89c002508f887 WatchSource:0}: Error finding container 94186b195fc523a729810f9f312639661ad241079bd2f8b5d4c89c002508f887: Status 404 returned error can't find the container with id 94186b195fc523a729810f9f312639661ad241079bd2f8b5d4c89c002508f887 Jul 07 15:21:36.248053 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:36.247944 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-bxk7h" event={"ID":"2cc62d89-fbb0-4e39-95f0-ab676d24f36b","Type":"ContainerStarted","Data":"0bbb34d389f1b31850f04b9450751a26e36170e1fa44e229baecc25c7b0a15cc"} Jul 07 15:21:36.248217 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:36.248097 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-bxk7h" Jul 07 15:21:36.249180 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:36.249155 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s786f" event={"ID":"071d986a-437b-4772-94f0-ff5856883e62","Type":"ContainerStarted","Data":"94186b195fc523a729810f9f312639661ad241079bd2f8b5d4c89c002508f887"} Jul 07 15:21:36.268554 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:36.268499 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-bxk7h" podStartSLOduration=1.950803635 podStartE2EDuration="4.268484055s" totalImagesPullingTime="2.31768042s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:21:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:21:36.267621707 +0000 UTC m=+514.275099158" watchObservedRunningTime="2026-07-07 15:21:36.268484055 +0000 UTC m=+514.275961510" Jul 07 15:21:36.631098 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:36.631062 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-s4f5c"] Jul 07 15:21:36.633866 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:36.633843 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s4f5c" Jul 07 15:21:36.642484 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:36.642460 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 07 15:21:36.670536 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:36.670508 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 07 15:21:36.670536 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:36.670522 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-z8hxh\"" Jul 07 15:21:36.673134 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:36.673111 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-s4f5c"] Jul 07 15:21:36.730043 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:36.730009 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/36a474cd-2744-486e-82fe-3a75834a5658-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-s4f5c\" (UID: \"36a474cd-2744-486e-82fe-3a75834a5658\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s4f5c" Jul 07 15:21:36.730215 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:36.730068 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sn76f\" (UniqueName: \"kubernetes.io/projected/36a474cd-2744-486e-82fe-3a75834a5658-kube-api-access-sn76f\") pod \"kuadrant-console-plugin-66b5774985-s4f5c\" (UID: \"36a474cd-2744-486e-82fe-3a75834a5658\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s4f5c" Jul 07 15:21:36.730215 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:36.730144 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/36a474cd-2744-486e-82fe-3a75834a5658-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-s4f5c\" (UID: \"36a474cd-2744-486e-82fe-3a75834a5658\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s4f5c" Jul 07 15:21:36.831722 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:36.831683 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/36a474cd-2744-486e-82fe-3a75834a5658-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-s4f5c\" (UID: \"36a474cd-2744-486e-82fe-3a75834a5658\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s4f5c" Jul 07 15:21:36.831910 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:36.831817 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/36a474cd-2744-486e-82fe-3a75834a5658-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-s4f5c\" (UID: \"36a474cd-2744-486e-82fe-3a75834a5658\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s4f5c" Jul 07 15:21:36.831910 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:36.831864 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sn76f\" (UniqueName: \"kubernetes.io/projected/36a474cd-2744-486e-82fe-3a75834a5658-kube-api-access-sn76f\") pod \"kuadrant-console-plugin-66b5774985-s4f5c\" (UID: \"36a474cd-2744-486e-82fe-3a75834a5658\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s4f5c" Jul 07 15:21:36.832691 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:36.832661 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/36a474cd-2744-486e-82fe-3a75834a5658-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-s4f5c\" (UID: \"36a474cd-2744-486e-82fe-3a75834a5658\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s4f5c" Jul 07 15:21:36.835367 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:36.835343 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/36a474cd-2744-486e-82fe-3a75834a5658-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-s4f5c\" (UID: \"36a474cd-2744-486e-82fe-3a75834a5658\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s4f5c" Jul 07 15:21:36.840427 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:36.840406 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sn76f\" (UniqueName: \"kubernetes.io/projected/36a474cd-2744-486e-82fe-3a75834a5658-kube-api-access-sn76f\") pod \"kuadrant-console-plugin-66b5774985-s4f5c\" (UID: \"36a474cd-2744-486e-82fe-3a75834a5658\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s4f5c" Jul 07 15:21:36.945240 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:36.945157 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s4f5c" Jul 07 15:21:37.116802 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:37.116777 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-s4f5c"] Jul 07 15:21:37.119212 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:21:37.119185 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod36a474cd_2744_486e_82fe_3a75834a5658.slice/crio-00bb1758e6b7c4582a6607b43ae95a9a1f7b1525c0f1ecedeb49e4fdc1c27493 WatchSource:0}: Error finding container 00bb1758e6b7c4582a6607b43ae95a9a1f7b1525c0f1ecedeb49e4fdc1c27493: Status 404 returned error can't find the container with id 00bb1758e6b7c4582a6607b43ae95a9a1f7b1525c0f1ecedeb49e4fdc1c27493 Jul 07 15:21:37.258067 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:37.257971 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s4f5c" event={"ID":"36a474cd-2744-486e-82fe-3a75834a5658","Type":"ContainerStarted","Data":"00bb1758e6b7c4582a6607b43ae95a9a1f7b1525c0f1ecedeb49e4fdc1c27493"} Jul 07 15:21:38.268254 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:38.268160 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s786f" event={"ID":"071d986a-437b-4772-94f0-ff5856883e62","Type":"ContainerStarted","Data":"73deedab7dca3ee68ba57d7c2b30a1776b98b489b3c3c66244d5be86710b2d3f"} Jul 07 15:21:38.268777 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:38.268306 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s786f" Jul 07 15:21:38.291801 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:38.291744 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s786f" podStartSLOduration=1.394444298 podStartE2EDuration="3.291727883s" totalImagesPullingTime="1.897283585s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:21:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:21:38.28877239 +0000 UTC m=+516.296249853" watchObservedRunningTime="2026-07-07 15:21:38.291727883 +0000 UTC m=+516.299205333" Jul 07 15:21:45.242727 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:45.242696 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-pwpzp" Jul 07 15:21:45.243266 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:45.242870 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m95fv" Jul 07 15:21:47.201173 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:47.201132 2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m95fv"] Jul 07 15:21:47.201692 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:47.201392 2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m95fv" podUID="0ec1dd40-981d-4ca3-99c0-68380384f986" containerName="manager" containerID="cri-o://b31431939476c8d319b8be46f5c0333b2cba10c0ff3185171b2e0371af2176c8" gracePeriod=2 Jul 07 15:21:47.207840 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:47.207813 2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m95fv"] Jul 07 15:21:47.234523 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:47.234488 2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s786f"] Jul 07 15:21:47.235193 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:47.234961 2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s786f" podUID="071d986a-437b-4772-94f0-ff5856883e62" containerName="manager" containerID="cri-o://73deedab7dca3ee68ba57d7c2b30a1776b98b489b3c3c66244d5be86710b2d3f" gracePeriod=2 Jul 07 15:21:47.237019 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:47.236996 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s786f" Jul 07 15:21:47.237257 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:21:47.237230 2549 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-m95fv\" is forbidden: User \"system:node:ip-10-0-138-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-11.ec2.internal' and this object" podUID="0ec1dd40-981d-4ca3-99c0-68380384f986" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m95fv" Jul 07 15:21:47.251765 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:47.251700 2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s786f"] Jul 07 15:21:47.252497 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:21:47.252458 2549 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-m95fv\" is forbidden: User \"system:node:ip-10-0-138-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-11.ec2.internal' and this object" podUID="0ec1dd40-981d-4ca3-99c0-68380384f986" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m95fv" Jul 07 15:21:47.254171 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:21:47.254137 2549 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-s786f\" is forbidden: User \"system:node:ip-10-0-138-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-11.ec2.internal' and this object" podUID="071d986a-437b-4772-94f0-ff5856883e62" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s786f" Jul 07 15:21:47.255481 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:47.255454 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6wphg"] Jul 07 15:21:47.256268 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:47.256246 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ec1dd40-981d-4ca3-99c0-68380384f986" containerName="manager" Jul 07 15:21:47.256985 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:47.256966 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ec1dd40-981d-4ca3-99c0-68380384f986" containerName="manager" Jul 07 15:21:47.257109 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:47.257097 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="071d986a-437b-4772-94f0-ff5856883e62" containerName="manager" Jul 07 15:21:47.257197 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:47.257187 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="071d986a-437b-4772-94f0-ff5856883e62" containerName="manager" Jul 07 15:21:47.257419 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:47.257405 2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="071d986a-437b-4772-94f0-ff5856883e62" containerName="manager" Jul 07 15:21:47.257531 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:47.257520 2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ec1dd40-981d-4ca3-99c0-68380384f986" containerName="manager" Jul 07 15:21:47.265543 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:47.264387 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-bxk7h" Jul 07 15:21:47.265543 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:47.264481 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6wphg" Jul 07 15:21:47.266890 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:47.266866 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v75tr"] Jul 07 15:21:47.268196 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:21:47.268162 2549 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-m95fv\" is forbidden: User \"system:node:ip-10-0-138-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-11.ec2.internal' and this object" podUID="0ec1dd40-981d-4ca3-99c0-68380384f986" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m95fv" Jul 07 15:21:47.270883 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:47.270864 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v75tr" Jul 07 15:21:47.292010 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:47.291976 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6wphg"] Jul 07 15:21:47.297192 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:47.297166 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v75tr"] Jul 07 15:21:47.338221 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:47.338184 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0f40b280-40bb-4b17-8c93-6193e8c63c78-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6wphg\" (UID: \"0f40b280-40bb-4b17-8c93-6193e8c63c78\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6wphg" Jul 07 15:21:47.338411 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:47.338234 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86gmj\" (UniqueName: \"kubernetes.io/projected/0f40b280-40bb-4b17-8c93-6193e8c63c78-kube-api-access-86gmj\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6wphg\" (UID: \"0f40b280-40bb-4b17-8c93-6193e8c63c78\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6wphg" Jul 07 15:21:47.338478 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:47.338410 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rlph\" (UniqueName: \"kubernetes.io/projected/6d747b0b-7182-4179-b7db-7864774c0b76-kube-api-access-2rlph\") pod \"limitador-operator-controller-manager-859d7b78bd-v75tr\" (UID: \"6d747b0b-7182-4179-b7db-7864774c0b76\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v75tr" Jul 07 15:21:47.347393 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:21:47.347351 2549 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-s786f\" is forbidden: User \"system:node:ip-10-0-138-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-11.ec2.internal' and this object" podUID="071d986a-437b-4772-94f0-ff5856883e62" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s786f" Jul 07 15:21:47.349067 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:21:47.349037 2549 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-s786f\" is forbidden: User \"system:node:ip-10-0-138-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-11.ec2.internal' and this object" podUID="071d986a-437b-4772-94f0-ff5856883e62" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s786f" Jul 07 15:21:47.375225 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:21:47.375179 2549 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-m95fv\" is forbidden: User \"system:node:ip-10-0-138-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-11.ec2.internal' and this object" podUID="0ec1dd40-981d-4ca3-99c0-68380384f986" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m95fv" Jul 07 15:21:47.410679 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:47.410645 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jqg5w"] Jul 07 15:21:47.416546 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:47.416521 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jqg5w" Jul 07 15:21:47.418854 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:21:47.418817 2549 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-s786f\" is forbidden: User \"system:node:ip-10-0-138-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-11.ec2.internal' and this object" podUID="071d986a-437b-4772-94f0-ff5856883e62" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s786f" Jul 07 15:21:47.432932 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:47.432903 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jqg5w"] Jul 07 15:21:47.439886 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:47.439855 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0f40b280-40bb-4b17-8c93-6193e8c63c78-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6wphg\" (UID: \"0f40b280-40bb-4b17-8c93-6193e8c63c78\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6wphg" Jul 07 15:21:47.440032 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:47.439906 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86gmj\" (UniqueName: \"kubernetes.io/projected/0f40b280-40bb-4b17-8c93-6193e8c63c78-kube-api-access-86gmj\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6wphg\" (UID: \"0f40b280-40bb-4b17-8c93-6193e8c63c78\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6wphg" Jul 07 15:21:47.440032 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:47.440008 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rlph\" (UniqueName: \"kubernetes.io/projected/6d747b0b-7182-4179-b7db-7864774c0b76-kube-api-access-2rlph\") pod \"limitador-operator-controller-manager-859d7b78bd-v75tr\" (UID: \"6d747b0b-7182-4179-b7db-7864774c0b76\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v75tr" Jul 07 15:21:47.440359 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:47.440258 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0f40b280-40bb-4b17-8c93-6193e8c63c78-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6wphg\" (UID: \"0f40b280-40bb-4b17-8c93-6193e8c63c78\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6wphg" Jul 07 15:21:47.448722 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:21:47.448691 2549 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-m95fv\" is forbidden: User \"system:node:ip-10-0-138-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-11.ec2.internal' and this object" podUID="0ec1dd40-981d-4ca3-99c0-68380384f986" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m95fv" Jul 07 15:21:47.453279 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:47.453217 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rlph\" (UniqueName: \"kubernetes.io/projected/6d747b0b-7182-4179-b7db-7864774c0b76-kube-api-access-2rlph\") pod \"limitador-operator-controller-manager-859d7b78bd-v75tr\" (UID: \"6d747b0b-7182-4179-b7db-7864774c0b76\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v75tr" Jul 07 15:21:47.453398 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:47.453312 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-86gmj\" (UniqueName: \"kubernetes.io/projected/0f40b280-40bb-4b17-8c93-6193e8c63c78-kube-api-access-86gmj\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6wphg\" (UID: \"0f40b280-40bb-4b17-8c93-6193e8c63c78\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6wphg" Jul 07 15:21:47.540746 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:47.540700 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b1ca7077-8a18-444c-9b3e-12d2a9bbb5df-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-jqg5w\" (UID: \"b1ca7077-8a18-444c-9b3e-12d2a9bbb5df\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jqg5w" Jul 07 15:21:47.540911 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:47.540847 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rckhp\" (UniqueName: \"kubernetes.io/projected/b1ca7077-8a18-444c-9b3e-12d2a9bbb5df-kube-api-access-rckhp\") pod \"kuadrant-operator-controller-manager-86878c769f-jqg5w\" (UID: \"b1ca7077-8a18-444c-9b3e-12d2a9bbb5df\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jqg5w" Jul 07 15:21:47.622297 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:47.622253 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6wphg" Jul 07 15:21:47.629444 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:47.629411 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v75tr" Jul 07 15:21:47.641856 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:47.641824 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b1ca7077-8a18-444c-9b3e-12d2a9bbb5df-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-jqg5w\" (UID: \"b1ca7077-8a18-444c-9b3e-12d2a9bbb5df\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jqg5w" Jul 07 15:21:47.642021 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:47.641931 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rckhp\" (UniqueName: \"kubernetes.io/projected/b1ca7077-8a18-444c-9b3e-12d2a9bbb5df-kube-api-access-rckhp\") pod \"kuadrant-operator-controller-manager-86878c769f-jqg5w\" (UID: \"b1ca7077-8a18-444c-9b3e-12d2a9bbb5df\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jqg5w" Jul 07 15:21:47.642280 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:47.642257 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b1ca7077-8a18-444c-9b3e-12d2a9bbb5df-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-jqg5w\" (UID: \"b1ca7077-8a18-444c-9b3e-12d2a9bbb5df\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jqg5w" Jul 07 15:21:47.658367 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:47.658341 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rckhp\" (UniqueName: \"kubernetes.io/projected/b1ca7077-8a18-444c-9b3e-12d2a9bbb5df-kube-api-access-rckhp\") pod \"kuadrant-operator-controller-manager-86878c769f-jqg5w\" (UID: \"b1ca7077-8a18-444c-9b3e-12d2a9bbb5df\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jqg5w" Jul 07 15:21:47.728854 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:21:47.728770 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jqg5w" Jul 07 15:22:00.425442 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:00.425038 2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m95fv" Jul 07 15:22:00.427360 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:22:00.427309 2549 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-m95fv\" is forbidden: User \"system:node:ip-10-0-138-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-11.ec2.internal' and this object" podUID="0ec1dd40-981d-4ca3-99c0-68380384f986" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m95fv" Jul 07 15:22:00.433402 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:00.432912 2549 generic.go:356] "Generic (PLEG): container finished" podID="0ec1dd40-981d-4ca3-99c0-68380384f986" containerID="b31431939476c8d319b8be46f5c0333b2cba10c0ff3185171b2e0371af2176c8" exitCode=0 Jul 07 15:22:00.433402 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:00.433039 2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m95fv" Jul 07 15:22:00.433402 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:00.433187 2549 scope.go:117] "RemoveContainer" containerID="b31431939476c8d319b8be46f5c0333b2cba10c0ff3185171b2e0371af2176c8" Jul 07 15:22:00.437613 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:22:00.437375 2549 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-m95fv\" is forbidden: User \"system:node:ip-10-0-138-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-11.ec2.internal' and this object" podUID="0ec1dd40-981d-4ca3-99c0-68380384f986" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m95fv" Jul 07 15:22:00.438726 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:00.438447 2549 generic.go:356] "Generic (PLEG): container finished" podID="071d986a-437b-4772-94f0-ff5856883e62" containerID="73deedab7dca3ee68ba57d7c2b30a1776b98b489b3c3c66244d5be86710b2d3f" exitCode=0 Jul 07 15:22:00.450316 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:00.450247 2549 scope.go:117] "RemoveContainer" containerID="b31431939476c8d319b8be46f5c0333b2cba10c0ff3185171b2e0371af2176c8" Jul 07 15:22:00.450880 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:22:00.450662 2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b31431939476c8d319b8be46f5c0333b2cba10c0ff3185171b2e0371af2176c8\": container with ID starting with b31431939476c8d319b8be46f5c0333b2cba10c0ff3185171b2e0371af2176c8 not found: ID does not exist" containerID="b31431939476c8d319b8be46f5c0333b2cba10c0ff3185171b2e0371af2176c8" Jul 07 15:22:00.450880 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:00.450796 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b31431939476c8d319b8be46f5c0333b2cba10c0ff3185171b2e0371af2176c8"} err="failed to get container status \"b31431939476c8d319b8be46f5c0333b2cba10c0ff3185171b2e0371af2176c8\": rpc error: code = NotFound desc = could not find container \"b31431939476c8d319b8be46f5c0333b2cba10c0ff3185171b2e0371af2176c8\": container with ID starting with b31431939476c8d319b8be46f5c0333b2cba10c0ff3185171b2e0371af2176c8 not found: ID does not exist" Jul 07 15:22:00.472598 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:00.472506 2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s786f" Jul 07 15:22:00.474671 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:22:00.474619 2549 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-s786f\" is forbidden: User \"system:node:ip-10-0-138-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-11.ec2.internal' and this object" podUID="071d986a-437b-4772-94f0-ff5856883e62" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s786f" Jul 07 15:22:00.476450 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:22:00.476419 2549 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-m95fv\" is forbidden: User \"system:node:ip-10-0-138-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-11.ec2.internal' and this object" podUID="0ec1dd40-981d-4ca3-99c0-68380384f986" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m95fv" Jul 07 15:22:00.490683 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:00.490517 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jqg5w"] Jul 07 15:22:00.490683 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:22:00.490638 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1ca7077_8a18_444c_9b3e_12d2a9bbb5df.slice/crio-053a20038febe9ffcfb607d1ff6335f7c0a097c818c7ad477e2226f03142e555 WatchSource:0}: Error finding container 053a20038febe9ffcfb607d1ff6335f7c0a097c818c7ad477e2226f03142e555: Status 404 returned error can't find the container with id 053a20038febe9ffcfb607d1ff6335f7c0a097c818c7ad477e2226f03142e555 Jul 07 15:22:00.509889 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:00.509863 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v75tr"] Jul 07 15:22:00.512088 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:22:00.512061 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d747b0b_7182_4179_b7db_7864774c0b76.slice/crio-38f21202248d54c40391c6aae9c537a7e6fd228da398b3e7ff63783a202513ce WatchSource:0}: Error finding container 38f21202248d54c40391c6aae9c537a7e6fd228da398b3e7ff63783a202513ce: Status 404 returned error can't find the container with id 38f21202248d54c40391c6aae9c537a7e6fd228da398b3e7ff63783a202513ce Jul 07 15:22:00.545432 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:00.545402 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6wphg"] Jul 07 15:22:00.547049 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:22:00.547024 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f40b280_40bb_4b17_8c93_6193e8c63c78.slice/crio-f9c164ad05e445e257b6ec3228009acf84c007b916cfaae7add0b67f15669e94 WatchSource:0}: Error finding container f9c164ad05e445e257b6ec3228009acf84c007b916cfaae7add0b67f15669e94: Status 404 returned error can't find the container with id f9c164ad05e445e257b6ec3228009acf84c007b916cfaae7add0b67f15669e94 Jul 07 15:22:00.578398 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:00.578020 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w9j75\" (UniqueName: \"kubernetes.io/projected/071d986a-437b-4772-94f0-ff5856883e62-kube-api-access-w9j75\") pod \"071d986a-437b-4772-94f0-ff5856883e62\" (UID: \"071d986a-437b-4772-94f0-ff5856883e62\") " Jul 07 15:22:00.578398 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:00.578114 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rknfp\" (UniqueName: \"kubernetes.io/projected/0ec1dd40-981d-4ca3-99c0-68380384f986-kube-api-access-rknfp\") pod \"0ec1dd40-981d-4ca3-99c0-68380384f986\" (UID: \"0ec1dd40-981d-4ca3-99c0-68380384f986\") " Jul 07 15:22:00.578398 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:00.578184 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0ec1dd40-981d-4ca3-99c0-68380384f986-extensions-socket-volume\") pod \"0ec1dd40-981d-4ca3-99c0-68380384f986\" (UID: \"0ec1dd40-981d-4ca3-99c0-68380384f986\") " Jul 07 15:22:00.579050 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:00.579019 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ec1dd40-981d-4ca3-99c0-68380384f986-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "0ec1dd40-981d-4ca3-99c0-68380384f986" (UID: "0ec1dd40-981d-4ca3-99c0-68380384f986"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:22:00.580312 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:00.580278 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/071d986a-437b-4772-94f0-ff5856883e62-kube-api-access-w9j75" (OuterVolumeSpecName: "kube-api-access-w9j75") pod "071d986a-437b-4772-94f0-ff5856883e62" (UID: "071d986a-437b-4772-94f0-ff5856883e62"). InnerVolumeSpecName "kube-api-access-w9j75". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:22:00.580399 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:00.580351 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ec1dd40-981d-4ca3-99c0-68380384f986-kube-api-access-rknfp" (OuterVolumeSpecName: "kube-api-access-rknfp") pod "0ec1dd40-981d-4ca3-99c0-68380384f986" (UID: "0ec1dd40-981d-4ca3-99c0-68380384f986"). InnerVolumeSpecName "kube-api-access-rknfp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:22:00.680224 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:00.680173 2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rknfp\" (UniqueName: \"kubernetes.io/projected/0ec1dd40-981d-4ca3-99c0-68380384f986-kube-api-access-rknfp\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:22:00.680224 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:00.680226 2549 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0ec1dd40-981d-4ca3-99c0-68380384f986-extensions-socket-volume\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:22:00.680487 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:00.680242 2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w9j75\" (UniqueName: \"kubernetes.io/projected/071d986a-437b-4772-94f0-ff5856883e62-kube-api-access-w9j75\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:22:00.743500 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:22:00.743468 2549 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-m95fv\" is forbidden: User \"system:node:ip-10-0-138-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-11.ec2.internal' and this object" podUID="0ec1dd40-981d-4ca3-99c0-68380384f986" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m95fv" Jul 07 15:22:01.445632 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:01.445569 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s4f5c" event={"ID":"36a474cd-2744-486e-82fe-3a75834a5658","Type":"ContainerStarted","Data":"c404414208252e4bdc113f4b4a503a69024f869b245d5500027235b53f2c2274"} Jul 07 15:22:01.447127 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:01.447101 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v75tr" event={"ID":"6d747b0b-7182-4179-b7db-7864774c0b76","Type":"ContainerStarted","Data":"38d779e3f04995ebc04bf835c729dee06d43fcdf6789d7c46fee107fdfa116a0"} Jul 07 15:22:01.447254 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:01.447158 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v75tr" Jul 07 15:22:01.447254 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:01.447185 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v75tr" event={"ID":"6d747b0b-7182-4179-b7db-7864774c0b76","Type":"ContainerStarted","Data":"38f21202248d54c40391c6aae9c537a7e6fd228da398b3e7ff63783a202513ce"} Jul 07 15:22:01.449321 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:01.449298 2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s786f" Jul 07 15:22:01.449420 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:01.449346 2549 scope.go:117] "RemoveContainer" containerID="73deedab7dca3ee68ba57d7c2b30a1776b98b489b3c3c66244d5be86710b2d3f" Jul 07 15:22:01.451066 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:01.451045 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6wphg" event={"ID":"0f40b280-40bb-4b17-8c93-6193e8c63c78","Type":"ContainerStarted","Data":"3f8fb4b4fa274aa444232202853fcec4d8852eb65ed96144c79dd257f3c65c4b"} Jul 07 15:22:01.451161 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:01.451094 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6wphg" event={"ID":"0f40b280-40bb-4b17-8c93-6193e8c63c78","Type":"ContainerStarted","Data":"f9c164ad05e445e257b6ec3228009acf84c007b916cfaae7add0b67f15669e94"} Jul 07 15:22:01.451161 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:01.451121 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6wphg" Jul 07 15:22:01.453051 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:01.453022 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jqg5w" event={"ID":"b1ca7077-8a18-444c-9b3e-12d2a9bbb5df","Type":"ContainerStarted","Data":"4aafd8de82bea52ee102e2f0ccaae56842b56170f0bd8944c06b7156a6966e34"} Jul 07 15:22:01.453157 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:01.453074 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jqg5w" event={"ID":"b1ca7077-8a18-444c-9b3e-12d2a9bbb5df","Type":"ContainerStarted","Data":"053a20038febe9ffcfb607d1ff6335f7c0a097c818c7ad477e2226f03142e555"} Jul 07 15:22:01.453229 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:01.453172 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jqg5w" Jul 07 15:22:01.464431 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:01.464385 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s4f5c" podStartSLOduration=2.266495139 podStartE2EDuration="25.464371605s" totalImagesPullingTime="23.197876466s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:21:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:22:01.463420655 +0000 UTC m=+539.470898117" watchObservedRunningTime="2026-07-07 15:22:01.464371605 +0000 UTC m=+539.471849056" Jul 07 15:22:01.465341 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:22:01.465313 2549 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-m95fv\" is forbidden: User \"system:node:ip-10-0-138-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-11.ec2.internal' and this object" podUID="0ec1dd40-981d-4ca3-99c0-68380384f986" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m95fv" Jul 07 15:22:01.467784 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:22:01.467758 2549 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-s786f\" is forbidden: User \"system:node:ip-10-0-138-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-11.ec2.internal' and this object" podUID="071d986a-437b-4772-94f0-ff5856883e62" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s786f" Jul 07 15:22:01.499838 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:01.499731 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v75tr" podStartSLOduration=14.499705852 podStartE2EDuration="14.499705852s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:21:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:22:01.496024977 +0000 UTC m=+539.503502428" watchObservedRunningTime="2026-07-07 15:22:01.499705852 +0000 UTC m=+539.507183304" Jul 07 15:22:01.522963 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:01.522894 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jqg5w" podStartSLOduration=14.522872881 podStartE2EDuration="14.522872881s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:21:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:22:01.521448429 +0000 UTC m=+539.528925880" watchObservedRunningTime="2026-07-07 15:22:01.522872881 +0000 UTC m=+539.530350333" Jul 07 15:22:01.523143 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:22:01.523056 2549 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-m95fv\" is forbidden: User \"system:node:ip-10-0-138-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-11.ec2.internal' and this object" podUID="0ec1dd40-981d-4ca3-99c0-68380384f986" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m95fv" Jul 07 15:22:01.546570 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:01.546522 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6wphg" podStartSLOduration=14.546508497 podStartE2EDuration="14.546508497s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:21:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:22:01.544092259 +0000 UTC m=+539.551569710" watchObservedRunningTime="2026-07-07 15:22:01.546508497 +0000 UTC m=+539.553985947" Jul 07 15:22:02.533766 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:22:02.533733 2549 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-m95fv\" is forbidden: User \"system:node:ip-10-0-138-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-11.ec2.internal' and this object" podUID="0ec1dd40-981d-4ca3-99c0-68380384f986" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m95fv" Jul 07 15:22:02.535468 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:22:02.535444 2549 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-s786f\" is forbidden: User \"system:node:ip-10-0-138-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-11.ec2.internal' and this object" podUID="071d986a-437b-4772-94f0-ff5856883e62" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s786f" Jul 07 15:22:02.535468 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:02.535451 2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="071d986a-437b-4772-94f0-ff5856883e62" path="/var/lib/kubelet/pods/071d986a-437b-4772-94f0-ff5856883e62/volumes" Jul 07 15:22:02.535906 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:02.535893 2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ec1dd40-981d-4ca3-99c0-68380384f986" path="/var/lib/kubelet/pods/0ec1dd40-981d-4ca3-99c0-68380384f986/volumes" Jul 07 15:22:12.460709 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:12.460669 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jqg5w" Jul 07 15:22:12.461380 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:12.460766 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v75tr" Jul 07 15:22:12.461380 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:12.460939 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6wphg" Jul 07 15:22:12.540774 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:12.540744 2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6wphg"] Jul 07 15:22:12.540989 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:12.540965 2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6wphg" podUID="0f40b280-40bb-4b17-8c93-6193e8c63c78" containerName="manager" containerID="cri-o://3f8fb4b4fa274aa444232202853fcec4d8852eb65ed96144c79dd257f3c65c4b" gracePeriod=10 Jul 07 15:22:13.406360 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:13.406322 2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6wphg" Jul 07 15:22:13.512569 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:13.512475 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-86gmj\" (UniqueName: \"kubernetes.io/projected/0f40b280-40bb-4b17-8c93-6193e8c63c78-kube-api-access-86gmj\") pod \"0f40b280-40bb-4b17-8c93-6193e8c63c78\" (UID: \"0f40b280-40bb-4b17-8c93-6193e8c63c78\") " Jul 07 15:22:13.512569 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:13.512563 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0f40b280-40bb-4b17-8c93-6193e8c63c78-extensions-socket-volume\") pod \"0f40b280-40bb-4b17-8c93-6193e8c63c78\" (UID: \"0f40b280-40bb-4b17-8c93-6193e8c63c78\") " Jul 07 15:22:13.513262 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:13.513217 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f40b280-40bb-4b17-8c93-6193e8c63c78-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "0f40b280-40bb-4b17-8c93-6193e8c63c78" (UID: "0f40b280-40bb-4b17-8c93-6193e8c63c78"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:22:13.515772 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:13.515733 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f40b280-40bb-4b17-8c93-6193e8c63c78-kube-api-access-86gmj" (OuterVolumeSpecName: "kube-api-access-86gmj") pod "0f40b280-40bb-4b17-8c93-6193e8c63c78" (UID: "0f40b280-40bb-4b17-8c93-6193e8c63c78"). InnerVolumeSpecName "kube-api-access-86gmj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:22:13.527053 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:13.526998 2549 generic.go:356] "Generic (PLEG): container finished" podID="0f40b280-40bb-4b17-8c93-6193e8c63c78" containerID="3f8fb4b4fa274aa444232202853fcec4d8852eb65ed96144c79dd257f3c65c4b" exitCode=0 Jul 07 15:22:13.527053 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:13.527036 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6wphg" event={"ID":"0f40b280-40bb-4b17-8c93-6193e8c63c78","Type":"ContainerDied","Data":"3f8fb4b4fa274aa444232202853fcec4d8852eb65ed96144c79dd257f3c65c4b"} Jul 07 15:22:13.527259 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:13.527061 2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6wphg" Jul 07 15:22:13.527259 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:13.527105 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6wphg" event={"ID":"0f40b280-40bb-4b17-8c93-6193e8c63c78","Type":"ContainerDied","Data":"f9c164ad05e445e257b6ec3228009acf84c007b916cfaae7add0b67f15669e94"} Jul 07 15:22:13.527259 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:13.527137 2549 scope.go:117] "RemoveContainer" containerID="3f8fb4b4fa274aa444232202853fcec4d8852eb65ed96144c79dd257f3c65c4b" Jul 07 15:22:13.536876 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:13.536858 2549 scope.go:117] "RemoveContainer" containerID="3f8fb4b4fa274aa444232202853fcec4d8852eb65ed96144c79dd257f3c65c4b" Jul 07 15:22:13.537125 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:22:13.537104 2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f8fb4b4fa274aa444232202853fcec4d8852eb65ed96144c79dd257f3c65c4b\": container with ID starting with 3f8fb4b4fa274aa444232202853fcec4d8852eb65ed96144c79dd257f3c65c4b not found: ID does not exist" containerID="3f8fb4b4fa274aa444232202853fcec4d8852eb65ed96144c79dd257f3c65c4b" Jul 07 15:22:13.537192 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:13.537132 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f8fb4b4fa274aa444232202853fcec4d8852eb65ed96144c79dd257f3c65c4b"} err="failed to get container status \"3f8fb4b4fa274aa444232202853fcec4d8852eb65ed96144c79dd257f3c65c4b\": rpc error: code = NotFound desc = could not find container \"3f8fb4b4fa274aa444232202853fcec4d8852eb65ed96144c79dd257f3c65c4b\": container with ID starting with 3f8fb4b4fa274aa444232202853fcec4d8852eb65ed96144c79dd257f3c65c4b not found: ID does not exist" Jul 07 15:22:13.572711 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:13.572678 2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6wphg"] Jul 07 15:22:13.586330 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:13.586303 2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6wphg"] Jul 07 15:22:13.614398 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:13.614364 2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-86gmj\" (UniqueName: \"kubernetes.io/projected/0f40b280-40bb-4b17-8c93-6193e8c63c78-kube-api-access-86gmj\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:22:13.614398 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:13.614393 2549 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0f40b280-40bb-4b17-8c93-6193e8c63c78-extensions-socket-volume\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:22:14.538955 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:14.538925 2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f40b280-40bb-4b17-8c93-6193e8c63c78" path="/var/lib/kubelet/pods/0f40b280-40bb-4b17-8c93-6193e8c63c78/volumes" Jul 07 15:22:30.769141 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:30.769109 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:22:30.769635 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:30.769553 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f40b280-40bb-4b17-8c93-6193e8c63c78" containerName="manager" Jul 07 15:22:30.769635 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:30.769564 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f40b280-40bb-4b17-8c93-6193e8c63c78" containerName="manager" Jul 07 15:22:30.769722 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:30.769664 2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f40b280-40bb-4b17-8c93-6193e8c63c78" containerName="manager" Jul 07 15:22:30.814805 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:30.814769 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:22:30.815060 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:30.814927 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-xf9m5" Jul 07 15:22:30.817419 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:30.817386 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 07 15:22:30.818330 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:30.818311 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:22:30.974092 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:30.974059 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8hk5\" (UniqueName: \"kubernetes.io/projected/1b5f5855-5c60-45a1-a6a1-bffae8500df6-kube-api-access-t8hk5\") pod \"limitador-limitador-54d6d988d5-xf9m5\" (UID: \"1b5f5855-5c60-45a1-a6a1-bffae8500df6\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xf9m5" Jul 07 15:22:30.974282 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:30.974119 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1b5f5855-5c60-45a1-a6a1-bffae8500df6-config-file\") pod \"limitador-limitador-54d6d988d5-xf9m5\" (UID: \"1b5f5855-5c60-45a1-a6a1-bffae8500df6\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xf9m5" Jul 07 15:22:31.075490 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:31.075453 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8hk5\" (UniqueName: \"kubernetes.io/projected/1b5f5855-5c60-45a1-a6a1-bffae8500df6-kube-api-access-t8hk5\") pod \"limitador-limitador-54d6d988d5-xf9m5\" (UID: \"1b5f5855-5c60-45a1-a6a1-bffae8500df6\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xf9m5" Jul 07 15:22:31.075704 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:31.075518 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1b5f5855-5c60-45a1-a6a1-bffae8500df6-config-file\") pod \"limitador-limitador-54d6d988d5-xf9m5\" (UID: \"1b5f5855-5c60-45a1-a6a1-bffae8500df6\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xf9m5" Jul 07 15:22:31.076160 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:31.076133 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1b5f5855-5c60-45a1-a6a1-bffae8500df6-config-file\") pod \"limitador-limitador-54d6d988d5-xf9m5\" (UID: \"1b5f5855-5c60-45a1-a6a1-bffae8500df6\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xf9m5" Jul 07 15:22:31.089699 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:31.089670 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8hk5\" (UniqueName: \"kubernetes.io/projected/1b5f5855-5c60-45a1-a6a1-bffae8500df6-kube-api-access-t8hk5\") pod \"limitador-limitador-54d6d988d5-xf9m5\" (UID: \"1b5f5855-5c60-45a1-a6a1-bffae8500df6\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xf9m5" Jul 07 15:22:31.127160 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:31.127123 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-xf9m5" Jul 07 15:22:31.274633 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:31.274606 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:22:31.276375 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:22:31.276349 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b5f5855_5c60_45a1_a6a1_bffae8500df6.slice/crio-dcec32111f0c9257db694c8414df8e27cbe766f465497f9f2a6d3c8a5f5853c9 WatchSource:0}: Error finding container dcec32111f0c9257db694c8414df8e27cbe766f465497f9f2a6d3c8a5f5853c9: Status 404 returned error can't find the container with id dcec32111f0c9257db694c8414df8e27cbe766f465497f9f2a6d3c8a5f5853c9 Jul 07 15:22:31.638120 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:31.638089 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-xf9m5" event={"ID":"1b5f5855-5c60-45a1-a6a1-bffae8500df6","Type":"ContainerStarted","Data":"dcec32111f0c9257db694c8414df8e27cbe766f465497f9f2a6d3c8a5f5853c9"} Jul 07 15:22:34.661660 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:34.661619 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-xf9m5" event={"ID":"1b5f5855-5c60-45a1-a6a1-bffae8500df6","Type":"ContainerStarted","Data":"7dece3cd8391ae4a842c451d3f837b1e3b2c55d36f7a674939648e9a12d317e4"} Jul 07 15:22:34.662130 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:34.661808 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-xf9m5" Jul 07 15:22:34.682553 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:34.682500 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-xf9m5" podStartSLOduration=1.775724345 podStartE2EDuration="4.682483909s" totalImagesPullingTime="2.906759564s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:22:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:22:34.679306541 +0000 UTC m=+572.686784013" watchObservedRunningTime="2026-07-07 15:22:34.682483909 +0000 UTC m=+572.689961360" Jul 07 15:22:45.668855 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:22:45.668827 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-xf9m5" Jul 07 15:23:00.511201 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:23:00.511166 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-d8b5bddb4-x4b8v"] Jul 07 15:23:00.513891 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:23:00.513875 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d8b5bddb4-x4b8v" Jul 07 15:23:00.516254 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:23:00.516237 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-h86w4\"" Jul 07 15:23:00.526151 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:23:00.526122 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-d8b5bddb4-x4b8v"] Jul 07 15:23:00.542941 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:23:00.542914 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2q2qk\" (UniqueName: \"kubernetes.io/projected/07966beb-b248-4443-979d-b4ed50c9efb0-kube-api-access-2q2qk\") pod \"authorino-d8b5bddb4-x4b8v\" (UID: \"07966beb-b248-4443-979d-b4ed50c9efb0\") " pod="kuadrant-system/authorino-d8b5bddb4-x4b8v" Jul 07 15:23:00.644116 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:23:00.644074 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2q2qk\" (UniqueName: \"kubernetes.io/projected/07966beb-b248-4443-979d-b4ed50c9efb0-kube-api-access-2q2qk\") pod \"authorino-d8b5bddb4-x4b8v\" (UID: \"07966beb-b248-4443-979d-b4ed50c9efb0\") " pod="kuadrant-system/authorino-d8b5bddb4-x4b8v" Jul 07 15:23:00.661338 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:23:00.661312 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2q2qk\" (UniqueName: \"kubernetes.io/projected/07966beb-b248-4443-979d-b4ed50c9efb0-kube-api-access-2q2qk\") pod \"authorino-d8b5bddb4-x4b8v\" (UID: \"07966beb-b248-4443-979d-b4ed50c9efb0\") " pod="kuadrant-system/authorino-d8b5bddb4-x4b8v" Jul 07 15:23:00.782519 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:23:00.782427 2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-d8b5bddb4-x4b8v"] Jul 07 15:23:00.782757 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:23:00.782744 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d8b5bddb4-x4b8v" Jul 07 15:23:00.829654 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:23:00.829625 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5bcc995576-kh6kz"] Jul 07 15:23:00.832392 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:23:00.832373 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5bcc995576-kh6kz" Jul 07 15:23:00.835014 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:23:00.834872 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 07 15:23:00.847723 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:23:00.847665 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5bcc995576-kh6kz"] Jul 07 15:23:00.930315 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:23:00.930286 2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-d8b5bddb4-x4b8v"] Jul 07 15:23:00.932092 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:23:00.932067 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod07966beb_b248_4443_979d_b4ed50c9efb0.slice/crio-59c202679c13f9ef52d647ab292bf6874a3d94ab68adad6e82175c6d582ab79b WatchSource:0}: Error finding container 59c202679c13f9ef52d647ab292bf6874a3d94ab68adad6e82175c6d582ab79b: Status 404 returned error can't find the container with id 59c202679c13f9ef52d647ab292bf6874a3d94ab68adad6e82175c6d582ab79b Jul 07 15:23:00.947024 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:23:00.947001 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c24zq\" (UniqueName: \"kubernetes.io/projected/a99b599d-4fd7-46e1-a7ee-f904762dfb46-kube-api-access-c24zq\") pod \"authorino-5bcc995576-kh6kz\" (UID: \"a99b599d-4fd7-46e1-a7ee-f904762dfb46\") " pod="kuadrant-system/authorino-5bcc995576-kh6kz" Jul 07 15:23:00.947150 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:23:00.947076 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a99b599d-4fd7-46e1-a7ee-f904762dfb46-tls-cert\") pod \"authorino-5bcc995576-kh6kz\" (UID: \"a99b599d-4fd7-46e1-a7ee-f904762dfb46\") " pod="kuadrant-system/authorino-5bcc995576-kh6kz" Jul 07 15:23:01.047842 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:23:01.047760 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c24zq\" (UniqueName: \"kubernetes.io/projected/a99b599d-4fd7-46e1-a7ee-f904762dfb46-kube-api-access-c24zq\") pod \"authorino-5bcc995576-kh6kz\" (UID: \"a99b599d-4fd7-46e1-a7ee-f904762dfb46\") " pod="kuadrant-system/authorino-5bcc995576-kh6kz" Jul 07 15:23:01.047842 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:23:01.047840 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a99b599d-4fd7-46e1-a7ee-f904762dfb46-tls-cert\") pod \"authorino-5bcc995576-kh6kz\" (UID: \"a99b599d-4fd7-46e1-a7ee-f904762dfb46\") " pod="kuadrant-system/authorino-5bcc995576-kh6kz" Jul 07 15:23:01.050299 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:23:01.050276 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a99b599d-4fd7-46e1-a7ee-f904762dfb46-tls-cert\") pod \"authorino-5bcc995576-kh6kz\" (UID: \"a99b599d-4fd7-46e1-a7ee-f904762dfb46\") " pod="kuadrant-system/authorino-5bcc995576-kh6kz" Jul 07 15:23:01.059664 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:23:01.059637 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c24zq\" (UniqueName: \"kubernetes.io/projected/a99b599d-4fd7-46e1-a7ee-f904762dfb46-kube-api-access-c24zq\") pod \"authorino-5bcc995576-kh6kz\" (UID: \"a99b599d-4fd7-46e1-a7ee-f904762dfb46\") " pod="kuadrant-system/authorino-5bcc995576-kh6kz" Jul 07 15:23:01.148594 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:23:01.148553 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5bcc995576-kh6kz" Jul 07 15:23:01.292381 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:23:01.292350 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5bcc995576-kh6kz"] Jul 07 15:23:01.295480 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:23:01.295456 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda99b599d_4fd7_46e1_a7ee_f904762dfb46.slice/crio-ba61db98d13951c77277d6766101198ea561381fcd9a9a9d14b8c18b9d1fc6c2 WatchSource:0}: Error finding container ba61db98d13951c77277d6766101198ea561381fcd9a9a9d14b8c18b9d1fc6c2: Status 404 returned error can't find the container with id ba61db98d13951c77277d6766101198ea561381fcd9a9a9d14b8c18b9d1fc6c2 Jul 07 15:23:01.827056 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:23:01.827012 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d8b5bddb4-x4b8v" event={"ID":"07966beb-b248-4443-979d-b4ed50c9efb0","Type":"ContainerStarted","Data":"59c202679c13f9ef52d647ab292bf6874a3d94ab68adad6e82175c6d582ab79b"} Jul 07 15:23:01.828572 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:23:01.828546 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5bcc995576-kh6kz" event={"ID":"a99b599d-4fd7-46e1-a7ee-f904762dfb46","Type":"ContainerStarted","Data":"ba61db98d13951c77277d6766101198ea561381fcd9a9a9d14b8c18b9d1fc6c2"} Jul 07 15:23:02.442118 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:23:02.442086 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kb4ml_3301f6ef-6671-4755-9047-16959836232c/console-operator/2.log" Jul 07 15:23:02.442722 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:23:02.442703 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kb4ml_3301f6ef-6671-4755-9047-16959836232c/console-operator/2.log" Jul 07 15:23:04.862267 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:23:04.862229 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d8b5bddb4-x4b8v" event={"ID":"07966beb-b248-4443-979d-b4ed50c9efb0","Type":"ContainerStarted","Data":"95d5636a1d8b2895b222129712d1a60a1e6b899364fd3d59dc6f33d02eccfd83"} Jul 07 15:23:04.862872 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:23:04.862252 2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-d8b5bddb4-x4b8v" podUID="07966beb-b248-4443-979d-b4ed50c9efb0" containerName="authorino" containerID="cri-o://95d5636a1d8b2895b222129712d1a60a1e6b899364fd3d59dc6f33d02eccfd83" gracePeriod=30 Jul 07 15:23:04.864174 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:23:04.864144 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5bcc995576-kh6kz" event={"ID":"a99b599d-4fd7-46e1-a7ee-f904762dfb46","Type":"ContainerStarted","Data":"da517974fae02fd5dc5cd97018f3c41b000504e74a8f5ea01d5d1e9aea5e1001"} Jul 07 15:23:04.881682 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:23:04.881635 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-d8b5bddb4-x4b8v" podStartSLOduration=1.135453121 podStartE2EDuration="4.881619842s" totalImagesPullingTime="3.746166721s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:23:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:23:04.878886978 +0000 UTC m=+602.886364429" watchObservedRunningTime="2026-07-07 15:23:04.881619842 +0000 UTC m=+602.889097292" Jul 07 15:23:04.899781 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:23:04.899729 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5bcc995576-kh6kz" podStartSLOduration=1.506185691 podStartE2EDuration="4.899718096s" totalImagesPullingTime="3.393532405s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:23:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:23:04.899101192 +0000 UTC m=+602.906578643" watchObservedRunningTime="2026-07-07 15:23:04.899718096 +0000 UTC m=+602.907195547" Jul 07 15:23:05.115630 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:23:05.115602 2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d8b5bddb4-x4b8v" Jul 07 15:23:05.193714 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:23:05.193682 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2q2qk\" (UniqueName: \"kubernetes.io/projected/07966beb-b248-4443-979d-b4ed50c9efb0-kube-api-access-2q2qk\") pod \"07966beb-b248-4443-979d-b4ed50c9efb0\" (UID: \"07966beb-b248-4443-979d-b4ed50c9efb0\") " Jul 07 15:23:05.195946 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:23:05.195923 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07966beb-b248-4443-979d-b4ed50c9efb0-kube-api-access-2q2qk" (OuterVolumeSpecName: "kube-api-access-2q2qk") pod "07966beb-b248-4443-979d-b4ed50c9efb0" (UID: "07966beb-b248-4443-979d-b4ed50c9efb0"). InnerVolumeSpecName "kube-api-access-2q2qk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:23:05.295160 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:23:05.295045 2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2q2qk\" (UniqueName: \"kubernetes.io/projected/07966beb-b248-4443-979d-b4ed50c9efb0-kube-api-access-2q2qk\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:23:05.871692 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:23:05.871651 2549 generic.go:356] "Generic (PLEG): container finished" podID="07966beb-b248-4443-979d-b4ed50c9efb0" containerID="95d5636a1d8b2895b222129712d1a60a1e6b899364fd3d59dc6f33d02eccfd83" exitCode=0 Jul 07 15:23:05.871692 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:23:05.871694 2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d8b5bddb4-x4b8v" Jul 07 15:23:05.872238 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:23:05.871744 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d8b5bddb4-x4b8v" event={"ID":"07966beb-b248-4443-979d-b4ed50c9efb0","Type":"ContainerDied","Data":"95d5636a1d8b2895b222129712d1a60a1e6b899364fd3d59dc6f33d02eccfd83"} Jul 07 15:23:05.872238 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:23:05.871789 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d8b5bddb4-x4b8v" event={"ID":"07966beb-b248-4443-979d-b4ed50c9efb0","Type":"ContainerDied","Data":"59c202679c13f9ef52d647ab292bf6874a3d94ab68adad6e82175c6d582ab79b"} Jul 07 15:23:05.872238 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:23:05.871812 2549 scope.go:117] "RemoveContainer" containerID="95d5636a1d8b2895b222129712d1a60a1e6b899364fd3d59dc6f33d02eccfd83" Jul 07 15:23:05.881158 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:23:05.881142 2549 scope.go:117] "RemoveContainer" containerID="95d5636a1d8b2895b222129712d1a60a1e6b899364fd3d59dc6f33d02eccfd83" Jul 07 15:23:05.881440 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:23:05.881422 2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95d5636a1d8b2895b222129712d1a60a1e6b899364fd3d59dc6f33d02eccfd83\": container with ID starting with 95d5636a1d8b2895b222129712d1a60a1e6b899364fd3d59dc6f33d02eccfd83 not found: ID does not exist" containerID="95d5636a1d8b2895b222129712d1a60a1e6b899364fd3d59dc6f33d02eccfd83" Jul 07 15:23:05.881505 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:23:05.881452 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95d5636a1d8b2895b222129712d1a60a1e6b899364fd3d59dc6f33d02eccfd83"} err="failed to get container status \"95d5636a1d8b2895b222129712d1a60a1e6b899364fd3d59dc6f33d02eccfd83\": rpc error: code = NotFound desc = could not find container \"95d5636a1d8b2895b222129712d1a60a1e6b899364fd3d59dc6f33d02eccfd83\": container with ID starting with 95d5636a1d8b2895b222129712d1a60a1e6b899364fd3d59dc6f33d02eccfd83 not found: ID does not exist" Jul 07 15:23:05.894903 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:23:05.894878 2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-d8b5bddb4-x4b8v"] Jul 07 15:23:05.898176 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:23:05.898155 2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-d8b5bddb4-x4b8v"] Jul 07 15:23:06.535104 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:23:06.535071 2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07966beb-b248-4443-979d-b4ed50c9efb0" path="/var/lib/kubelet/pods/07966beb-b248-4443-979d-b4ed50c9efb0/volumes" Jul 07 15:23:46.248482 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:23:46.248449 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:24:05.645775 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:24:05.645739 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:24:12.439909 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:24:12.439871 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:24:29.142985 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:24:29.142921 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-mrvqg"] Jul 07 15:24:29.143555 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:24:29.143538 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07966beb-b248-4443-979d-b4ed50c9efb0" containerName="authorino" Jul 07 15:24:29.143632 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:24:29.143557 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="07966beb-b248-4443-979d-b4ed50c9efb0" containerName="authorino" Jul 07 15:24:29.143671 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:24:29.143659 2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="07966beb-b248-4443-979d-b4ed50c9efb0" containerName="authorino" Jul 07 15:24:29.145665 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:24:29.145650 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mrvqg" Jul 07 15:24:29.148685 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:24:29.148662 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 07 15:24:29.148685 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:24:29.148674 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 07 15:24:29.148899 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:24:29.148678 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-6kqtt\"" Jul 07 15:24:29.148899 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:24:29.148662 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 07 15:24:29.160857 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:24:29.160835 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-mrvqg"] Jul 07 15:24:29.195597 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:24:29.195561 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkjhg\" (UniqueName: \"kubernetes.io/projected/565a55e9-c4c2-4f3c-8767-7d4111677f81-kube-api-access-rkjhg\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mrvqg\" (UID: \"565a55e9-c4c2-4f3c-8767-7d4111677f81\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mrvqg" Jul 07 15:24:29.195744 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:24:29.195650 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/565a55e9-c4c2-4f3c-8767-7d4111677f81-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mrvqg\" (UID: \"565a55e9-c4c2-4f3c-8767-7d4111677f81\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mrvqg" Jul 07 15:24:29.195744 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:24:29.195725 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/565a55e9-c4c2-4f3c-8767-7d4111677f81-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mrvqg\" (UID: \"565a55e9-c4c2-4f3c-8767-7d4111677f81\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mrvqg" Jul 07 15:24:29.195819 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:24:29.195760 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/565a55e9-c4c2-4f3c-8767-7d4111677f81-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mrvqg\" (UID: \"565a55e9-c4c2-4f3c-8767-7d4111677f81\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mrvqg" Jul 07 15:24:29.195819 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:24:29.195788 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/565a55e9-c4c2-4f3c-8767-7d4111677f81-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mrvqg\" (UID: \"565a55e9-c4c2-4f3c-8767-7d4111677f81\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mrvqg" Jul 07 15:24:29.195889 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:24:29.195846 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/565a55e9-c4c2-4f3c-8767-7d4111677f81-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mrvqg\" (UID: \"565a55e9-c4c2-4f3c-8767-7d4111677f81\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mrvqg" Jul 07 15:24:29.296891 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:24:29.296854 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/565a55e9-c4c2-4f3c-8767-7d4111677f81-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mrvqg\" (UID: \"565a55e9-c4c2-4f3c-8767-7d4111677f81\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mrvqg" Jul 07 15:24:29.296891 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:24:29.296896 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/565a55e9-c4c2-4f3c-8767-7d4111677f81-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mrvqg\" (UID: \"565a55e9-c4c2-4f3c-8767-7d4111677f81\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mrvqg" Jul 07 15:24:29.297150 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:24:29.297018 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/565a55e9-c4c2-4f3c-8767-7d4111677f81-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mrvqg\" (UID: \"565a55e9-c4c2-4f3c-8767-7d4111677f81\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mrvqg" Jul 07 15:24:29.297150 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:24:29.297066 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/565a55e9-c4c2-4f3c-8767-7d4111677f81-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mrvqg\" (UID: \"565a55e9-c4c2-4f3c-8767-7d4111677f81\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mrvqg" Jul 07 15:24:29.297150 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:24:29.297132 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkjhg\" (UniqueName: \"kubernetes.io/projected/565a55e9-c4c2-4f3c-8767-7d4111677f81-kube-api-access-rkjhg\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mrvqg\" (UID: \"565a55e9-c4c2-4f3c-8767-7d4111677f81\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mrvqg" Jul 07 15:24:29.297316 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:24:29.297165 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/565a55e9-c4c2-4f3c-8767-7d4111677f81-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mrvqg\" (UID: \"565a55e9-c4c2-4f3c-8767-7d4111677f81\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mrvqg" Jul 07 15:24:29.297413 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:24:29.297394 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/565a55e9-c4c2-4f3c-8767-7d4111677f81-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mrvqg\" (UID: \"565a55e9-c4c2-4f3c-8767-7d4111677f81\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mrvqg" Jul 07 15:24:29.297497 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:24:29.297477 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/565a55e9-c4c2-4f3c-8767-7d4111677f81-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mrvqg\" (UID: \"565a55e9-c4c2-4f3c-8767-7d4111677f81\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mrvqg" Jul 07 15:24:29.297558 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:24:29.297521 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/565a55e9-c4c2-4f3c-8767-7d4111677f81-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mrvqg\" (UID: \"565a55e9-c4c2-4f3c-8767-7d4111677f81\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mrvqg" Jul 07 15:24:29.299259 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:24:29.299237 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/565a55e9-c4c2-4f3c-8767-7d4111677f81-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mrvqg\" (UID: \"565a55e9-c4c2-4f3c-8767-7d4111677f81\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mrvqg" Jul 07 15:24:29.299415 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:24:29.299399 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/565a55e9-c4c2-4f3c-8767-7d4111677f81-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mrvqg\" (UID: \"565a55e9-c4c2-4f3c-8767-7d4111677f81\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mrvqg" Jul 07 15:24:29.310754 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:24:29.310731 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkjhg\" (UniqueName: \"kubernetes.io/projected/565a55e9-c4c2-4f3c-8767-7d4111677f81-kube-api-access-rkjhg\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mrvqg\" (UID: \"565a55e9-c4c2-4f3c-8767-7d4111677f81\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mrvqg" Jul 07 15:24:29.465100 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:24:29.465009 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mrvqg" Jul 07 15:24:29.620034 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:24:29.620009 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-mrvqg"] Jul 07 15:24:29.620698 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:24:29.620675 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod565a55e9_c4c2_4f3c_8767_7d4111677f81.slice/crio-a59be7f350cf604c94268b293fa16d8c7d6edca3376a3153128f63240e01cca0 WatchSource:0}: Error finding container a59be7f350cf604c94268b293fa16d8c7d6edca3376a3153128f63240e01cca0: Status 404 returned error can't find the container with id a59be7f350cf604c94268b293fa16d8c7d6edca3376a3153128f63240e01cca0 Jul 07 15:24:29.622034 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:24:29.622014 2549 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 15:24:30.238549 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:24:30.238507 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:24:30.372869 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:24:30.372823 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mrvqg" event={"ID":"565a55e9-c4c2-4f3c-8767-7d4111677f81","Type":"ContainerStarted","Data":"a59be7f350cf604c94268b293fa16d8c7d6edca3376a3153128f63240e01cca0"} Jul 07 15:24:32.387661 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:24:32.387625 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mrvqg" event={"ID":"565a55e9-c4c2-4f3c-8767-7d4111677f81","Type":"ContainerStarted","Data":"2b929185cd9c342eb226b007f362ba07971b39f2b92c904343c4843aa0e683b2"} Jul 07 15:24:32.422103 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:24:32.422047 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mrvqg" podStartSLOduration=1.5234386290000002 podStartE2EDuration="3.422030999s" totalImagesPullingTime="1.89859237s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:24:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:24:32.420962955 +0000 UTC m=+690.428440407" watchObservedRunningTime="2026-07-07 15:24:32.422030999 +0000 UTC m=+690.429508450" Jul 07 15:24:37.351352 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:24:37.351312 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:24:39.465141 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:24:39.465099 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mrvqg" Jul 07 15:24:39.465547 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:24:39.465192 2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mrvqg" Jul 07 15:24:39.478237 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:24:39.478206 2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mrvqg" Jul 07 15:24:40.445530 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:24:40.445503 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mrvqg" Jul 07 15:24:54.436932 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:24:54.436850 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:25:06.440283 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:25:06.440241 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:26:05.256960 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:26:05.256928 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-fdd655b5-86qvh"] Jul 07 15:26:05.260759 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:26:05.260738 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-fdd655b5-86qvh" Jul 07 15:26:05.270735 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:26:05.270711 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-fdd655b5-86qvh"] Jul 07 15:26:05.348784 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:26:05.348741 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/07d18ca8-06d5-4526-9639-bb4db61ff4f6-tls-cert\") pod \"authorino-fdd655b5-86qvh\" (UID: \"07d18ca8-06d5-4526-9639-bb4db61ff4f6\") " pod="kuadrant-system/authorino-fdd655b5-86qvh" Jul 07 15:26:05.348986 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:26:05.348928 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnb74\" (UniqueName: \"kubernetes.io/projected/07d18ca8-06d5-4526-9639-bb4db61ff4f6-kube-api-access-lnb74\") pod \"authorino-fdd655b5-86qvh\" (UID: \"07d18ca8-06d5-4526-9639-bb4db61ff4f6\") " pod="kuadrant-system/authorino-fdd655b5-86qvh" Jul 07 15:26:05.449859 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:26:05.449820 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnb74\" (UniqueName: \"kubernetes.io/projected/07d18ca8-06d5-4526-9639-bb4db61ff4f6-kube-api-access-lnb74\") pod \"authorino-fdd655b5-86qvh\" (UID: \"07d18ca8-06d5-4526-9639-bb4db61ff4f6\") " pod="kuadrant-system/authorino-fdd655b5-86qvh" Jul 07 15:26:05.450063 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:26:05.449866 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/07d18ca8-06d5-4526-9639-bb4db61ff4f6-tls-cert\") pod \"authorino-fdd655b5-86qvh\" (UID: \"07d18ca8-06d5-4526-9639-bb4db61ff4f6\") " pod="kuadrant-system/authorino-fdd655b5-86qvh" Jul 07 15:26:05.452302 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:26:05.452276 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/07d18ca8-06d5-4526-9639-bb4db61ff4f6-tls-cert\") pod \"authorino-fdd655b5-86qvh\" (UID: \"07d18ca8-06d5-4526-9639-bb4db61ff4f6\") " pod="kuadrant-system/authorino-fdd655b5-86qvh" Jul 07 15:26:05.465913 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:26:05.465885 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnb74\" (UniqueName: \"kubernetes.io/projected/07d18ca8-06d5-4526-9639-bb4db61ff4f6-kube-api-access-lnb74\") pod \"authorino-fdd655b5-86qvh\" (UID: \"07d18ca8-06d5-4526-9639-bb4db61ff4f6\") " pod="kuadrant-system/authorino-fdd655b5-86qvh" Jul 07 15:26:05.571042 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:26:05.571008 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-fdd655b5-86qvh" Jul 07 15:26:05.751685 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:26:05.751652 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-fdd655b5-86qvh"] Jul 07 15:26:05.758016 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:26:05.757986 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod07d18ca8_06d5_4526_9639_bb4db61ff4f6.slice/crio-87751a84124574595c4f137b2edefcfdfcbeafbfc01b51d70120100d27d73e03 WatchSource:0}: Error finding container 87751a84124574595c4f137b2edefcfdfcbeafbfc01b51d70120100d27d73e03: Status 404 returned error can't find the container with id 87751a84124574595c4f137b2edefcfdfcbeafbfc01b51d70120100d27d73e03 Jul 07 15:26:05.951234 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:26:05.951116 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-fdd655b5-86qvh" event={"ID":"07d18ca8-06d5-4526-9639-bb4db61ff4f6","Type":"ContainerStarted","Data":"87751a84124574595c4f137b2edefcfdfcbeafbfc01b51d70120100d27d73e03"} Jul 07 15:26:06.960836 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:26:06.960797 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-fdd655b5-86qvh" event={"ID":"07d18ca8-06d5-4526-9639-bb4db61ff4f6","Type":"ContainerStarted","Data":"1885250493ebaf87f977f8854c302bfdf3af8ca7c8c25052bdc6a2eded6da255"} Jul 07 15:26:07.002531 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:26:07.002456 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-fdd655b5-86qvh" podStartSLOduration=1.565698631 podStartE2EDuration="2.002431788s" totalImagesPullingTime="436.733157ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:26:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:26:06.99933191 +0000 UTC m=+785.006809372" watchObservedRunningTime="2026-07-07 15:26:07.002431788 +0000 UTC m=+785.009909239" Jul 07 15:26:07.169355 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:26:07.169319 2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5bcc995576-kh6kz"] Jul 07 15:26:07.169595 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:26:07.169550 2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5bcc995576-kh6kz" podUID="a99b599d-4fd7-46e1-a7ee-f904762dfb46" containerName="authorino" containerID="cri-o://da517974fae02fd5dc5cd97018f3c41b000504e74a8f5ea01d5d1e9aea5e1001" gracePeriod=30 Jul 07 15:26:07.427951 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:26:07.427915 2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5bcc995576-kh6kz" Jul 07 15:26:07.471312 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:26:07.471275 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c24zq\" (UniqueName: \"kubernetes.io/projected/a99b599d-4fd7-46e1-a7ee-f904762dfb46-kube-api-access-c24zq\") pod \"a99b599d-4fd7-46e1-a7ee-f904762dfb46\" (UID: \"a99b599d-4fd7-46e1-a7ee-f904762dfb46\") " Jul 07 15:26:07.471508 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:26:07.471335 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a99b599d-4fd7-46e1-a7ee-f904762dfb46-tls-cert\") pod \"a99b599d-4fd7-46e1-a7ee-f904762dfb46\" (UID: \"a99b599d-4fd7-46e1-a7ee-f904762dfb46\") " Jul 07 15:26:07.474155 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:26:07.474110 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a99b599d-4fd7-46e1-a7ee-f904762dfb46-kube-api-access-c24zq" (OuterVolumeSpecName: "kube-api-access-c24zq") pod "a99b599d-4fd7-46e1-a7ee-f904762dfb46" (UID: "a99b599d-4fd7-46e1-a7ee-f904762dfb46"). InnerVolumeSpecName "kube-api-access-c24zq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:26:07.486781 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:26:07.486751 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a99b599d-4fd7-46e1-a7ee-f904762dfb46-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "a99b599d-4fd7-46e1-a7ee-f904762dfb46" (UID: "a99b599d-4fd7-46e1-a7ee-f904762dfb46"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:26:07.573285 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:26:07.573249 2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c24zq\" (UniqueName: \"kubernetes.io/projected/a99b599d-4fd7-46e1-a7ee-f904762dfb46-kube-api-access-c24zq\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:26:07.573285 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:26:07.573285 2549 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a99b599d-4fd7-46e1-a7ee-f904762dfb46-tls-cert\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:26:07.968302 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:26:07.968219 2549 generic.go:356] "Generic (PLEG): container finished" podID="a99b599d-4fd7-46e1-a7ee-f904762dfb46" containerID="da517974fae02fd5dc5cd97018f3c41b000504e74a8f5ea01d5d1e9aea5e1001" exitCode=0 Jul 07 15:26:07.968302 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:26:07.968255 2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5bcc995576-kh6kz" Jul 07 15:26:07.968794 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:26:07.968296 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5bcc995576-kh6kz" event={"ID":"a99b599d-4fd7-46e1-a7ee-f904762dfb46","Type":"ContainerDied","Data":"da517974fae02fd5dc5cd97018f3c41b000504e74a8f5ea01d5d1e9aea5e1001"} Jul 07 15:26:07.968794 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:26:07.968341 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5bcc995576-kh6kz" event={"ID":"a99b599d-4fd7-46e1-a7ee-f904762dfb46","Type":"ContainerDied","Data":"ba61db98d13951c77277d6766101198ea561381fcd9a9a9d14b8c18b9d1fc6c2"} Jul 07 15:26:07.968794 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:26:07.968359 2549 scope.go:117] "RemoveContainer" containerID="da517974fae02fd5dc5cd97018f3c41b000504e74a8f5ea01d5d1e9aea5e1001" Jul 07 15:26:07.981186 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:26:07.981165 2549 scope.go:117] "RemoveContainer" containerID="da517974fae02fd5dc5cd97018f3c41b000504e74a8f5ea01d5d1e9aea5e1001" Jul 07 15:26:07.981480 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:26:07.981459 2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da517974fae02fd5dc5cd97018f3c41b000504e74a8f5ea01d5d1e9aea5e1001\": container with ID starting with da517974fae02fd5dc5cd97018f3c41b000504e74a8f5ea01d5d1e9aea5e1001 not found: ID does not exist" containerID="da517974fae02fd5dc5cd97018f3c41b000504e74a8f5ea01d5d1e9aea5e1001" Jul 07 15:26:07.981606 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:26:07.981486 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da517974fae02fd5dc5cd97018f3c41b000504e74a8f5ea01d5d1e9aea5e1001"} err="failed to get container status \"da517974fae02fd5dc5cd97018f3c41b000504e74a8f5ea01d5d1e9aea5e1001\": rpc error: code = NotFound desc = could not find container \"da517974fae02fd5dc5cd97018f3c41b000504e74a8f5ea01d5d1e9aea5e1001\": container with ID starting with da517974fae02fd5dc5cd97018f3c41b000504e74a8f5ea01d5d1e9aea5e1001 not found: ID does not exist" Jul 07 15:26:07.995212 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:26:07.995186 2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5bcc995576-kh6kz"] Jul 07 15:26:08.001050 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:26:08.001015 2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5bcc995576-kh6kz"] Jul 07 15:26:08.536322 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:26:08.536291 2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a99b599d-4fd7-46e1-a7ee-f904762dfb46" path="/var/lib/kubelet/pods/a99b599d-4fd7-46e1-a7ee-f904762dfb46/volumes" Jul 07 15:26:34.461914 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:26:34.461871 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:26:44.239227 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:26:44.239195 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:26:55.973079 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:26:55.973036 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:27:04.348516 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:27:04.348480 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:27:15.671905 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:27:15.671866 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:27:43.382385 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:27:43.382339 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:27:57.258654 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:27:57.258544 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:28:02.521776 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:28:02.521747 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kb4ml_3301f6ef-6671-4755-9047-16959836232c/console-operator/2.log" Jul 07 15:28:02.526945 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:28:02.526922 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kb4ml_3301f6ef-6671-4755-9047-16959836232c/console-operator/2.log" Jul 07 15:28:32.943226 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:28:32.943190 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:28:42.241629 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:28:42.241590 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:28:50.780634 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:28:50.780569 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:29:00.145045 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:29:00.145009 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:29:10.856954 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:29:10.856921 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:29:20.152905 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:29:20.152874 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:29:31.346788 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:29:31.346753 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:29:40.752643 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:29:40.752597 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:30:00.144595 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:30:00.144546 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29723970-m8rzm"] Jul 07 15:30:00.145079 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:30:00.145000 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a99b599d-4fd7-46e1-a7ee-f904762dfb46" containerName="authorino" Jul 07 15:30:00.145079 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:30:00.145012 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="a99b599d-4fd7-46e1-a7ee-f904762dfb46" containerName="authorino" Jul 07 15:30:00.145157 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:30:00.145105 2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="a99b599d-4fd7-46e1-a7ee-f904762dfb46" containerName="authorino" Jul 07 15:30:00.148068 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:30:00.148051 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723970-m8rzm" Jul 07 15:30:00.150593 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:30:00.150563 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-2hd94\"" Jul 07 15:30:00.173467 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:30:00.173443 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29723970-m8rzm"] Jul 07 15:30:00.252316 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:30:00.252283 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rxtt\" (UniqueName: \"kubernetes.io/projected/5dc0b063-d8d0-44e6-a341-258a4b41d89f-kube-api-access-8rxtt\") pod \"maas-api-key-cleanup-29723970-m8rzm\" (UID: \"5dc0b063-d8d0-44e6-a341-258a4b41d89f\") " pod="opendatahub/maas-api-key-cleanup-29723970-m8rzm" Jul 07 15:30:00.352792 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:30:00.352753 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rxtt\" (UniqueName: \"kubernetes.io/projected/5dc0b063-d8d0-44e6-a341-258a4b41d89f-kube-api-access-8rxtt\") pod \"maas-api-key-cleanup-29723970-m8rzm\" (UID: \"5dc0b063-d8d0-44e6-a341-258a4b41d89f\") " pod="opendatahub/maas-api-key-cleanup-29723970-m8rzm" Jul 07 15:30:00.364688 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:30:00.364655 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rxtt\" (UniqueName: \"kubernetes.io/projected/5dc0b063-d8d0-44e6-a341-258a4b41d89f-kube-api-access-8rxtt\") pod \"maas-api-key-cleanup-29723970-m8rzm\" (UID: \"5dc0b063-d8d0-44e6-a341-258a4b41d89f\") " pod="opendatahub/maas-api-key-cleanup-29723970-m8rzm" Jul 07 15:30:00.458080 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:30:00.458003 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723970-m8rzm" Jul 07 15:30:00.603380 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:30:00.603354 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29723970-m8rzm"] Jul 07 15:30:00.605446 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:30:00.605416 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dc0b063_d8d0_44e6_a341_258a4b41d89f.slice/crio-41d8640fad4749f1562aa47551d46ac6c70a41f87de10d76a264ca196e377e44 WatchSource:0}: Error finding container 41d8640fad4749f1562aa47551d46ac6c70a41f87de10d76a264ca196e377e44: Status 404 returned error can't find the container with id 41d8640fad4749f1562aa47551d46ac6c70a41f87de10d76a264ca196e377e44 Jul 07 15:30:00.607452 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:30:00.607434 2549 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 15:30:01.389103 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:30:01.389067 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723970-m8rzm" event={"ID":"5dc0b063-d8d0-44e6-a341-258a4b41d89f","Type":"ContainerStarted","Data":"41d8640fad4749f1562aa47551d46ac6c70a41f87de10d76a264ca196e377e44"} Jul 07 15:30:03.834362 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:30:03.834327 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:30:06.423985 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:30:06.423896 2549 generic.go:356] "Generic (PLEG): container finished" podID="5dc0b063-d8d0-44e6-a341-258a4b41d89f" containerID="82d7a354168f53a253c32cc11170d185a0c66beafc08a3845c2be8cccc8e7633" exitCode=0 Jul 07 15:30:06.423985 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:30:06.423968 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723970-m8rzm" event={"ID":"5dc0b063-d8d0-44e6-a341-258a4b41d89f","Type":"ContainerDied","Data":"82d7a354168f53a253c32cc11170d185a0c66beafc08a3845c2be8cccc8e7633"} Jul 07 15:30:07.593461 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:30:07.593428 2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723970-m8rzm" Jul 07 15:30:07.633916 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:30:07.633885 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8rxtt\" (UniqueName: \"kubernetes.io/projected/5dc0b063-d8d0-44e6-a341-258a4b41d89f-kube-api-access-8rxtt\") pod \"5dc0b063-d8d0-44e6-a341-258a4b41d89f\" (UID: \"5dc0b063-d8d0-44e6-a341-258a4b41d89f\") " Jul 07 15:30:07.636217 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:30:07.636179 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5dc0b063-d8d0-44e6-a341-258a4b41d89f-kube-api-access-8rxtt" (OuterVolumeSpecName: "kube-api-access-8rxtt") pod "5dc0b063-d8d0-44e6-a341-258a4b41d89f" (UID: "5dc0b063-d8d0-44e6-a341-258a4b41d89f"). InnerVolumeSpecName "kube-api-access-8rxtt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:30:07.735369 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:30:07.735289 2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8rxtt\" (UniqueName: \"kubernetes.io/projected/5dc0b063-d8d0-44e6-a341-258a4b41d89f-kube-api-access-8rxtt\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:30:08.438820 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:30:08.438789 2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723970-m8rzm" Jul 07 15:30:08.438994 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:30:08.438827 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723970-m8rzm" event={"ID":"5dc0b063-d8d0-44e6-a341-258a4b41d89f","Type":"ContainerDied","Data":"41d8640fad4749f1562aa47551d46ac6c70a41f87de10d76a264ca196e377e44"} Jul 07 15:30:08.438994 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:30:08.438874 2549 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="41d8640fad4749f1562aa47551d46ac6c70a41f87de10d76a264ca196e377e44" Jul 07 15:30:15.548789 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:30:15.548754 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:30:40.564368 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:30:40.564336 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:30:52.649575 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:30:52.649537 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:31:07.355858 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:31:07.355776 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:31:16.773094 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:31:16.773050 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:31:27.682223 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:31:27.682187 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:31:37.344761 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:31:37.344723 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:31:47.264695 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:31:47.264660 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:32:12.047176 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:32:12.047143 2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jqg5w"] Jul 07 15:32:12.047817 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:32:12.047344 2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jqg5w" podUID="b1ca7077-8a18-444c-9b3e-12d2a9bbb5df" containerName="manager" containerID="cri-o://4aafd8de82bea52ee102e2f0ccaae56842b56170f0bd8944c06b7156a6966e34" gracePeriod=10 Jul 07 15:32:16.005903 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:32:16.005875 2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jqg5w" Jul 07 15:32:16.102989 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:32:16.102957 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b1ca7077-8a18-444c-9b3e-12d2a9bbb5df-extensions-socket-volume\") pod \"b1ca7077-8a18-444c-9b3e-12d2a9bbb5df\" (UID: \"b1ca7077-8a18-444c-9b3e-12d2a9bbb5df\") " Jul 07 15:32:16.103164 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:32:16.103092 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rckhp\" (UniqueName: \"kubernetes.io/projected/b1ca7077-8a18-444c-9b3e-12d2a9bbb5df-kube-api-access-rckhp\") pod \"b1ca7077-8a18-444c-9b3e-12d2a9bbb5df\" (UID: \"b1ca7077-8a18-444c-9b3e-12d2a9bbb5df\") " Jul 07 15:32:16.103316 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:32:16.103292 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1ca7077-8a18-444c-9b3e-12d2a9bbb5df-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b1ca7077-8a18-444c-9b3e-12d2a9bbb5df" (UID: "b1ca7077-8a18-444c-9b3e-12d2a9bbb5df"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:32:16.103420 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:32:16.103406 2549 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b1ca7077-8a18-444c-9b3e-12d2a9bbb5df-extensions-socket-volume\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:32:16.105291 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:32:16.105260 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1ca7077-8a18-444c-9b3e-12d2a9bbb5df-kube-api-access-rckhp" (OuterVolumeSpecName: "kube-api-access-rckhp") pod "b1ca7077-8a18-444c-9b3e-12d2a9bbb5df" (UID: "b1ca7077-8a18-444c-9b3e-12d2a9bbb5df"). InnerVolumeSpecName "kube-api-access-rckhp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:32:16.204495 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:32:16.204456 2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rckhp\" (UniqueName: \"kubernetes.io/projected/b1ca7077-8a18-444c-9b3e-12d2a9bbb5df-kube-api-access-rckhp\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:32:16.238641 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:32:16.238609 2549 generic.go:356] "Generic (PLEG): container finished" podID="b1ca7077-8a18-444c-9b3e-12d2a9bbb5df" containerID="4aafd8de82bea52ee102e2f0ccaae56842b56170f0bd8944c06b7156a6966e34" exitCode=0 Jul 07 15:32:16.238841 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:32:16.238659 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jqg5w" event={"ID":"b1ca7077-8a18-444c-9b3e-12d2a9bbb5df","Type":"ContainerDied","Data":"4aafd8de82bea52ee102e2f0ccaae56842b56170f0bd8944c06b7156a6966e34"} Jul 07 15:32:16.238841 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:32:16.238711 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jqg5w" event={"ID":"b1ca7077-8a18-444c-9b3e-12d2a9bbb5df","Type":"ContainerDied","Data":"053a20038febe9ffcfb607d1ff6335f7c0a097c818c7ad477e2226f03142e555"} Jul 07 15:32:16.238841 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:32:16.238723 2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jqg5w" Jul 07 15:32:16.238841 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:32:16.238740 2549 scope.go:117] "RemoveContainer" containerID="4aafd8de82bea52ee102e2f0ccaae56842b56170f0bd8944c06b7156a6966e34" Jul 07 15:32:16.257410 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:32:16.257390 2549 scope.go:117] "RemoveContainer" containerID="4aafd8de82bea52ee102e2f0ccaae56842b56170f0bd8944c06b7156a6966e34" Jul 07 15:32:16.257730 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:32:16.257701 2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4aafd8de82bea52ee102e2f0ccaae56842b56170f0bd8944c06b7156a6966e34\": container with ID starting with 4aafd8de82bea52ee102e2f0ccaae56842b56170f0bd8944c06b7156a6966e34 not found: ID does not exist" containerID="4aafd8de82bea52ee102e2f0ccaae56842b56170f0bd8944c06b7156a6966e34" Jul 07 15:32:16.257791 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:32:16.257730 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4aafd8de82bea52ee102e2f0ccaae56842b56170f0bd8944c06b7156a6966e34"} err="failed to get container status \"4aafd8de82bea52ee102e2f0ccaae56842b56170f0bd8944c06b7156a6966e34\": rpc error: code = NotFound desc = could not find container \"4aafd8de82bea52ee102e2f0ccaae56842b56170f0bd8944c06b7156a6966e34\": container with ID starting with 4aafd8de82bea52ee102e2f0ccaae56842b56170f0bd8944c06b7156a6966e34 not found: ID does not exist" Jul 07 15:32:16.270969 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:32:16.270947 2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jqg5w"] Jul 07 15:32:16.274541 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:32:16.274520 2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jqg5w"] Jul 07 15:32:16.535628 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:32:16.535508 2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1ca7077-8a18-444c-9b3e-12d2a9bbb5df" path="/var/lib/kubelet/pods/b1ca7077-8a18-444c-9b3e-12d2a9bbb5df/volumes" Jul 07 15:33:02.565448 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:33:02.565416 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kb4ml_3301f6ef-6671-4755-9047-16959836232c/console-operator/2.log" Jul 07 15:33:02.570995 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:33:02.570973 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kb4ml_3301f6ef-6671-4755-9047-16959836232c/console-operator/2.log" Jul 07 15:33:13.236897 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:33:13.236860 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7ksmp"] Jul 07 15:33:13.237306 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:33:13.237293 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1ca7077-8a18-444c-9b3e-12d2a9bbb5df" containerName="manager" Jul 07 15:33:13.237306 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:33:13.237307 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1ca7077-8a18-444c-9b3e-12d2a9bbb5df" containerName="manager" Jul 07 15:33:13.237385 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:33:13.237325 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5dc0b063-d8d0-44e6-a341-258a4b41d89f" containerName="cleanup" Jul 07 15:33:13.237385 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:33:13.237331 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="5dc0b063-d8d0-44e6-a341-258a4b41d89f" containerName="cleanup" Jul 07 15:33:13.237449 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:33:13.237418 2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="5dc0b063-d8d0-44e6-a341-258a4b41d89f" containerName="cleanup" Jul 07 15:33:13.237449 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:33:13.237429 2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1ca7077-8a18-444c-9b3e-12d2a9bbb5df" containerName="manager" Jul 07 15:33:13.240985 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:33:13.240969 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7ksmp" Jul 07 15:33:13.244752 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:33:13.244729 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-wpnqz\"" Jul 07 15:33:13.266818 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:33:13.266791 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7ksmp"] Jul 07 15:33:13.340263 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:33:13.340235 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/db31a5e5-1e8c-4309-90f6-8debeb83bc31-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7ksmp\" (UID: \"db31a5e5-1e8c-4309-90f6-8debeb83bc31\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7ksmp" Jul 07 15:33:13.340419 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:33:13.340316 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pf2kx\" (UniqueName: \"kubernetes.io/projected/db31a5e5-1e8c-4309-90f6-8debeb83bc31-kube-api-access-pf2kx\") pod \"kuadrant-operator-controller-manager-86878c769f-7ksmp\" (UID: \"db31a5e5-1e8c-4309-90f6-8debeb83bc31\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7ksmp" Jul 07 15:33:13.440941 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:33:13.440909 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pf2kx\" (UniqueName: \"kubernetes.io/projected/db31a5e5-1e8c-4309-90f6-8debeb83bc31-kube-api-access-pf2kx\") pod \"kuadrant-operator-controller-manager-86878c769f-7ksmp\" (UID: \"db31a5e5-1e8c-4309-90f6-8debeb83bc31\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7ksmp" Jul 07 15:33:13.441094 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:33:13.440982 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/db31a5e5-1e8c-4309-90f6-8debeb83bc31-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7ksmp\" (UID: \"db31a5e5-1e8c-4309-90f6-8debeb83bc31\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7ksmp" Jul 07 15:33:13.441363 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:33:13.441345 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/db31a5e5-1e8c-4309-90f6-8debeb83bc31-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7ksmp\" (UID: \"db31a5e5-1e8c-4309-90f6-8debeb83bc31\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7ksmp" Jul 07 15:33:13.449278 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:33:13.449248 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pf2kx\" (UniqueName: \"kubernetes.io/projected/db31a5e5-1e8c-4309-90f6-8debeb83bc31-kube-api-access-pf2kx\") pod \"kuadrant-operator-controller-manager-86878c769f-7ksmp\" (UID: \"db31a5e5-1e8c-4309-90f6-8debeb83bc31\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7ksmp" Jul 07 15:33:13.550925 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:33:13.550848 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7ksmp" Jul 07 15:33:13.694855 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:33:13.694822 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7ksmp"] Jul 07 15:33:13.695940 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:33:13.695919 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb31a5e5_1e8c_4309_90f6_8debeb83bc31.slice/crio-ca561c19657b9b8c5719cf4799494848350bbec19227a9b5b2c896157d7ec38c WatchSource:0}: Error finding container ca561c19657b9b8c5719cf4799494848350bbec19227a9b5b2c896157d7ec38c: Status 404 returned error can't find the container with id ca561c19657b9b8c5719cf4799494848350bbec19227a9b5b2c896157d7ec38c Jul 07 15:33:14.582408 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:33:14.582371 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7ksmp" event={"ID":"db31a5e5-1e8c-4309-90f6-8debeb83bc31","Type":"ContainerStarted","Data":"d27e952b19c20ebb4b3304a6d17fd129a2fae956e328e921ffcc13327d4a7a0c"} Jul 07 15:33:14.582817 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:33:14.582434 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7ksmp" Jul 07 15:33:14.582817 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:33:14.582458 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7ksmp" event={"ID":"db31a5e5-1e8c-4309-90f6-8debeb83bc31","Type":"ContainerStarted","Data":"ca561c19657b9b8c5719cf4799494848350bbec19227a9b5b2c896157d7ec38c"} Jul 07 15:33:14.609363 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:33:14.609316 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7ksmp" podStartSLOduration=1.609303008 podStartE2EDuration="1.609303008s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:33:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:33:14.605408111 +0000 UTC m=+1212.612885572" watchObservedRunningTime="2026-07-07 15:33:14.609303008 +0000 UTC m=+1212.616780459" Jul 07 15:33:25.591307 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:33:25.591279 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7ksmp" Jul 07 15:33:34.244622 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:33:34.244563 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:33:43.040777 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:33:43.040742 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:33:55.348215 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:33:55.348131 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:34:06.953272 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:34:06.953236 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:34:16.586089 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:34:16.586057 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:34:28.140303 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:34:28.140265 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:34:37.445525 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:34:37.445481 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:34:48.742008 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:34:48.741974 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:34:59.642419 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:34:59.642383 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:35:07.156112 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:35:07.156076 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:35:16.352771 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:35:16.352733 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:35:22.755535 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:35:22.755491 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:35:45.548260 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:35:45.548227 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:35:55.055820 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:35:55.055783 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:36:04.652023 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:36:04.651988 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:36:15.544209 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:36:15.544174 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:36:33.052945 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:36:33.052914 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:36:44.335925 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:36:44.335894 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:36:53.740812 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:36:53.740775 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:37:03.146951 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:37:03.146914 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:37:13.041201 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:37:13.041161 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:37:22.545125 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:37:22.545090 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:37:35.162715 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:37:35.162674 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:37:44.557818 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:37:44.557780 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:37:53.906006 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:37:53.905967 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:38:02.604912 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:38:02.604879 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kb4ml_3301f6ef-6671-4755-9047-16959836232c/console-operator/2.log" Jul 07 15:38:02.614166 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:38:02.614139 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kb4ml_3301f6ef-6671-4755-9047-16959836232c/console-operator/2.log" Jul 07 15:38:05.474066 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:38:05.474034 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:38:18.142807 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:38:18.142771 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:38:26.892701 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:38:26.892610 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:38:39.351447 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:38:39.351413 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:38:51.755262 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:38:51.755223 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:39:04.949004 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:39:04.948971 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:39:21.381083 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:39:21.381042 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:39:29.851051 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:39:29.851015 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:39:53.949943 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:39:53.949904 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:40:02.938189 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:40:02.938157 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:41:27.554368 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:41:27.554287 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:41:35.253722 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:41:35.253689 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:41:43.142718 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:41:43.142685 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:41:50.859457 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:41:50.859422 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:42:01.754386 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:42:01.754352 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:42:35.149035 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:42:35.148995 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:42:46.086984 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:42:46.086945 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:43:02.646898 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:43:02.646867 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kb4ml_3301f6ef-6671-4755-9047-16959836232c/console-operator/2.log" Jul 07 15:43:02.658391 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:43:02.658365 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kb4ml_3301f6ef-6671-4755-9047-16959836232c/console-operator/2.log" Jul 07 15:43:14.449098 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:43:14.449060 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:43:28.047453 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:43:28.047420 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:44:07.539770 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:44:07.539741 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:44:22.671243 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:44:22.671209 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:44:31.951753 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:44:31.951718 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:45:00.146428 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:45:00.146392 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29723985-lzwl4"] Jul 07 15:45:00.150343 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:45:00.150326 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723985-lzwl4" Jul 07 15:45:00.152886 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:45:00.152864 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-2hd94\"" Jul 07 15:45:00.157973 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:45:00.157933 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gphtf\" (UniqueName: \"kubernetes.io/projected/15069e2e-354d-4d4e-a7e6-d5be9afc6def-kube-api-access-gphtf\") pod \"maas-api-key-cleanup-29723985-lzwl4\" (UID: \"15069e2e-354d-4d4e-a7e6-d5be9afc6def\") " pod="opendatahub/maas-api-key-cleanup-29723985-lzwl4" Jul 07 15:45:00.160127 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:45:00.160104 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29723985-lzwl4"] Jul 07 15:45:00.258527 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:45:00.258492 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gphtf\" (UniqueName: \"kubernetes.io/projected/15069e2e-354d-4d4e-a7e6-d5be9afc6def-kube-api-access-gphtf\") pod \"maas-api-key-cleanup-29723985-lzwl4\" (UID: \"15069e2e-354d-4d4e-a7e6-d5be9afc6def\") " pod="opendatahub/maas-api-key-cleanup-29723985-lzwl4" Jul 07 15:45:00.269936 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:45:00.269908 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gphtf\" (UniqueName: \"kubernetes.io/projected/15069e2e-354d-4d4e-a7e6-d5be9afc6def-kube-api-access-gphtf\") pod \"maas-api-key-cleanup-29723985-lzwl4\" (UID: \"15069e2e-354d-4d4e-a7e6-d5be9afc6def\") " pod="opendatahub/maas-api-key-cleanup-29723985-lzwl4" Jul 07 15:45:00.461509 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:45:00.461424 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723985-lzwl4" Jul 07 15:45:00.599326 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:45:00.597599 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29723985-lzwl4"] Jul 07 15:45:00.602928 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:45:00.602897 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15069e2e_354d_4d4e_a7e6_d5be9afc6def.slice/crio-2734ec2e4c6fcd03a6aa3c2e61e9f292ac0c03e7166677dd76a2221731e931fc WatchSource:0}: Error finding container 2734ec2e4c6fcd03a6aa3c2e61e9f292ac0c03e7166677dd76a2221731e931fc: Status 404 returned error can't find the container with id 2734ec2e4c6fcd03a6aa3c2e61e9f292ac0c03e7166677dd76a2221731e931fc Jul 07 15:45:00.928217 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:45:00.928185 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723985-lzwl4" event={"ID":"15069e2e-354d-4d4e-a7e6-d5be9afc6def","Type":"ContainerStarted","Data":"dfc5fa16d9d65b45c9def2b706a6b96e3e457cff64e663231bf2bbb81fe3ea1a"} Jul 07 15:45:00.928391 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:45:00.928239 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723985-lzwl4" event={"ID":"15069e2e-354d-4d4e-a7e6-d5be9afc6def","Type":"ContainerStarted","Data":"2734ec2e4c6fcd03a6aa3c2e61e9f292ac0c03e7166677dd76a2221731e931fc"} Jul 07 15:45:00.943818 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:45:00.943761 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29723985-lzwl4" podStartSLOduration=0.94374113 podStartE2EDuration="943.74113ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:45:00.943503852 +0000 UTC m=+1918.950981303" watchObservedRunningTime="2026-07-07 15:45:00.94374113 +0000 UTC m=+1918.951218575" Jul 07 15:45:01.935455 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:45:01.935416 2549 generic.go:356] "Generic (PLEG): container finished" podID="15069e2e-354d-4d4e-a7e6-d5be9afc6def" containerID="dfc5fa16d9d65b45c9def2b706a6b96e3e457cff64e663231bf2bbb81fe3ea1a" exitCode=0 Jul 07 15:45:01.935902 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:45:01.935503 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723985-lzwl4" event={"ID":"15069e2e-354d-4d4e-a7e6-d5be9afc6def","Type":"ContainerDied","Data":"dfc5fa16d9d65b45c9def2b706a6b96e3e457cff64e663231bf2bbb81fe3ea1a"} Jul 07 15:45:03.098724 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:45:03.098697 2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723985-lzwl4" Jul 07 15:45:03.186022 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:45:03.185990 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gphtf\" (UniqueName: \"kubernetes.io/projected/15069e2e-354d-4d4e-a7e6-d5be9afc6def-kube-api-access-gphtf\") pod \"15069e2e-354d-4d4e-a7e6-d5be9afc6def\" (UID: \"15069e2e-354d-4d4e-a7e6-d5be9afc6def\") " Jul 07 15:45:03.188060 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:45:03.188026 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15069e2e-354d-4d4e-a7e6-d5be9afc6def-kube-api-access-gphtf" (OuterVolumeSpecName: "kube-api-access-gphtf") pod "15069e2e-354d-4d4e-a7e6-d5be9afc6def" (UID: "15069e2e-354d-4d4e-a7e6-d5be9afc6def"). InnerVolumeSpecName "kube-api-access-gphtf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:45:03.287183 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:45:03.287091 2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gphtf\" (UniqueName: \"kubernetes.io/projected/15069e2e-354d-4d4e-a7e6-d5be9afc6def-kube-api-access-gphtf\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:45:03.953784 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:45:03.953747 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723985-lzwl4" event={"ID":"15069e2e-354d-4d4e-a7e6-d5be9afc6def","Type":"ContainerDied","Data":"2734ec2e4c6fcd03a6aa3c2e61e9f292ac0c03e7166677dd76a2221731e931fc"} Jul 07 15:45:03.953968 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:45:03.953802 2549 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2734ec2e4c6fcd03a6aa3c2e61e9f292ac0c03e7166677dd76a2221731e931fc" Jul 07 15:45:03.953968 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:45:03.953762 2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723985-lzwl4" Jul 07 15:45:18.838458 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:45:18.838423 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:45:29.158075 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:45:29.158040 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:45:42.142612 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:45:42.142565 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:46:11.332076 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:11.331991 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:46:19.565014 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:19.564957 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-85746cdfd6-nv87t"] Jul 07 15:46:19.565537 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:19.565492 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15069e2e-354d-4d4e-a7e6-d5be9afc6def" containerName="cleanup" Jul 07 15:46:19.565537 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:19.565516 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="15069e2e-354d-4d4e-a7e6-d5be9afc6def" containerName="cleanup" Jul 07 15:46:19.565754 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:19.565656 2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="15069e2e-354d-4d4e-a7e6-d5be9afc6def" containerName="cleanup" Jul 07 15:46:19.569304 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:19.569284 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85746cdfd6-nv87t" Jul 07 15:46:19.582016 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:19.581990 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85746cdfd6-nv87t"] Jul 07 15:46:19.715759 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:19.715725 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd10de4f-55c8-44a6-980b-12bd90d2c9b8-trusted-ca-bundle\") pod \"console-85746cdfd6-nv87t\" (UID: \"dd10de4f-55c8-44a6-980b-12bd90d2c9b8\") " pod="openshift-console/console-85746cdfd6-nv87t" Jul 07 15:46:19.715914 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:19.715772 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dd10de4f-55c8-44a6-980b-12bd90d2c9b8-service-ca\") pod \"console-85746cdfd6-nv87t\" (UID: \"dd10de4f-55c8-44a6-980b-12bd90d2c9b8\") " pod="openshift-console/console-85746cdfd6-nv87t" Jul 07 15:46:19.715914 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:19.715829 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dd10de4f-55c8-44a6-980b-12bd90d2c9b8-console-serving-cert\") pod \"console-85746cdfd6-nv87t\" (UID: \"dd10de4f-55c8-44a6-980b-12bd90d2c9b8\") " pod="openshift-console/console-85746cdfd6-nv87t" Jul 07 15:46:19.715914 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:19.715868 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dd10de4f-55c8-44a6-980b-12bd90d2c9b8-console-oauth-config\") pod \"console-85746cdfd6-nv87t\" (UID: \"dd10de4f-55c8-44a6-980b-12bd90d2c9b8\") " pod="openshift-console/console-85746cdfd6-nv87t" Jul 07 15:46:19.715914 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:19.715902 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dd10de4f-55c8-44a6-980b-12bd90d2c9b8-oauth-serving-cert\") pod \"console-85746cdfd6-nv87t\" (UID: \"dd10de4f-55c8-44a6-980b-12bd90d2c9b8\") " pod="openshift-console/console-85746cdfd6-nv87t" Jul 07 15:46:19.716080 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:19.715958 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzttw\" (UniqueName: \"kubernetes.io/projected/dd10de4f-55c8-44a6-980b-12bd90d2c9b8-kube-api-access-dzttw\") pod \"console-85746cdfd6-nv87t\" (UID: \"dd10de4f-55c8-44a6-980b-12bd90d2c9b8\") " pod="openshift-console/console-85746cdfd6-nv87t" Jul 07 15:46:19.716080 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:19.715980 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dd10de4f-55c8-44a6-980b-12bd90d2c9b8-console-config\") pod \"console-85746cdfd6-nv87t\" (UID: \"dd10de4f-55c8-44a6-980b-12bd90d2c9b8\") " pod="openshift-console/console-85746cdfd6-nv87t" Jul 07 15:46:19.816933 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:19.816845 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd10de4f-55c8-44a6-980b-12bd90d2c9b8-trusted-ca-bundle\") pod \"console-85746cdfd6-nv87t\" (UID: \"dd10de4f-55c8-44a6-980b-12bd90d2c9b8\") " pod="openshift-console/console-85746cdfd6-nv87t" Jul 07 15:46:19.816933 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:19.816902 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dd10de4f-55c8-44a6-980b-12bd90d2c9b8-service-ca\") pod \"console-85746cdfd6-nv87t\" (UID: \"dd10de4f-55c8-44a6-980b-12bd90d2c9b8\") " pod="openshift-console/console-85746cdfd6-nv87t" Jul 07 15:46:19.817154 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:19.816950 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dd10de4f-55c8-44a6-980b-12bd90d2c9b8-console-serving-cert\") pod \"console-85746cdfd6-nv87t\" (UID: \"dd10de4f-55c8-44a6-980b-12bd90d2c9b8\") " pod="openshift-console/console-85746cdfd6-nv87t" Jul 07 15:46:19.817154 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:19.816972 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dd10de4f-55c8-44a6-980b-12bd90d2c9b8-console-oauth-config\") pod \"console-85746cdfd6-nv87t\" (UID: \"dd10de4f-55c8-44a6-980b-12bd90d2c9b8\") " pod="openshift-console/console-85746cdfd6-nv87t" Jul 07 15:46:19.817154 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:19.816996 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dd10de4f-55c8-44a6-980b-12bd90d2c9b8-oauth-serving-cert\") pod \"console-85746cdfd6-nv87t\" (UID: \"dd10de4f-55c8-44a6-980b-12bd90d2c9b8\") " pod="openshift-console/console-85746cdfd6-nv87t" Jul 07 15:46:19.817154 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:19.817032 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzttw\" (UniqueName: \"kubernetes.io/projected/dd10de4f-55c8-44a6-980b-12bd90d2c9b8-kube-api-access-dzttw\") pod \"console-85746cdfd6-nv87t\" (UID: \"dd10de4f-55c8-44a6-980b-12bd90d2c9b8\") " pod="openshift-console/console-85746cdfd6-nv87t" Jul 07 15:46:19.817154 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:19.817062 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dd10de4f-55c8-44a6-980b-12bd90d2c9b8-console-config\") pod \"console-85746cdfd6-nv87t\" (UID: \"dd10de4f-55c8-44a6-980b-12bd90d2c9b8\") " pod="openshift-console/console-85746cdfd6-nv87t" Jul 07 15:46:19.817880 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:19.817855 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dd10de4f-55c8-44a6-980b-12bd90d2c9b8-console-config\") pod \"console-85746cdfd6-nv87t\" (UID: \"dd10de4f-55c8-44a6-980b-12bd90d2c9b8\") " pod="openshift-console/console-85746cdfd6-nv87t" Jul 07 15:46:19.817990 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:19.817882 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dd10de4f-55c8-44a6-980b-12bd90d2c9b8-oauth-serving-cert\") pod \"console-85746cdfd6-nv87t\" (UID: \"dd10de4f-55c8-44a6-980b-12bd90d2c9b8\") " pod="openshift-console/console-85746cdfd6-nv87t" Jul 07 15:46:19.817990 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:19.817885 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd10de4f-55c8-44a6-980b-12bd90d2c9b8-trusted-ca-bundle\") pod \"console-85746cdfd6-nv87t\" (UID: \"dd10de4f-55c8-44a6-980b-12bd90d2c9b8\") " pod="openshift-console/console-85746cdfd6-nv87t" Jul 07 15:46:19.818269 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:19.818229 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dd10de4f-55c8-44a6-980b-12bd90d2c9b8-service-ca\") pod \"console-85746cdfd6-nv87t\" (UID: \"dd10de4f-55c8-44a6-980b-12bd90d2c9b8\") " pod="openshift-console/console-85746cdfd6-nv87t" Jul 07 15:46:19.819528 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:19.819499 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dd10de4f-55c8-44a6-980b-12bd90d2c9b8-console-oauth-config\") pod \"console-85746cdfd6-nv87t\" (UID: \"dd10de4f-55c8-44a6-980b-12bd90d2c9b8\") " pod="openshift-console/console-85746cdfd6-nv87t" Jul 07 15:46:19.819638 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:19.819551 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dd10de4f-55c8-44a6-980b-12bd90d2c9b8-console-serving-cert\") pod \"console-85746cdfd6-nv87t\" (UID: \"dd10de4f-55c8-44a6-980b-12bd90d2c9b8\") " pod="openshift-console/console-85746cdfd6-nv87t" Jul 07 15:46:19.824754 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:19.824734 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzttw\" (UniqueName: \"kubernetes.io/projected/dd10de4f-55c8-44a6-980b-12bd90d2c9b8-kube-api-access-dzttw\") pod \"console-85746cdfd6-nv87t\" (UID: \"dd10de4f-55c8-44a6-980b-12bd90d2c9b8\") " pod="openshift-console/console-85746cdfd6-nv87t" Jul 07 15:46:19.883992 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:19.883961 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85746cdfd6-nv87t" Jul 07 15:46:20.031416 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:20.031393 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85746cdfd6-nv87t"] Jul 07 15:46:20.033137 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:46:20.033107 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd10de4f_55c8_44a6_980b_12bd90d2c9b8.slice/crio-8f6b29c5c3f84c2ab6006888f2b6c7ea2a13f7cdb55f5ef4afb8c6429a69c714 WatchSource:0}: Error finding container 8f6b29c5c3f84c2ab6006888f2b6c7ea2a13f7cdb55f5ef4afb8c6429a69c714: Status 404 returned error can't find the container with id 8f6b29c5c3f84c2ab6006888f2b6c7ea2a13f7cdb55f5ef4afb8c6429a69c714 Jul 07 15:46:20.449419 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:20.449324 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85746cdfd6-nv87t" event={"ID":"dd10de4f-55c8-44a6-980b-12bd90d2c9b8","Type":"ContainerStarted","Data":"ad9f845c1ce52bd575db762300446f9090a9134057cc8f8a6d3de29a9ca35256"} Jul 07 15:46:20.449419 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:20.449370 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85746cdfd6-nv87t" event={"ID":"dd10de4f-55c8-44a6-980b-12bd90d2c9b8","Type":"ContainerStarted","Data":"8f6b29c5c3f84c2ab6006888f2b6c7ea2a13f7cdb55f5ef4afb8c6429a69c714"} Jul 07 15:46:20.469896 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:20.469848 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-85746cdfd6-nv87t" podStartSLOduration=1.4698346899999999 podStartE2EDuration="1.46983469s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:46:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:46:20.467297143 +0000 UTC m=+1998.474774595" watchObservedRunningTime="2026-07-07 15:46:20.46983469 +0000 UTC m=+1998.477312141" Jul 07 15:46:28.340571 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:28.340538 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:46:29.884275 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:29.884239 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-85746cdfd6-nv87t" Jul 07 15:46:29.884793 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:29.884303 2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-85746cdfd6-nv87t" Jul 07 15:46:29.889084 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:29.889062 2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-85746cdfd6-nv87t" Jul 07 15:46:30.516010 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:30.515985 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-85746cdfd6-nv87t" Jul 07 15:46:30.567139 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:30.567097 2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-754c6f66b8-jdbdh"] Jul 07 15:46:40.039969 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:40.039936 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:46:55.598483 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:55.598444 2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-754c6f66b8-jdbdh" podUID="d287bac1-b960-4585-b80a-0b3a93f04782" containerName="console" containerID="cri-o://4623568c843a6cccf4922fed2448cf62ac8607905b659d82cd410bfacb514fd2" gracePeriod=15 Jul 07 15:46:55.858844 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:55.858788 2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-754c6f66b8-jdbdh" Jul 07 15:46:55.870123 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:55.870104 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d287bac1-b960-4585-b80a-0b3a93f04782-oauth-serving-cert\") pod \"d287bac1-b960-4585-b80a-0b3a93f04782\" (UID: \"d287bac1-b960-4585-b80a-0b3a93f04782\") " Jul 07 15:46:55.870225 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:55.870171 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d287bac1-b960-4585-b80a-0b3a93f04782-console-oauth-config\") pod \"d287bac1-b960-4585-b80a-0b3a93f04782\" (UID: \"d287bac1-b960-4585-b80a-0b3a93f04782\") " Jul 07 15:46:55.870225 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:55.870210 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d287bac1-b960-4585-b80a-0b3a93f04782-console-config\") pod \"d287bac1-b960-4585-b80a-0b3a93f04782\" (UID: \"d287bac1-b960-4585-b80a-0b3a93f04782\") " Jul 07 15:46:55.870333 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:55.870247 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d287bac1-b960-4585-b80a-0b3a93f04782-trusted-ca-bundle\") pod \"d287bac1-b960-4585-b80a-0b3a93f04782\" (UID: \"d287bac1-b960-4585-b80a-0b3a93f04782\") " Jul 07 15:46:55.870333 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:55.870300 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d287bac1-b960-4585-b80a-0b3a93f04782-service-ca\") pod \"d287bac1-b960-4585-b80a-0b3a93f04782\" (UID: \"d287bac1-b960-4585-b80a-0b3a93f04782\") " Jul 07 15:46:55.870442 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:55.870347 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-947hl\" (UniqueName: \"kubernetes.io/projected/d287bac1-b960-4585-b80a-0b3a93f04782-kube-api-access-947hl\") pod \"d287bac1-b960-4585-b80a-0b3a93f04782\" (UID: \"d287bac1-b960-4585-b80a-0b3a93f04782\") " Jul 07 15:46:55.870442 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:55.870395 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d287bac1-b960-4585-b80a-0b3a93f04782-console-serving-cert\") pod \"d287bac1-b960-4585-b80a-0b3a93f04782\" (UID: \"d287bac1-b960-4585-b80a-0b3a93f04782\") " Jul 07 15:46:55.870642 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:55.870609 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d287bac1-b960-4585-b80a-0b3a93f04782-console-config" (OuterVolumeSpecName: "console-config") pod "d287bac1-b960-4585-b80a-0b3a93f04782" (UID: "d287bac1-b960-4585-b80a-0b3a93f04782"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:46:55.870737 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:55.870685 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d287bac1-b960-4585-b80a-0b3a93f04782-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d287bac1-b960-4585-b80a-0b3a93f04782" (UID: "d287bac1-b960-4585-b80a-0b3a93f04782"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:46:55.870798 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:55.870732 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d287bac1-b960-4585-b80a-0b3a93f04782-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d287bac1-b960-4585-b80a-0b3a93f04782" (UID: "d287bac1-b960-4585-b80a-0b3a93f04782"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:46:55.870798 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:55.870747 2549 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d287bac1-b960-4585-b80a-0b3a93f04782-console-config\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:46:55.870798 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:55.870755 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d287bac1-b960-4585-b80a-0b3a93f04782-service-ca" (OuterVolumeSpecName: "service-ca") pod "d287bac1-b960-4585-b80a-0b3a93f04782" (UID: "d287bac1-b960-4585-b80a-0b3a93f04782"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:46:55.872351 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:55.872322 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d287bac1-b960-4585-b80a-0b3a93f04782-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d287bac1-b960-4585-b80a-0b3a93f04782" (UID: "d287bac1-b960-4585-b80a-0b3a93f04782"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:46:55.872509 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:55.872478 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d287bac1-b960-4585-b80a-0b3a93f04782-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d287bac1-b960-4585-b80a-0b3a93f04782" (UID: "d287bac1-b960-4585-b80a-0b3a93f04782"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:46:55.872557 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:55.872531 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d287bac1-b960-4585-b80a-0b3a93f04782-kube-api-access-947hl" (OuterVolumeSpecName: "kube-api-access-947hl") pod "d287bac1-b960-4585-b80a-0b3a93f04782" (UID: "d287bac1-b960-4585-b80a-0b3a93f04782"). InnerVolumeSpecName "kube-api-access-947hl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:46:55.971639 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:55.971608 2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-947hl\" (UniqueName: \"kubernetes.io/projected/d287bac1-b960-4585-b80a-0b3a93f04782-kube-api-access-947hl\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:46:55.971639 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:55.971638 2549 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d287bac1-b960-4585-b80a-0b3a93f04782-console-serving-cert\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:46:55.971832 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:55.971651 2549 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d287bac1-b960-4585-b80a-0b3a93f04782-oauth-serving-cert\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:46:55.971832 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:55.971664 2549 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d287bac1-b960-4585-b80a-0b3a93f04782-console-oauth-config\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:46:55.971832 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:55.971676 2549 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d287bac1-b960-4585-b80a-0b3a93f04782-trusted-ca-bundle\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:46:55.971832 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:55.971688 2549 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d287bac1-b960-4585-b80a-0b3a93f04782-service-ca\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:46:56.667287 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:56.667256 2549 generic.go:356] "Generic (PLEG): container finished" podID="d287bac1-b960-4585-b80a-0b3a93f04782" containerID="4623568c843a6cccf4922fed2448cf62ac8607905b659d82cd410bfacb514fd2" exitCode=0 Jul 07 15:46:56.667674 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:56.667306 2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-754c6f66b8-jdbdh" Jul 07 15:46:56.667674 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:56.667309 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-754c6f66b8-jdbdh" event={"ID":"d287bac1-b960-4585-b80a-0b3a93f04782","Type":"ContainerDied","Data":"4623568c843a6cccf4922fed2448cf62ac8607905b659d82cd410bfacb514fd2"} Jul 07 15:46:56.667674 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:56.667439 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-754c6f66b8-jdbdh" event={"ID":"d287bac1-b960-4585-b80a-0b3a93f04782","Type":"ContainerDied","Data":"d21b5353c0f8a3da0bde970131c0f3ee9100b10fbb5d07f75027f069045ac854"} Jul 07 15:46:56.667674 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:56.667466 2549 scope.go:117] "RemoveContainer" containerID="4623568c843a6cccf4922fed2448cf62ac8607905b659d82cd410bfacb514fd2" Jul 07 15:46:56.676866 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:56.676852 2549 scope.go:117] "RemoveContainer" containerID="4623568c843a6cccf4922fed2448cf62ac8607905b659d82cd410bfacb514fd2" Jul 07 15:46:56.677101 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:46:56.677080 2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4623568c843a6cccf4922fed2448cf62ac8607905b659d82cd410bfacb514fd2\": container with ID starting with 4623568c843a6cccf4922fed2448cf62ac8607905b659d82cd410bfacb514fd2 not found: ID does not exist" containerID="4623568c843a6cccf4922fed2448cf62ac8607905b659d82cd410bfacb514fd2" Jul 07 15:46:56.677167 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:56.677113 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4623568c843a6cccf4922fed2448cf62ac8607905b659d82cd410bfacb514fd2"} err="failed to get container status \"4623568c843a6cccf4922fed2448cf62ac8607905b659d82cd410bfacb514fd2\": rpc error: code = NotFound desc = could not find container \"4623568c843a6cccf4922fed2448cf62ac8607905b659d82cd410bfacb514fd2\": container with ID starting with 4623568c843a6cccf4922fed2448cf62ac8607905b659d82cd410bfacb514fd2 not found: ID does not exist" Jul 07 15:46:56.685266 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:56.685248 2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-754c6f66b8-jdbdh"] Jul 07 15:46:56.691190 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:56.691169 2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-754c6f66b8-jdbdh"] Jul 07 15:46:58.535667 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:46:58.535635 2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d287bac1-b960-4585-b80a-0b3a93f04782" path="/var/lib/kubelet/pods/d287bac1-b960-4585-b80a-0b3a93f04782/volumes" Jul 07 15:47:15.334654 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:47:15.334616 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:47:28.250009 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:47:28.249708 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:47:40.438196 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:47:40.438165 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:47:54.252191 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:47:54.252155 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:48:02.700401 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:48:02.700370 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kb4ml_3301f6ef-6671-4755-9047-16959836232c/console-operator/2.log" Jul 07 15:48:02.713920 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:48:02.713898 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kb4ml_3301f6ef-6671-4755-9047-16959836232c/console-operator/2.log" Jul 07 15:48:08.431311 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:48:08.431276 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:48:18.738808 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:48:18.738771 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:48:26.735036 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:48:26.734987 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:48:57.832657 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:48:57.832559 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:49:14.741525 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:49:14.741486 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:49:32.336457 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:49:32.336417 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:49:47.798143 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:49:47.798102 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:50:03.076315 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:50:03.076277 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:50:21.090928 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:50:21.090899 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:50:36.357625 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:50:36.357577 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:50:53.994030 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:50:53.993997 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:51:02.577018 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:51:02.576986 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-1f2ee2/sub-test-model-1f2ee2-kserve-847c97d497-22rfm"] Jul 07 15:51:02.577469 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:51:02.577457 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d287bac1-b960-4585-b80a-0b3a93f04782" containerName="console" Jul 07 15:51:02.577469 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:51:02.577470 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="d287bac1-b960-4585-b80a-0b3a93f04782" containerName="console" Jul 07 15:51:02.577546 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:51:02.577527 2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="d287bac1-b960-4585-b80a-0b3a93f04782" containerName="console" Jul 07 15:51:02.580303 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:51:02.580286 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1f2ee2/sub-test-model-1f2ee2-kserve-847c97d497-22rfm" Jul 07 15:51:02.585411 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:51:02.585390 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-1f2ee2\"/\"openshift-service-ca.crt\"" Jul 07 15:51:02.585613 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:51:02.585574 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-1f2ee2\"/\"default-dockercfg-7g94j\"" Jul 07 15:51:02.586371 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:51:02.586349 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-1f2ee2\"/\"kube-root-ca.crt\"" Jul 07 15:51:02.586512 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:51:02.586492 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-1f2ee2\"/\"sub-test-model-1f2ee2-kserve-self-signed-certs\"" Jul 07 15:51:02.598151 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:51:02.598123 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-1f2ee2/sub-test-model-1f2ee2-kserve-847c97d497-22rfm"] Jul 07 15:51:02.656924 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:51:02.656889 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e106dcb-e7f5-4d7d-9364-066e13c7bc17-model-cache\") pod \"sub-test-model-1f2ee2-kserve-847c97d497-22rfm\" (UID: \"3e106dcb-e7f5-4d7d-9364-066e13c7bc17\") " pod="ai-tenant-e2e-shared-a-1f2ee2/sub-test-model-1f2ee2-kserve-847c97d497-22rfm" Jul 07 15:51:02.657204 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:51:02.657184 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e106dcb-e7f5-4d7d-9364-066e13c7bc17-tmp-dir\") pod \"sub-test-model-1f2ee2-kserve-847c97d497-22rfm\" (UID: \"3e106dcb-e7f5-4d7d-9364-066e13c7bc17\") " pod="ai-tenant-e2e-shared-a-1f2ee2/sub-test-model-1f2ee2-kserve-847c97d497-22rfm" Jul 07 15:51:02.657359 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:51:02.657345 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e106dcb-e7f5-4d7d-9364-066e13c7bc17-home\") pod \"sub-test-model-1f2ee2-kserve-847c97d497-22rfm\" (UID: \"3e106dcb-e7f5-4d7d-9364-066e13c7bc17\") " pod="ai-tenant-e2e-shared-a-1f2ee2/sub-test-model-1f2ee2-kserve-847c97d497-22rfm" Jul 07 15:51:02.657471 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:51:02.657458 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e106dcb-e7f5-4d7d-9364-066e13c7bc17-dshm\") pod \"sub-test-model-1f2ee2-kserve-847c97d497-22rfm\" (UID: \"3e106dcb-e7f5-4d7d-9364-066e13c7bc17\") " pod="ai-tenant-e2e-shared-a-1f2ee2/sub-test-model-1f2ee2-kserve-847c97d497-22rfm" Jul 07 15:51:02.657606 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:51:02.657571 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbtzj\" (UniqueName: \"kubernetes.io/projected/3e106dcb-e7f5-4d7d-9364-066e13c7bc17-kube-api-access-wbtzj\") pod \"sub-test-model-1f2ee2-kserve-847c97d497-22rfm\" (UID: \"3e106dcb-e7f5-4d7d-9364-066e13c7bc17\") " pod="ai-tenant-e2e-shared-a-1f2ee2/sub-test-model-1f2ee2-kserve-847c97d497-22rfm" Jul 07 15:51:02.657808 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:51:02.657793 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e106dcb-e7f5-4d7d-9364-066e13c7bc17-tls-certs\") pod \"sub-test-model-1f2ee2-kserve-847c97d497-22rfm\" (UID: \"3e106dcb-e7f5-4d7d-9364-066e13c7bc17\") " pod="ai-tenant-e2e-shared-a-1f2ee2/sub-test-model-1f2ee2-kserve-847c97d497-22rfm" Jul 07 15:51:02.758397 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:51:02.758357 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e106dcb-e7f5-4d7d-9364-066e13c7bc17-model-cache\") pod \"sub-test-model-1f2ee2-kserve-847c97d497-22rfm\" (UID: \"3e106dcb-e7f5-4d7d-9364-066e13c7bc17\") " pod="ai-tenant-e2e-shared-a-1f2ee2/sub-test-model-1f2ee2-kserve-847c97d497-22rfm" Jul 07 15:51:02.758397 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:51:02.758400 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e106dcb-e7f5-4d7d-9364-066e13c7bc17-tmp-dir\") pod \"sub-test-model-1f2ee2-kserve-847c97d497-22rfm\" (UID: \"3e106dcb-e7f5-4d7d-9364-066e13c7bc17\") " pod="ai-tenant-e2e-shared-a-1f2ee2/sub-test-model-1f2ee2-kserve-847c97d497-22rfm" Jul 07 15:51:02.758648 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:51:02.758430 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e106dcb-e7f5-4d7d-9364-066e13c7bc17-home\") pod \"sub-test-model-1f2ee2-kserve-847c97d497-22rfm\" (UID: \"3e106dcb-e7f5-4d7d-9364-066e13c7bc17\") " pod="ai-tenant-e2e-shared-a-1f2ee2/sub-test-model-1f2ee2-kserve-847c97d497-22rfm" Jul 07 15:51:02.758648 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:51:02.758448 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e106dcb-e7f5-4d7d-9364-066e13c7bc17-dshm\") pod \"sub-test-model-1f2ee2-kserve-847c97d497-22rfm\" (UID: \"3e106dcb-e7f5-4d7d-9364-066e13c7bc17\") " pod="ai-tenant-e2e-shared-a-1f2ee2/sub-test-model-1f2ee2-kserve-847c97d497-22rfm" Jul 07 15:51:02.758648 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:51:02.758470 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbtzj\" (UniqueName: \"kubernetes.io/projected/3e106dcb-e7f5-4d7d-9364-066e13c7bc17-kube-api-access-wbtzj\") pod \"sub-test-model-1f2ee2-kserve-847c97d497-22rfm\" (UID: \"3e106dcb-e7f5-4d7d-9364-066e13c7bc17\") " pod="ai-tenant-e2e-shared-a-1f2ee2/sub-test-model-1f2ee2-kserve-847c97d497-22rfm" Jul 07 15:51:02.758648 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:51:02.758538 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e106dcb-e7f5-4d7d-9364-066e13c7bc17-tls-certs\") pod \"sub-test-model-1f2ee2-kserve-847c97d497-22rfm\" (UID: \"3e106dcb-e7f5-4d7d-9364-066e13c7bc17\") " pod="ai-tenant-e2e-shared-a-1f2ee2/sub-test-model-1f2ee2-kserve-847c97d497-22rfm" Jul 07 15:51:02.758872 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:51:02.758837 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e106dcb-e7f5-4d7d-9364-066e13c7bc17-model-cache\") pod \"sub-test-model-1f2ee2-kserve-847c97d497-22rfm\" (UID: \"3e106dcb-e7f5-4d7d-9364-066e13c7bc17\") " pod="ai-tenant-e2e-shared-a-1f2ee2/sub-test-model-1f2ee2-kserve-847c97d497-22rfm" Jul 07 15:51:02.758948 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:51:02.758876 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e106dcb-e7f5-4d7d-9364-066e13c7bc17-home\") pod \"sub-test-model-1f2ee2-kserve-847c97d497-22rfm\" (UID: \"3e106dcb-e7f5-4d7d-9364-066e13c7bc17\") " pod="ai-tenant-e2e-shared-a-1f2ee2/sub-test-model-1f2ee2-kserve-847c97d497-22rfm" Jul 07 15:51:02.759007 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:51:02.758980 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e106dcb-e7f5-4d7d-9364-066e13c7bc17-tmp-dir\") pod \"sub-test-model-1f2ee2-kserve-847c97d497-22rfm\" (UID: \"3e106dcb-e7f5-4d7d-9364-066e13c7bc17\") " pod="ai-tenant-e2e-shared-a-1f2ee2/sub-test-model-1f2ee2-kserve-847c97d497-22rfm" Jul 07 15:51:02.760943 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:51:02.760921 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e106dcb-e7f5-4d7d-9364-066e13c7bc17-dshm\") pod \"sub-test-model-1f2ee2-kserve-847c97d497-22rfm\" (UID: \"3e106dcb-e7f5-4d7d-9364-066e13c7bc17\") " pod="ai-tenant-e2e-shared-a-1f2ee2/sub-test-model-1f2ee2-kserve-847c97d497-22rfm" Jul 07 15:51:02.761323 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:51:02.761303 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e106dcb-e7f5-4d7d-9364-066e13c7bc17-tls-certs\") pod \"sub-test-model-1f2ee2-kserve-847c97d497-22rfm\" (UID: \"3e106dcb-e7f5-4d7d-9364-066e13c7bc17\") " pod="ai-tenant-e2e-shared-a-1f2ee2/sub-test-model-1f2ee2-kserve-847c97d497-22rfm" Jul 07 15:51:02.778084 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:51:02.778056 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbtzj\" (UniqueName: \"kubernetes.io/projected/3e106dcb-e7f5-4d7d-9364-066e13c7bc17-kube-api-access-wbtzj\") pod \"sub-test-model-1f2ee2-kserve-847c97d497-22rfm\" (UID: \"3e106dcb-e7f5-4d7d-9364-066e13c7bc17\") " pod="ai-tenant-e2e-shared-a-1f2ee2/sub-test-model-1f2ee2-kserve-847c97d497-22rfm" Jul 07 15:51:02.889982 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:51:02.889898 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1f2ee2/sub-test-model-1f2ee2-kserve-847c97d497-22rfm" Jul 07 15:51:03.055385 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:51:03.055358 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-1f2ee2/sub-test-model-1f2ee2-kserve-847c97d497-22rfm"] Jul 07 15:51:03.057223 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:51:03.057199 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3e106dcb_e7f5_4d7d_9364_066e13c7bc17.slice/crio-bd7150a221f4caa86d5e6ccc5b60e2244f0386645457f734236c3331e76e530c WatchSource:0}: Error finding container bd7150a221f4caa86d5e6ccc5b60e2244f0386645457f734236c3331e76e530c: Status 404 returned error can't find the container with id bd7150a221f4caa86d5e6ccc5b60e2244f0386645457f734236c3331e76e530c Jul 07 15:51:03.171293 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:51:03.171202 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1f2ee2/sub-test-model-1f2ee2-kserve-847c97d497-22rfm" event={"ID":"3e106dcb-e7f5-4d7d-9364-066e13c7bc17","Type":"ContainerStarted","Data":"261c194d18c7e9891b6d11772580eec6eb09a5f3389030057805e687419d70d0"} Jul 07 15:51:03.171293 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:51:03.171241 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1f2ee2/sub-test-model-1f2ee2-kserve-847c97d497-22rfm" event={"ID":"3e106dcb-e7f5-4d7d-9364-066e13c7bc17","Type":"ContainerStarted","Data":"bd7150a221f4caa86d5e6ccc5b60e2244f0386645457f734236c3331e76e530c"} Jul 07 15:51:03.201932 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:51:03.201878 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-1f2ee2/sub-test-model-1f2ee2-kserve-847c97d497-22rfm" podStartSLOduration=1.201861587 podStartE2EDuration="1.201861587s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:51:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:51:03.201488388 +0000 UTC m=+2281.208965842" watchObservedRunningTime="2026-07-07 15:51:03.201861587 +0000 UTC m=+2281.209339039" Jul 07 15:51:09.362507 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:51:09.362472 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:51:12.890990 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:51:12.890952 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-1f2ee2/sub-test-model-1f2ee2-kserve-847c97d497-22rfm" Jul 07 15:51:12.891392 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:51:12.891009 2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-1f2ee2/sub-test-model-1f2ee2-kserve-847c97d497-22rfm" Jul 07 15:51:12.903694 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:51:12.903673 2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-1f2ee2/sub-test-model-1f2ee2-kserve-847c97d497-22rfm" Jul 07 15:51:13.243718 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:51:13.243633 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-1f2ee2/sub-test-model-1f2ee2-kserve-847c97d497-22rfm" Jul 07 15:51:31.649713 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:51:31.649669 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:51:49.269537 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:51:49.269498 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:52:05.143315 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:52:05.143284 2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-1f2ee2/sub-test-model-1f2ee2-kserve-847c97d497-22rfm"] Jul 07 15:52:09.840044 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:52:09.840008 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:52:20.163625 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:52:20.163539 2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-1f2ee2/sub-test-model-1f2ee2-kserve-847c97d497-22rfm" podUID="3e106dcb-e7f5-4d7d-9364-066e13c7bc17" containerName="main" containerID="cri-o://261c194d18c7e9891b6d11772580eec6eb09a5f3389030057805e687419d70d0" gracePeriod=45 Jul 07 15:52:20.427362 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:52:20.427300 2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1f2ee2/sub-test-model-1f2ee2-kserve-847c97d497-22rfm" Jul 07 15:52:20.508926 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:52:20.508892 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e106dcb-e7f5-4d7d-9364-066e13c7bc17-home\") pod \"3e106dcb-e7f5-4d7d-9364-066e13c7bc17\" (UID: \"3e106dcb-e7f5-4d7d-9364-066e13c7bc17\") " Jul 07 15:52:20.508926 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:52:20.508927 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e106dcb-e7f5-4d7d-9364-066e13c7bc17-model-cache\") pod \"3e106dcb-e7f5-4d7d-9364-066e13c7bc17\" (UID: \"3e106dcb-e7f5-4d7d-9364-066e13c7bc17\") " Jul 07 15:52:20.509170 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:52:20.508957 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wbtzj\" (UniqueName: \"kubernetes.io/projected/3e106dcb-e7f5-4d7d-9364-066e13c7bc17-kube-api-access-wbtzj\") pod \"3e106dcb-e7f5-4d7d-9364-066e13c7bc17\" (UID: \"3e106dcb-e7f5-4d7d-9364-066e13c7bc17\") " Jul 07 15:52:20.509170 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:52:20.509009 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e106dcb-e7f5-4d7d-9364-066e13c7bc17-tmp-dir\") pod \"3e106dcb-e7f5-4d7d-9364-066e13c7bc17\" (UID: \"3e106dcb-e7f5-4d7d-9364-066e13c7bc17\") " Jul 07 15:52:20.509170 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:52:20.509029 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e106dcb-e7f5-4d7d-9364-066e13c7bc17-dshm\") pod \"3e106dcb-e7f5-4d7d-9364-066e13c7bc17\" (UID: \"3e106dcb-e7f5-4d7d-9364-066e13c7bc17\") " Jul 07 15:52:20.509170 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:52:20.509069 2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e106dcb-e7f5-4d7d-9364-066e13c7bc17-tls-certs\") pod \"3e106dcb-e7f5-4d7d-9364-066e13c7bc17\" (UID: \"3e106dcb-e7f5-4d7d-9364-066e13c7bc17\") " Jul 07 15:52:20.509395 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:52:20.509209 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e106dcb-e7f5-4d7d-9364-066e13c7bc17-home" (OuterVolumeSpecName: "home") pod "3e106dcb-e7f5-4d7d-9364-066e13c7bc17" (UID: "3e106dcb-e7f5-4d7d-9364-066e13c7bc17"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:52:20.509395 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:52:20.509228 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e106dcb-e7f5-4d7d-9364-066e13c7bc17-model-cache" (OuterVolumeSpecName: "model-cache") pod "3e106dcb-e7f5-4d7d-9364-066e13c7bc17" (UID: "3e106dcb-e7f5-4d7d-9364-066e13c7bc17"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:52:20.509395 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:52:20.509319 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e106dcb-e7f5-4d7d-9364-066e13c7bc17-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3e106dcb-e7f5-4d7d-9364-066e13c7bc17" (UID: "3e106dcb-e7f5-4d7d-9364-066e13c7bc17"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:52:20.509509 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:52:20.509407 2549 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e106dcb-e7f5-4d7d-9364-066e13c7bc17-model-cache\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:52:20.509509 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:52:20.509422 2549 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e106dcb-e7f5-4d7d-9364-066e13c7bc17-tmp-dir\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:52:20.509509 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:52:20.509433 2549 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e106dcb-e7f5-4d7d-9364-066e13c7bc17-home\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:52:20.511343 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:52:20.511322 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e106dcb-e7f5-4d7d-9364-066e13c7bc17-kube-api-access-wbtzj" (OuterVolumeSpecName: "kube-api-access-wbtzj") pod "3e106dcb-e7f5-4d7d-9364-066e13c7bc17" (UID: "3e106dcb-e7f5-4d7d-9364-066e13c7bc17"). InnerVolumeSpecName "kube-api-access-wbtzj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:52:20.511415 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:52:20.511322 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e106dcb-e7f5-4d7d-9364-066e13c7bc17-dshm" (OuterVolumeSpecName: "dshm") pod "3e106dcb-e7f5-4d7d-9364-066e13c7bc17" (UID: "3e106dcb-e7f5-4d7d-9364-066e13c7bc17"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:52:20.511415 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:52:20.511322 2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e106dcb-e7f5-4d7d-9364-066e13c7bc17-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3e106dcb-e7f5-4d7d-9364-066e13c7bc17" (UID: "3e106dcb-e7f5-4d7d-9364-066e13c7bc17"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:52:20.610006 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:52:20.609963 2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wbtzj\" (UniqueName: \"kubernetes.io/projected/3e106dcb-e7f5-4d7d-9364-066e13c7bc17-kube-api-access-wbtzj\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:52:20.610006 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:52:20.609995 2549 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e106dcb-e7f5-4d7d-9364-066e13c7bc17-dshm\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:52:20.610006 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:52:20.610005 2549 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e106dcb-e7f5-4d7d-9364-066e13c7bc17-tls-certs\") on node \"ip-10-0-138-11.ec2.internal\" DevicePath \"\"" Jul 07 15:52:20.669198 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:52:20.669163 2549 generic.go:356] "Generic (PLEG): container finished" podID="3e106dcb-e7f5-4d7d-9364-066e13c7bc17" containerID="261c194d18c7e9891b6d11772580eec6eb09a5f3389030057805e687419d70d0" exitCode=0 Jul 07 15:52:20.669378 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:52:20.669224 2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1f2ee2/sub-test-model-1f2ee2-kserve-847c97d497-22rfm" Jul 07 15:52:20.669378 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:52:20.669259 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1f2ee2/sub-test-model-1f2ee2-kserve-847c97d497-22rfm" event={"ID":"3e106dcb-e7f5-4d7d-9364-066e13c7bc17","Type":"ContainerDied","Data":"261c194d18c7e9891b6d11772580eec6eb09a5f3389030057805e687419d70d0"} Jul 07 15:52:20.669378 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:52:20.669318 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1f2ee2/sub-test-model-1f2ee2-kserve-847c97d497-22rfm" event={"ID":"3e106dcb-e7f5-4d7d-9364-066e13c7bc17","Type":"ContainerDied","Data":"bd7150a221f4caa86d5e6ccc5b60e2244f0386645457f734236c3331e76e530c"} Jul 07 15:52:20.669378 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:52:20.669341 2549 scope.go:117] "RemoveContainer" containerID="261c194d18c7e9891b6d11772580eec6eb09a5f3389030057805e687419d70d0" Jul 07 15:52:20.681137 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:52:20.681113 2549 scope.go:117] "RemoveContainer" containerID="261c194d18c7e9891b6d11772580eec6eb09a5f3389030057805e687419d70d0" Jul 07 15:52:20.681447 ip-10-0-138-11 kubenswrapper[2549]: E0707 15:52:20.681420 2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"261c194d18c7e9891b6d11772580eec6eb09a5f3389030057805e687419d70d0\": container with ID starting with 261c194d18c7e9891b6d11772580eec6eb09a5f3389030057805e687419d70d0 not found: ID does not exist" containerID="261c194d18c7e9891b6d11772580eec6eb09a5f3389030057805e687419d70d0" Jul 07 15:52:20.681559 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:52:20.681457 2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"261c194d18c7e9891b6d11772580eec6eb09a5f3389030057805e687419d70d0"} err="failed to get container status \"261c194d18c7e9891b6d11772580eec6eb09a5f3389030057805e687419d70d0\": rpc error: code = NotFound desc = could not find container \"261c194d18c7e9891b6d11772580eec6eb09a5f3389030057805e687419d70d0\": container with ID starting with 261c194d18c7e9891b6d11772580eec6eb09a5f3389030057805e687419d70d0 not found: ID does not exist" Jul 07 15:52:20.688938 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:52:20.688911 2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-1f2ee2/sub-test-model-1f2ee2-kserve-847c97d497-22rfm"] Jul 07 15:52:20.691406 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:52:20.691387 2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-1f2ee2/sub-test-model-1f2ee2-kserve-847c97d497-22rfm"] Jul 07 15:52:22.535829 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:52:22.535797 2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e106dcb-e7f5-4d7d-9364-066e13c7bc17" path="/var/lib/kubelet/pods/3e106dcb-e7f5-4d7d-9364-066e13c7bc17/volumes" Jul 07 15:52:23.936376 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:52:23.936346 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:52:54.339186 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:52:54.339146 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:53:02.763672 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:53:02.763645 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kb4ml_3301f6ef-6671-4755-9047-16959836232c/console-operator/2.log" Jul 07 15:53:02.774966 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:53:02.774930 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kb4ml_3301f6ef-6671-4755-9047-16959836232c/console-operator/2.log" Jul 07 15:53:11.542696 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:53:11.542659 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:53:26.437777 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:53:26.437699 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:53:39.149082 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:53:39.149044 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:54:21.446720 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:21.446686 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xf9m5"] Jul 07 15:54:26.502497 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:26.502465 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-fdd655b5-86qvh_07d18ca8-06d5-4526-9639-bb4db61ff4f6/authorino/0.log" Jul 07 15:54:31.067144 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:31.067107 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-t97r8_ff832223-8f66-4299-94e1-35b79a4ccc22/manager/0.log" Jul 07 15:54:31.189968 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:31.189931 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-9f7tp_27ffe7dc-0f8f-4d32-882b-6b0829ab71ad/manager/0.log" Jul 07 15:54:31.449726 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:31.449622 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29723970-m8rzm_5dc0b063-d8d0-44e6-a341-258a4b41d89f/cleanup/0.log" Jul 07 15:54:31.571519 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:31.571482 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29723985-lzwl4_15069e2e-354d-4d4e-a7e6-d5be9afc6def/cleanup/0.log" Jul 07 15:54:31.809710 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:31.809625 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-dc8f7_9b60c5f2-1af3-4b36-adcb-f1ade7775ca5/server/0.log" Jul 07 15:54:31.933432 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:31.933402 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-7fshj_e74b6418-6103-41d5-9828-988395d57e82/manager/0.log" Jul 07 15:54:32.295687 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:32.295662 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dcbf69668-wldx7_8779b525-5eb5-43f7-8168-ab732a67c10a/copy-manifests/0.log" Jul 07 15:54:32.314945 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:32.314918 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dcbf69668-wldx7_8779b525-5eb5-43f7-8168-ab732a67c10a/manager/0.log" Jul 07 15:54:33.172268 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:33.172241 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tfcvf_2d88f62c-1bee-4cf7-a1e8-5d009e7920f4/extract/0.log" Jul 07 15:54:33.178452 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:33.178428 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tfcvf_2d88f62c-1bee-4cf7-a1e8-5d009e7920f4/util/0.log" Jul 07 15:54:33.187890 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:33.187872 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tfcvf_2d88f62c-1bee-4cf7-a1e8-5d009e7920f4/pull/0.log" Jul 07 15:54:33.298051 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:33.298022 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jmnk5_6b8fcac7-bd9c-48f1-819a-918c59419cce/util/0.log" Jul 07 15:54:33.305479 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:33.305458 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jmnk5_6b8fcac7-bd9c-48f1-819a-918c59419cce/pull/0.log" Jul 07 15:54:33.312748 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:33.312723 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jmnk5_6b8fcac7-bd9c-48f1-819a-918c59419cce/extract/0.log" Jul 07 15:54:33.420646 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:33.420619 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vhnst_2e0bce97-2257-42da-83d3-e2d9fe891e0e/extract/0.log" Jul 07 15:54:33.427190 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:33.427125 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vhnst_2e0bce97-2257-42da-83d3-e2d9fe891e0e/util/0.log" Jul 07 15:54:33.434259 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:33.434237 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vhnst_2e0bce97-2257-42da-83d3-e2d9fe891e0e/pull/0.log" Jul 07 15:54:33.557812 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:33.557776 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1flzkq_f56fd4a1-89f3-476f-98dc-19005651962b/extract/0.log" Jul 07 15:54:33.564723 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:33.564697 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1flzkq_f56fd4a1-89f3-476f-98dc-19005651962b/util/0.log" Jul 07 15:54:33.572662 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:33.572637 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1flzkq_f56fd4a1-89f3-476f-98dc-19005651962b/pull/0.log" Jul 07 15:54:33.693221 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:33.693143 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-fdd655b5-86qvh_07d18ca8-06d5-4526-9639-bb4db61ff4f6/authorino/0.log" Jul 07 15:54:33.807792 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:33.807765 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-bxk7h_2cc62d89-fbb0-4e39-95f0-ab676d24f36b/manager/0.log" Jul 07 15:54:33.922622 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:33.922596 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-pwpzp_372030ff-029f-461b-8199-5c810f934692/manager/0.log" Jul 07 15:54:34.044334 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:34.044266 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-s4f5c_36a474cd-2744-486e-82fe-3a75834a5658/kuadrant-console-plugin/0.log" Jul 07 15:54:34.167417 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:34.167390 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-qg79z_79f7e333-21f4-430c-98dc-cdcc9880ba4c/registry-server/0.log" Jul 07 15:54:34.283591 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:34.283564 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-7ksmp_db31a5e5-1e8c-4309-90f6-8debeb83bc31/manager/0.log" Jul 07 15:54:34.393027 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:34.393004 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-xf9m5_1b5f5855-5c60-45a1-a6a1-bffae8500df6/limitador/0.log" Jul 07 15:54:34.509371 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:34.509342 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-v75tr_6d747b0b-7182-4179-b7db-7864774c0b76/manager/0.log" Jul 07 15:54:35.095708 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:35.095682 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5c6c759bc-5c7v6_01d82b99-af4a-4369-9164-360e15d583ac/kube-auth-proxy/0.log" Jul 07 15:54:35.663449 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:35.663415 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-64478b6d7c-vxvp5_77ec27ef-75d0-454a-9cd5-803a776c4985/router/0.log" Jul 07 15:54:36.256610 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:36.256569 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-mrvqg_565a55e9-c4c2-4f3c-8767-7d4111677f81/main/0.log" Jul 07 15:54:40.494670 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:40.494635 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-wvtd8/must-gather-zv9l5"] Jul 07 15:54:40.495139 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:40.495122 2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e106dcb-e7f5-4d7d-9364-066e13c7bc17" containerName="main" Jul 07 15:54:40.495186 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:40.495141 2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e106dcb-e7f5-4d7d-9364-066e13c7bc17" containerName="main" Jul 07 15:54:40.495249 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:40.495239 2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e106dcb-e7f5-4d7d-9364-066e13c7bc17" containerName="main" Jul 07 15:54:40.498347 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:40.498327 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wvtd8/must-gather-zv9l5" Jul 07 15:54:40.501597 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:40.501560 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wvtd8\"/\"openshift-service-ca.crt\"" Jul 07 15:54:40.501732 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:40.501563 2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wvtd8\"/\"kube-root-ca.crt\"" Jul 07 15:54:40.501817 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:40.501804 2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-wvtd8\"/\"default-dockercfg-4d48b\"" Jul 07 15:54:40.510555 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:40.510531 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wvtd8/must-gather-zv9l5"] Jul 07 15:54:40.624036 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:40.624005 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/af41ba91-a560-4b2f-91b5-b55fcc848f23-must-gather-output\") pod \"must-gather-zv9l5\" (UID: \"af41ba91-a560-4b2f-91b5-b55fcc848f23\") " pod="openshift-must-gather-wvtd8/must-gather-zv9l5" Jul 07 15:54:40.624192 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:40.624061 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lblx7\" (UniqueName: \"kubernetes.io/projected/af41ba91-a560-4b2f-91b5-b55fcc848f23-kube-api-access-lblx7\") pod \"must-gather-zv9l5\" (UID: \"af41ba91-a560-4b2f-91b5-b55fcc848f23\") " pod="openshift-must-gather-wvtd8/must-gather-zv9l5" Jul 07 15:54:40.724744 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:40.724714 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lblx7\" (UniqueName: \"kubernetes.io/projected/af41ba91-a560-4b2f-91b5-b55fcc848f23-kube-api-access-lblx7\") pod \"must-gather-zv9l5\" (UID: \"af41ba91-a560-4b2f-91b5-b55fcc848f23\") " pod="openshift-must-gather-wvtd8/must-gather-zv9l5" Jul 07 15:54:40.724897 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:40.724806 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/af41ba91-a560-4b2f-91b5-b55fcc848f23-must-gather-output\") pod \"must-gather-zv9l5\" (UID: \"af41ba91-a560-4b2f-91b5-b55fcc848f23\") " pod="openshift-must-gather-wvtd8/must-gather-zv9l5" Jul 07 15:54:40.725083 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:40.725063 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/af41ba91-a560-4b2f-91b5-b55fcc848f23-must-gather-output\") pod \"must-gather-zv9l5\" (UID: \"af41ba91-a560-4b2f-91b5-b55fcc848f23\") " pod="openshift-must-gather-wvtd8/must-gather-zv9l5" Jul 07 15:54:40.740519 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:40.740494 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lblx7\" (UniqueName: \"kubernetes.io/projected/af41ba91-a560-4b2f-91b5-b55fcc848f23-kube-api-access-lblx7\") pod \"must-gather-zv9l5\" (UID: \"af41ba91-a560-4b2f-91b5-b55fcc848f23\") " pod="openshift-must-gather-wvtd8/must-gather-zv9l5" Jul 07 15:54:40.807379 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:40.807316 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wvtd8/must-gather-zv9l5" Jul 07 15:54:40.953673 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:40.953641 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wvtd8/must-gather-zv9l5"] Jul 07 15:54:40.956159 ip-10-0-138-11 kubenswrapper[2549]: W0707 15:54:40.956126 2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaf41ba91_a560_4b2f_91b5_b55fcc848f23.slice/crio-a3e3138424b4642b4480b25cc59c25afd3411f68e68cb990711269ea4bf7bf98 WatchSource:0}: Error finding container a3e3138424b4642b4480b25cc59c25afd3411f68e68cb990711269ea4bf7bf98: Status 404 returned error can't find the container with id a3e3138424b4642b4480b25cc59c25afd3411f68e68cb990711269ea4bf7bf98 Jul 07 15:54:40.958350 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:40.958330 2549 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 15:54:41.521023 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:41.520988 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wvtd8/must-gather-zv9l5" event={"ID":"af41ba91-a560-4b2f-91b5-b55fcc848f23","Type":"ContainerStarted","Data":"a3e3138424b4642b4480b25cc59c25afd3411f68e68cb990711269ea4bf7bf98"} Jul 07 15:54:42.539192 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:42.539149 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wvtd8/must-gather-zv9l5" event={"ID":"af41ba91-a560-4b2f-91b5-b55fcc848f23","Type":"ContainerStarted","Data":"2500920a052b78b63caf98d0157499986d4f704e4ee9ff629991ea80de2e8df9"} Jul 07 15:54:42.539696 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:42.539206 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wvtd8/must-gather-zv9l5" event={"ID":"af41ba91-a560-4b2f-91b5-b55fcc848f23","Type":"ContainerStarted","Data":"3cd0576d7031661c66ec7bedaf266b9662cd4cc2571da7a15dc5cafa103da71b"} Jul 07 15:54:42.556067 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:42.556011 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-wvtd8/must-gather-zv9l5" podStartSLOduration=1.579207653 podStartE2EDuration="2.555991279s" totalImagesPullingTime="976.783626ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:54:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:54:42.555386214 +0000 UTC m=+2500.562863666" watchObservedRunningTime="2026-07-07 15:54:42.555991279 +0000 UTC m=+2500.563468730" Jul 07 15:54:43.789464 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:43.789426 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-xt54w_473572dc-5da2-44c1-918d-a409370c493f/global-pull-secret-syncer/0.log" Jul 07 15:54:43.897720 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:43.897687 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-dvrfw_31fc8ff5-b255-4523-9758-bf21b0409efa/konnectivity-agent/0.log" Jul 07 15:54:44.004535 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:44.004507 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-11.ec2.internal_f3ea74c0531243ed68ab1cca77160244/haproxy/0.log" Jul 07 15:54:48.399157 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:48.399121 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tfcvf_2d88f62c-1bee-4cf7-a1e8-5d009e7920f4/extract/0.log" Jul 07 15:54:48.432214 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:48.432181 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tfcvf_2d88f62c-1bee-4cf7-a1e8-5d009e7920f4/util/0.log" Jul 07 15:54:48.468676 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:48.468641 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tfcvf_2d88f62c-1bee-4cf7-a1e8-5d009e7920f4/pull/0.log" Jul 07 15:54:48.526210 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:48.526136 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jmnk5_6b8fcac7-bd9c-48f1-819a-918c59419cce/extract/0.log" Jul 07 15:54:48.580384 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:48.580347 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jmnk5_6b8fcac7-bd9c-48f1-819a-918c59419cce/util/0.log" Jul 07 15:54:48.631327 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:48.631293 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jmnk5_6b8fcac7-bd9c-48f1-819a-918c59419cce/pull/0.log" Jul 07 15:54:48.677861 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:48.677784 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vhnst_2e0bce97-2257-42da-83d3-e2d9fe891e0e/extract/0.log" Jul 07 15:54:48.714728 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:48.714692 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vhnst_2e0bce97-2257-42da-83d3-e2d9fe891e0e/util/0.log" Jul 07 15:54:48.766169 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:48.766090 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vhnst_2e0bce97-2257-42da-83d3-e2d9fe891e0e/pull/0.log" Jul 07 15:54:48.819856 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:48.819820 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1flzkq_f56fd4a1-89f3-476f-98dc-19005651962b/extract/0.log" Jul 07 15:54:48.850771 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:48.850745 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1flzkq_f56fd4a1-89f3-476f-98dc-19005651962b/util/0.log" Jul 07 15:54:48.885455 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:48.885424 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1flzkq_f56fd4a1-89f3-476f-98dc-19005651962b/pull/0.log" Jul 07 15:54:49.328607 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:49.325407 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-fdd655b5-86qvh_07d18ca8-06d5-4526-9639-bb4db61ff4f6/authorino/0.log" Jul 07 15:54:49.396982 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:49.396946 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-bxk7h_2cc62d89-fbb0-4e39-95f0-ab676d24f36b/manager/0.log" Jul 07 15:54:49.475306 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:49.475268 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-pwpzp_372030ff-029f-461b-8199-5c810f934692/manager/0.log" Jul 07 15:54:49.515464 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:49.515418 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-s4f5c_36a474cd-2744-486e-82fe-3a75834a5658/kuadrant-console-plugin/0.log" Jul 07 15:54:49.584752 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:49.584644 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-qg79z_79f7e333-21f4-430c-98dc-cdcc9880ba4c/registry-server/0.log" Jul 07 15:54:49.822359 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:49.820838 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-7ksmp_db31a5e5-1e8c-4309-90f6-8debeb83bc31/manager/0.log" Jul 07 15:54:49.869318 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:49.869230 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-xf9m5_1b5f5855-5c60-45a1-a6a1-bffae8500df6/limitador/0.log" Jul 07 15:54:49.982686 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:49.982651 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-v75tr_6d747b0b-7182-4179-b7db-7864774c0b76/manager/0.log" Jul 07 15:54:51.661377 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:51.661322 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f4f360be-ae5c-4ef3-a09e-7d7f993cbc42/alertmanager/0.log" Jul 07 15:54:51.692282 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:51.692171 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f4f360be-ae5c-4ef3-a09e-7d7f993cbc42/config-reloader/0.log" Jul 07 15:54:51.722841 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:51.722796 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f4f360be-ae5c-4ef3-a09e-7d7f993cbc42/kube-rbac-proxy-web/0.log" Jul 07 15:54:51.754104 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:51.754058 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f4f360be-ae5c-4ef3-a09e-7d7f993cbc42/kube-rbac-proxy/0.log" Jul 07 15:54:51.785741 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:51.785643 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f4f360be-ae5c-4ef3-a09e-7d7f993cbc42/kube-rbac-proxy-metric/0.log" Jul 07 15:54:51.826844 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:51.826806 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f4f360be-ae5c-4ef3-a09e-7d7f993cbc42/prom-label-proxy/0.log" Jul 07 15:54:51.856672 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:51.856607 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f4f360be-ae5c-4ef3-a09e-7d7f993cbc42/init-config-reloader/0.log" Jul 07 15:54:51.910650 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:51.910576 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-ddv8m_8cb2a4ff-480a-47b8-b781-f695be5d74a1/cluster-monitoring-operator/0.log" Jul 07 15:54:52.032284 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:52.032253 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6894f65b55-6lfnz_c8d6306d-fa3e-49e3-964e-9666a18a5477/metrics-server/0.log" Jul 07 15:54:52.093381 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:52.093349 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bthjj_7745743b-cc63-477b-b67c-5f2f81872dcc/node-exporter/0.log" Jul 07 15:54:52.118613 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:52.118574 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bthjj_7745743b-cc63-477b-b67c-5f2f81872dcc/kube-rbac-proxy/0.log" Jul 07 15:54:52.149365 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:52.149256 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bthjj_7745743b-cc63-477b-b67c-5f2f81872dcc/init-textfile/0.log" Jul 07 15:54:52.364245 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:52.364215 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-5pxlr_45876fcf-512b-4537-a4f3-6c16985e00f2/kube-rbac-proxy-main/0.log" Jul 07 15:54:52.388788 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:52.388755 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-5pxlr_45876fcf-512b-4537-a4f3-6c16985e00f2/kube-rbac-proxy-self/0.log" Jul 07 15:54:52.412213 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:52.412184 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-5pxlr_45876fcf-512b-4537-a4f3-6c16985e00f2/openshift-state-metrics/0.log" Jul 07 15:54:52.737142 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:52.737053 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-7qqd8_d73abd9e-6eb2-4aa5-8ce9-edaeac1b8721/prometheus-operator-admission-webhook/0.log" Jul 07 15:54:52.781969 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:52.781936 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5675fd947d-fm6jm_4e04181b-70b5-421d-8635-0b4abe99c717/telemeter-client/0.log" Jul 07 15:54:52.814195 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:52.814164 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5675fd947d-fm6jm_4e04181b-70b5-421d-8635-0b4abe99c717/reload/0.log" Jul 07 15:54:52.845148 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:52.845120 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5675fd947d-fm6jm_4e04181b-70b5-421d-8635-0b4abe99c717/kube-rbac-proxy/0.log" Jul 07 15:54:52.885666 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:52.885633 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-67656c78c5-x6f5l_2e8905ad-6c70-43de-aba9-19eda719d58b/thanos-query/0.log" Jul 07 15:54:52.915657 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:52.915614 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-67656c78c5-x6f5l_2e8905ad-6c70-43de-aba9-19eda719d58b/kube-rbac-proxy-web/0.log" Jul 07 15:54:52.951722 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:52.951683 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-67656c78c5-x6f5l_2e8905ad-6c70-43de-aba9-19eda719d58b/kube-rbac-proxy/0.log" Jul 07 15:54:52.980131 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:52.980105 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-67656c78c5-x6f5l_2e8905ad-6c70-43de-aba9-19eda719d58b/prom-label-proxy/0.log" Jul 07 15:54:53.013456 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:53.013382 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-67656c78c5-x6f5l_2e8905ad-6c70-43de-aba9-19eda719d58b/kube-rbac-proxy-rules/0.log" Jul 07 15:54:53.045507 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:53.045475 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-67656c78c5-x6f5l_2e8905ad-6c70-43de-aba9-19eda719d58b/kube-rbac-proxy-metrics/0.log" Jul 07 15:54:53.639817 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:53.639786 2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-wvtd8/perf-node-gather-daemonset-gljlx"] Jul 07 15:54:53.646132 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:53.646095 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wvtd8/perf-node-gather-daemonset-gljlx" Jul 07 15:54:53.655226 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:53.655186 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wvtd8/perf-node-gather-daemonset-gljlx"] Jul 07 15:54:53.777351 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:53.777275 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/99fee951-aa9b-452b-a75c-588c6abcfbd2-proc\") pod \"perf-node-gather-daemonset-gljlx\" (UID: \"99fee951-aa9b-452b-a75c-588c6abcfbd2\") " pod="openshift-must-gather-wvtd8/perf-node-gather-daemonset-gljlx" Jul 07 15:54:53.777351 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:53.777330 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fqff\" (UniqueName: \"kubernetes.io/projected/99fee951-aa9b-452b-a75c-588c6abcfbd2-kube-api-access-5fqff\") pod \"perf-node-gather-daemonset-gljlx\" (UID: \"99fee951-aa9b-452b-a75c-588c6abcfbd2\") " pod="openshift-must-gather-wvtd8/perf-node-gather-daemonset-gljlx" Jul 07 15:54:53.777831 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:53.777388 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/99fee951-aa9b-452b-a75c-588c6abcfbd2-lib-modules\") pod \"perf-node-gather-daemonset-gljlx\" (UID: \"99fee951-aa9b-452b-a75c-588c6abcfbd2\") " pod="openshift-must-gather-wvtd8/perf-node-gather-daemonset-gljlx" Jul 07 15:54:53.777831 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:53.777406 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/99fee951-aa9b-452b-a75c-588c6abcfbd2-podres\") pod \"perf-node-gather-daemonset-gljlx\" (UID: \"99fee951-aa9b-452b-a75c-588c6abcfbd2\") " pod="openshift-must-gather-wvtd8/perf-node-gather-daemonset-gljlx" Jul 07 15:54:53.777831 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:53.777530 2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/99fee951-aa9b-452b-a75c-588c6abcfbd2-sys\") pod \"perf-node-gather-daemonset-gljlx\" (UID: \"99fee951-aa9b-452b-a75c-588c6abcfbd2\") " pod="openshift-must-gather-wvtd8/perf-node-gather-daemonset-gljlx" Jul 07 15:54:53.878189 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:53.878142 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/99fee951-aa9b-452b-a75c-588c6abcfbd2-proc\") pod \"perf-node-gather-daemonset-gljlx\" (UID: \"99fee951-aa9b-452b-a75c-588c6abcfbd2\") " pod="openshift-must-gather-wvtd8/perf-node-gather-daemonset-gljlx" Jul 07 15:54:53.878391 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:53.878218 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fqff\" (UniqueName: \"kubernetes.io/projected/99fee951-aa9b-452b-a75c-588c6abcfbd2-kube-api-access-5fqff\") pod \"perf-node-gather-daemonset-gljlx\" (UID: \"99fee951-aa9b-452b-a75c-588c6abcfbd2\") " pod="openshift-must-gather-wvtd8/perf-node-gather-daemonset-gljlx" Jul 07 15:54:53.878391 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:53.878263 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/99fee951-aa9b-452b-a75c-588c6abcfbd2-lib-modules\") pod \"perf-node-gather-daemonset-gljlx\" (UID: \"99fee951-aa9b-452b-a75c-588c6abcfbd2\") " pod="openshift-must-gather-wvtd8/perf-node-gather-daemonset-gljlx" Jul 07 15:54:53.878391 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:53.878291 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/99fee951-aa9b-452b-a75c-588c6abcfbd2-podres\") pod \"perf-node-gather-daemonset-gljlx\" (UID: \"99fee951-aa9b-452b-a75c-588c6abcfbd2\") " pod="openshift-must-gather-wvtd8/perf-node-gather-daemonset-gljlx" Jul 07 15:54:53.878391 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:53.878292 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/99fee951-aa9b-452b-a75c-588c6abcfbd2-proc\") pod \"perf-node-gather-daemonset-gljlx\" (UID: \"99fee951-aa9b-452b-a75c-588c6abcfbd2\") " pod="openshift-must-gather-wvtd8/perf-node-gather-daemonset-gljlx" Jul 07 15:54:53.878633 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:53.878431 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/99fee951-aa9b-452b-a75c-588c6abcfbd2-podres\") pod \"perf-node-gather-daemonset-gljlx\" (UID: \"99fee951-aa9b-452b-a75c-588c6abcfbd2\") " pod="openshift-must-gather-wvtd8/perf-node-gather-daemonset-gljlx" Jul 07 15:54:53.878633 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:53.878430 2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/99fee951-aa9b-452b-a75c-588c6abcfbd2-sys\") pod \"perf-node-gather-daemonset-gljlx\" (UID: \"99fee951-aa9b-452b-a75c-588c6abcfbd2\") " pod="openshift-must-gather-wvtd8/perf-node-gather-daemonset-gljlx" Jul 07 15:54:53.878633 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:53.878474 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/99fee951-aa9b-452b-a75c-588c6abcfbd2-sys\") pod \"perf-node-gather-daemonset-gljlx\" (UID: \"99fee951-aa9b-452b-a75c-588c6abcfbd2\") " pod="openshift-must-gather-wvtd8/perf-node-gather-daemonset-gljlx" Jul 07 15:54:53.878633 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:53.878566 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/99fee951-aa9b-452b-a75c-588c6abcfbd2-lib-modules\") pod \"perf-node-gather-daemonset-gljlx\" (UID: \"99fee951-aa9b-452b-a75c-588c6abcfbd2\") " pod="openshift-must-gather-wvtd8/perf-node-gather-daemonset-gljlx" Jul 07 15:54:53.888907 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:53.888877 2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fqff\" (UniqueName: \"kubernetes.io/projected/99fee951-aa9b-452b-a75c-588c6abcfbd2-kube-api-access-5fqff\") pod \"perf-node-gather-daemonset-gljlx\" (UID: \"99fee951-aa9b-452b-a75c-588c6abcfbd2\") " pod="openshift-must-gather-wvtd8/perf-node-gather-daemonset-gljlx" Jul 07 15:54:53.960422 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:53.960317 2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wvtd8/perf-node-gather-daemonset-gljlx" Jul 07 15:54:54.147512 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:54.147483 2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wvtd8/perf-node-gather-daemonset-gljlx"] Jul 07 15:54:54.308295 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:54.308264 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-zh2mx_4f1ac273-713a-4bb6-94d4-4ec09090a194/networking-console-plugin/0.log" Jul 07 15:54:54.647797 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:54.647752 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wvtd8/perf-node-gather-daemonset-gljlx" event={"ID":"99fee951-aa9b-452b-a75c-588c6abcfbd2","Type":"ContainerStarted","Data":"ad41cc96e927438b922d884eb7bf38f69d06514a47b062206da0b5b7f3b7aa49"} Jul 07 15:54:54.648013 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:54.647823 2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wvtd8/perf-node-gather-daemonset-gljlx" event={"ID":"99fee951-aa9b-452b-a75c-588c6abcfbd2","Type":"ContainerStarted","Data":"10478d27f0fc57af39916f4a986673f8048766864158d520cfd0eef391951294"} Jul 07 15:54:54.648013 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:54.647857 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-wvtd8/perf-node-gather-daemonset-gljlx" Jul 07 15:54:54.668886 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:54.668838 2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-wvtd8/perf-node-gather-daemonset-gljlx" podStartSLOduration=1.66882384 podStartE2EDuration="1.66882384s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:54:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:54:54.668133831 +0000 UTC m=+2512.675611284" watchObservedRunningTime="2026-07-07 15:54:54.66882384 +0000 UTC m=+2512.676301290" Jul 07 15:54:54.890717 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:54.890684 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kb4ml_3301f6ef-6671-4755-9047-16959836232c/console-operator/2.log" Jul 07 15:54:54.899604 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:54.899492 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-kb4ml_3301f6ef-6671-4755-9047-16959836232c/console-operator/3.log" Jul 07 15:54:55.442742 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:55.442689 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-85746cdfd6-nv87t_dd10de4f-55c8-44a6-980b-12bd90d2c9b8/console/0.log" Jul 07 15:54:56.128114 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:56.128085 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-9f95m_f18cae1a-0479-4be9-a321-e00e056a4786/volume-data-source-validator/0.log" Jul 07 15:54:57.370193 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:57.370166 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dgd58_42fc06ec-88cb-4308-9070-c4defaf37385/dns/0.log" Jul 07 15:54:57.395445 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:57.395418 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dgd58_42fc06ec-88cb-4308-9070-c4defaf37385/kube-rbac-proxy/0.log" Jul 07 15:54:57.506858 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:57.506828 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-t8hcc_da22234a-c3b5-450f-804c-6b6f6b238d3b/dns-node-resolver/0.log" Jul 07 15:54:58.337956 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:58.337929 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zd2vk_80a72d31-7f06-45f3-8c76-9dd94fdb476a/node-ca/0.log" Jul 07 15:54:59.561714 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:59.561678 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5c6c759bc-5c7v6_01d82b99-af4a-4369-9164-360e15d583ac/kube-auth-proxy/0.log" Jul 07 15:54:59.796238 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:54:59.796209 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-64478b6d7c-vxvp5_77ec27ef-75d0-454a-9cd5-803a776c4985/router/0.log" Jul 07 15:55:00.609118 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:55:00.609088 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7smq7_4d2b93c0-1bf2-4ca8-913f-6c1e3d2e60ac/serve-healthcheck-canary/0.log" Jul 07 15:55:00.666913 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:55:00.666759 2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-wvtd8/perf-node-gather-daemonset-gljlx" Jul 07 15:55:01.236529 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:55:01.236496 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-ntm4v_6bd34c29-c215-4974-98fd-7759cb30f6b8/insights-operator/0.log" Jul 07 15:55:01.240728 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:55:01.240702 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-ntm4v_6bd34c29-c215-4974-98fd-7759cb30f6b8/insights-operator/1.log" Jul 07 15:55:01.420693 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:55:01.420636 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-v9726_85486152-4b69-4aac-948c-8234b1028748/kube-rbac-proxy/0.log" Jul 07 15:55:01.448341 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:55:01.448316 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-v9726_85486152-4b69-4aac-948c-8234b1028748/exporter/0.log" Jul 07 15:55:01.476289 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:55:01.476258 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-v9726_85486152-4b69-4aac-948c-8234b1028748/extractor/0.log" Jul 07 15:55:04.399220 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:55:04.399142 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-t97r8_ff832223-8f66-4299-94e1-35b79a4ccc22/manager/0.log" Jul 07 15:55:04.448777 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:55:04.448727 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5c6c759bc-5c7v6_01d82b99-af4a-4369-9164-360e15d583ac/kube-auth-proxy/0.log" Jul 07 15:55:04.645627 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:55:04.645573 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-9f7tp_27ffe7dc-0f8f-4d32-882b-6b0829ab71ad/manager/0.log" Jul 07 15:55:04.795454 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:55:04.795376 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-64478b6d7c-vxvp5_77ec27ef-75d0-454a-9cd5-803a776c4985/router/0.log" Jul 07 15:55:04.837639 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:55:04.837611 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29723970-m8rzm_5dc0b063-d8d0-44e6-a341-258a4b41d89f/cleanup/0.log" Jul 07 15:55:04.887153 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:55:04.887121 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29723985-lzwl4_15069e2e-354d-4d4e-a7e6-d5be9afc6def/cleanup/0.log" Jul 07 15:55:05.071301 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:55:05.071274 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-dc8f7_9b60c5f2-1af3-4b36-adcb-f1ade7775ca5/server/0.log" Jul 07 15:55:05.187626 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:55:05.187557 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-7fshj_e74b6418-6103-41d5-9828-988395d57e82/manager/0.log" Jul 07 15:55:05.489292 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:55:05.489263 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dcbf69668-wldx7_8779b525-5eb5-43f7-8168-ab732a67c10a/manager/0.log" Jul 07 15:55:05.529576 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:55:05.529549 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dcbf69668-wldx7_8779b525-5eb5-43f7-8168-ab732a67c10a/copy-manifests/0.log" Jul 07 15:55:06.211360 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:55:06.211330 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5c6c759bc-5c7v6_01d82b99-af4a-4369-9164-360e15d583ac/kube-auth-proxy/0.log" Jul 07 15:55:06.366472 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:55:06.366443 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-64478b6d7c-vxvp5_77ec27ef-75d0-454a-9cd5-803a776c4985/router/0.log" Jul 07 15:55:07.033141 ip-10-0-138-11 kubenswrapper[2549]: I0707 15:55:07.033107 2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-rkz2v_1d9ffb2d-b0a5-4619-80cc-89bbb669887d/openshift-lws-operator/0.log"