Jul 06 20:08:09.449197 ip-10-0-137-178 systemd[1]: Starting Kubernetes Kubelet... Jul 06 20:08:09.937019 ip-10-0-137-178 kubenswrapper[2551]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 20:08:09.937019 ip-10-0-137-178 kubenswrapper[2551]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 06 20:08:09.937019 ip-10-0-137-178 kubenswrapper[2551]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 20:08:09.937019 ip-10-0-137-178 kubenswrapper[2551]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 06 20:08:09.937019 ip-10-0-137-178 kubenswrapper[2551]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 20:08:09.938054 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.937904 2551 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 06 20:08:09.943331 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943315 2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 20:08:09.943331 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943330 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 20:08:09.943331 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943334 2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 20:08:09.943423 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943338 2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 20:08:09.943423 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943341 2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 20:08:09.943423 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943345 2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 20:08:09.943423 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943347 2551 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 20:08:09.943423 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943350 2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 20:08:09.943423 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943353 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 20:08:09.943423 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943356 2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 20:08:09.943423 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943359 2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 20:08:09.943423 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943363 2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 20:08:09.943423 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943365 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 20:08:09.943423 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943368 2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 20:08:09.943423 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943378 2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 20:08:09.943423 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943380 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 20:08:09.943423 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943383 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 20:08:09.943423 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943385 2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 20:08:09.943423 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943388 2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 20:08:09.943423 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943391 2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 20:08:09.943423 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943393 2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 20:08:09.943423 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943396 2551 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 20:08:09.943423 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943398 2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 20:08:09.943889 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943401 2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 20:08:09.943889 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943403 2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 20:08:09.943889 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943406 2551 feature_gate.go:328] unrecognized feature gate: Example Jul 06 20:08:09.943889 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943409 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 20:08:09.943889 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943411 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 20:08:09.943889 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943414 2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 20:08:09.943889 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943416 2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 20:08:09.943889 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943419 2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 20:08:09.943889 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943422 2551 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 20:08:09.943889 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943424 2551 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 20:08:09.943889 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943427 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 20:08:09.943889 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943430 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 20:08:09.943889 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943433 2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 20:08:09.943889 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943435 2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 20:08:09.943889 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943438 2551 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 20:08:09.943889 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943440 2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 20:08:09.943889 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943443 2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 20:08:09.943889 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943446 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 20:08:09.943889 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943448 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 20:08:09.943889 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943451 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 20:08:09.944379 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943453 2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 20:08:09.944379 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943455 2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 20:08:09.944379 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943458 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 20:08:09.944379 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943460 2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 20:08:09.944379 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943462 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 20:08:09.944379 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943465 2551 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 20:08:09.944379 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943467 2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 20:08:09.944379 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943470 2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 20:08:09.944379 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943472 2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 20:08:09.944379 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943475 2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 20:08:09.944379 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943478 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 20:08:09.944379 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943480 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 20:08:09.944379 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943483 2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 20:08:09.944379 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943486 2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 20:08:09.944379 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943489 2551 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 20:08:09.944379 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943491 2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 20:08:09.944379 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943494 2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 20:08:09.944379 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943497 2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 20:08:09.944379 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943499 2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 20:08:09.944379 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943502 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 20:08:09.944912 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943504 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 20:08:09.944912 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943507 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 20:08:09.944912 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943509 2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 20:08:09.944912 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943512 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 20:08:09.944912 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943514 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 20:08:09.944912 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943516 2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 20:08:09.944912 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943519 2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 20:08:09.944912 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943521 2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 20:08:09.944912 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943525 2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 20:08:09.944912 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943528 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 20:08:09.944912 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943530 2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 20:08:09.944912 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943533 2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 20:08:09.944912 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943535 2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 20:08:09.944912 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943538 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 20:08:09.944912 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943540 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 20:08:09.944912 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943543 2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 20:08:09.944912 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943545 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 20:08:09.944912 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943548 2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 20:08:09.944912 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943550 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 20:08:09.945388 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943553 2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 20:08:09.945388 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943555 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 20:08:09.945388 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943557 2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 20:08:09.945388 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943560 2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 20:08:09.945388 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943562 2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 20:08:09.945388 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943565 2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 20:08:09.945388 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943568 2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 20:08:09.945388 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943571 2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 20:08:09.945388 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943574 2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 20:08:09.945388 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943578 2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 20:08:09.945388 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943581 2551 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 20:08:09.945388 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943584 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 20:08:09.945388 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943587 2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 20:08:09.945388 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943592 2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 20:08:09.945388 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943597 2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 20:08:09.945388 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943601 2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 20:08:09.945388 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.943605 2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 20:08:09.945388 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944317 2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 20:08:09.945388 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944323 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 20:08:09.945841 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944326 2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 20:08:09.945841 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944329 2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 20:08:09.945841 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944332 2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 20:08:09.945841 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944338 2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 20:08:09.945841 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944341 2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 20:08:09.945841 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944344 2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 20:08:09.945841 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944347 2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 20:08:09.945841 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944349 2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 20:08:09.945841 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944352 2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 20:08:09.945841 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944355 2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 20:08:09.945841 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944357 2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 20:08:09.945841 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944360 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 20:08:09.945841 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944362 2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 20:08:09.945841 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944365 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 20:08:09.945841 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944367 2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 20:08:09.945841 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944370 2551 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 20:08:09.945841 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944372 2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 20:08:09.945841 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944375 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 20:08:09.945841 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944377 2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 20:08:09.945841 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944380 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 20:08:09.946353 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944383 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 20:08:09.946353 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944385 2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 20:08:09.946353 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944388 2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 20:08:09.946353 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944390 2551 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 20:08:09.946353 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944393 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 20:08:09.946353 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944396 2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 20:08:09.946353 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944398 2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 20:08:09.946353 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944401 2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 20:08:09.946353 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944403 2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 20:08:09.946353 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944406 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 20:08:09.946353 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944408 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 20:08:09.946353 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944410 2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 20:08:09.946353 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944413 2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 20:08:09.946353 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944415 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 20:08:09.946353 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944418 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 20:08:09.946353 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944421 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 20:08:09.946353 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944424 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 20:08:09.946353 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944427 2551 feature_gate.go:328] unrecognized feature gate: Example Jul 06 20:08:09.946353 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944429 2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 20:08:09.946353 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944432 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 20:08:09.946853 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944434 2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 20:08:09.946853 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944437 2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 20:08:09.946853 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944439 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 20:08:09.946853 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944442 2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 20:08:09.946853 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944444 2551 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 20:08:09.946853 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944447 2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 20:08:09.946853 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944449 2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 20:08:09.946853 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944451 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 20:08:09.946853 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944454 2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 20:08:09.946853 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944459 2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 20:08:09.946853 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944462 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 20:08:09.946853 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944465 2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 20:08:09.946853 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944469 2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 20:08:09.946853 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944472 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 20:08:09.946853 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944475 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 20:08:09.946853 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944477 2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 20:08:09.946853 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944479 2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 20:08:09.946853 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944482 2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 20:08:09.946853 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944485 2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 20:08:09.947328 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944487 2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 20:08:09.947328 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944490 2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 20:08:09.947328 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944492 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 20:08:09.947328 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944494 2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 20:08:09.947328 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944497 2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 20:08:09.947328 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944499 2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 20:08:09.947328 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944502 2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 20:08:09.947328 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944504 2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 20:08:09.947328 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944508 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 20:08:09.947328 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944510 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 20:08:09.947328 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944514 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 20:08:09.947328 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944516 2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 20:08:09.947328 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944519 2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 20:08:09.947328 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944523 2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 20:08:09.947328 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944526 2551 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 20:08:09.947328 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944529 2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 20:08:09.947328 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944532 2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 20:08:09.947328 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944535 2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 20:08:09.947328 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944537 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 20:08:09.947790 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944540 2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 20:08:09.947790 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944543 2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 20:08:09.947790 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944545 2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 20:08:09.947790 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944548 2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 20:08:09.947790 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944551 2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 20:08:09.947790 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944554 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 20:08:09.947790 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944557 2551 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 20:08:09.947790 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944559 2551 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 20:08:09.947790 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944563 2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 20:08:09.947790 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944566 2551 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 20:08:09.947790 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944569 2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 20:08:09.947790 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944571 2551 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 20:08:09.947790 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944574 2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 20:08:09.947790 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944577 2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 20:08:09.947790 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944579 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 20:08:09.947790 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944582 2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 20:08:09.947790 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944584 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 20:08:09.947790 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944586 2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 20:08:09.947790 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.944589 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 20:08:09.947790 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945021 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 20:08:09.948300 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945027 2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 20:08:09.948300 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945030 2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 20:08:09.948300 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945033 2551 feature_gate.go:328] unrecognized feature gate: Example Jul 06 20:08:09.948300 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945043 2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 20:08:09.948300 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945046 2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 20:08:09.948300 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945049 2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 20:08:09.948300 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945052 2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 20:08:09.948300 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945054 2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 20:08:09.948300 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945057 2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 20:08:09.948300 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945060 2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 20:08:09.948300 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945062 2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 20:08:09.948300 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945078 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 20:08:09.948300 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945082 2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 20:08:09.948300 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945085 2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 20:08:09.948300 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945088 2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 20:08:09.948300 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945091 2551 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 20:08:09.948300 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945094 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 20:08:09.948300 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945097 2551 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 20:08:09.948300 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945100 2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 20:08:09.948300 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945103 2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 20:08:09.948795 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945106 2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 20:08:09.948795 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945109 2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 20:08:09.948795 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945111 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 20:08:09.948795 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945114 2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 20:08:09.948795 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945117 2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 20:08:09.948795 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945119 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 20:08:09.948795 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945122 2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 20:08:09.948795 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945124 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 20:08:09.948795 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945128 2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 20:08:09.948795 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945131 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 20:08:09.948795 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945134 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 20:08:09.948795 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945136 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 20:08:09.948795 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945141 2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 20:08:09.948795 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945144 2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 20:08:09.948795 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945146 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 20:08:09.948795 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945149 2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 20:08:09.948795 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945152 2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 20:08:09.948795 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945154 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 20:08:09.948795 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945157 2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 20:08:09.949321 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945160 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 20:08:09.949321 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945162 2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 20:08:09.949321 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945165 2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 20:08:09.949321 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945168 2551 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 20:08:09.949321 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945170 2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 20:08:09.949321 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945173 2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 20:08:09.949321 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945175 2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 20:08:09.949321 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945178 2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 20:08:09.949321 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945181 2551 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 20:08:09.949321 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945184 2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 20:08:09.949321 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945186 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 20:08:09.949321 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945189 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 20:08:09.949321 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945191 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 20:08:09.949321 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945194 2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 20:08:09.949321 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945196 2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 20:08:09.949321 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945199 2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 20:08:09.949321 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945201 2551 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 20:08:09.949321 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945203 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 20:08:09.949321 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945206 2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 20:08:09.949321 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945208 2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 20:08:09.949833 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945211 2551 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 20:08:09.949833 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945215 2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 20:08:09.949833 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945218 2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 20:08:09.949833 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945221 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 20:08:09.949833 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945224 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 20:08:09.949833 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945228 2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 20:08:09.949833 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945230 2551 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 20:08:09.949833 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945233 2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 20:08:09.949833 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945235 2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 20:08:09.949833 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945238 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 20:08:09.949833 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945241 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 20:08:09.949833 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945244 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 20:08:09.949833 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945246 2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 20:08:09.949833 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945249 2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 20:08:09.949833 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945251 2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 20:08:09.949833 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945254 2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 20:08:09.949833 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945256 2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 20:08:09.949833 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945259 2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 20:08:09.949833 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945261 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 20:08:09.950315 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945264 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 20:08:09.950315 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945266 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 20:08:09.950315 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945269 2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 20:08:09.950315 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945272 2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 20:08:09.950315 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945274 2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 20:08:09.950315 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945277 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 20:08:09.950315 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945280 2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 20:08:09.950315 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945282 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 20:08:09.950315 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945285 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 20:08:09.950315 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945288 2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 20:08:09.950315 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945290 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 20:08:09.950315 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945292 2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 20:08:09.950315 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945295 2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 20:08:09.950315 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945297 2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 20:08:09.950315 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945300 2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 20:08:09.950315 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945302 2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 20:08:09.950315 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945305 2551 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 20:08:09.950315 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945309 2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 20:08:09.950315 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945314 2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 20:08:09.950315 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.945317 2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 20:08:09.950817 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945397 2551 flags.go:64] FLAG: --address="0.0.0.0" Jul 06 20:08:09.950817 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945405 2551 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 06 20:08:09.950817 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945417 2551 flags.go:64] FLAG: --anonymous-auth="true" Jul 06 20:08:09.950817 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945421 2551 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 06 20:08:09.950817 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945427 2551 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 06 20:08:09.950817 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945431 2551 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 06 20:08:09.950817 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945435 2551 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 06 20:08:09.950817 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945439 2551 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 06 20:08:09.950817 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945442 2551 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 06 20:08:09.950817 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945445 2551 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 06 20:08:09.950817 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945449 2551 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 06 20:08:09.950817 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945452 2551 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 06 20:08:09.950817 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945455 2551 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 06 20:08:09.950817 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945458 2551 flags.go:64] FLAG: --cgroup-root="" Jul 06 20:08:09.950817 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945461 2551 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 06 20:08:09.950817 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945464 2551 flags.go:64] FLAG: --client-ca-file="" Jul 06 20:08:09.950817 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945467 2551 flags.go:64] FLAG: --cloud-provider="external" Jul 06 20:08:09.950817 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945470 2551 flags.go:64] FLAG: --cluster-dns="[]" Jul 06 20:08:09.950817 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945474 2551 flags.go:64] FLAG: --cluster-domain="" Jul 06 20:08:09.950817 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945477 2551 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 06 20:08:09.950817 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945480 2551 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 06 20:08:09.950817 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945484 2551 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 06 20:08:09.950817 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945488 2551 flags.go:64] FLAG: --container-log-max-files="5" Jul 06 20:08:09.950817 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945491 2551 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 06 20:08:09.951408 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945494 2551 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 06 20:08:09.951408 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945498 2551 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 06 20:08:09.951408 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945501 2551 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 06 20:08:09.951408 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945504 2551 flags.go:64] FLAG: --contention-profiling="false" Jul 06 20:08:09.951408 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945507 2551 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 06 20:08:09.951408 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945510 2551 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 06 20:08:09.951408 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945515 2551 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 06 20:08:09.951408 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945517 2551 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 06 20:08:09.951408 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945522 2551 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 06 20:08:09.951408 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945525 2551 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 06 20:08:09.951408 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945527 2551 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 06 20:08:09.951408 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945530 2551 flags.go:64] FLAG: --enable-load-reader="false" Jul 06 20:08:09.951408 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945533 2551 flags.go:64] FLAG: --enable-server="true" Jul 06 20:08:09.951408 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945538 2551 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 06 20:08:09.951408 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945543 2551 flags.go:64] FLAG: --event-burst="100" Jul 06 20:08:09.951408 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945546 2551 flags.go:64] FLAG: --event-qps="50" Jul 06 20:08:09.951408 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945549 2551 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 06 20:08:09.951408 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945552 2551 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 06 20:08:09.951408 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945555 2551 flags.go:64] FLAG: --eviction-hard="" Jul 06 20:08:09.951408 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945558 2551 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 06 20:08:09.951408 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945561 2551 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 06 20:08:09.951408 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945564 2551 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 06 20:08:09.951408 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945567 2551 flags.go:64] FLAG: --eviction-soft="" Jul 06 20:08:09.951408 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945570 2551 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 06 20:08:09.951408 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945573 2551 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 06 20:08:09.952153 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945576 2551 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 06 20:08:09.952153 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945579 2551 flags.go:64] FLAG: --experimental-mounter-path="" Jul 06 20:08:09.952153 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945582 2551 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 06 20:08:09.952153 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945585 2551 flags.go:64] FLAG: --fail-swap-on="true" Jul 06 20:08:09.952153 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945587 2551 flags.go:64] FLAG: --feature-gates="" Jul 06 20:08:09.952153 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945591 2551 flags.go:64] FLAG: --file-check-frequency="20s" Jul 06 20:08:09.952153 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945594 2551 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 06 20:08:09.952153 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945597 2551 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 06 20:08:09.952153 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945600 2551 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 06 20:08:09.952153 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945603 2551 flags.go:64] FLAG: --healthz-port="10248" Jul 06 20:08:09.952153 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945606 2551 flags.go:64] FLAG: --help="false" Jul 06 20:08:09.952153 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945609 2551 flags.go:64] FLAG: --hostname-override="ip-10-0-137-178.ec2.internal" Jul 06 20:08:09.952153 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945612 2551 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 06 20:08:09.952153 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945616 2551 flags.go:64] FLAG: --http-check-frequency="20s" Jul 06 20:08:09.952153 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945618 2551 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 06 20:08:09.952153 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945621 2551 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 06 20:08:09.952153 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945625 2551 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 06 20:08:09.952153 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945628 2551 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 06 20:08:09.952153 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945631 2551 flags.go:64] FLAG: --image-service-endpoint="" Jul 06 20:08:09.952153 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945633 2551 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 06 20:08:09.952153 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945637 2551 flags.go:64] FLAG: --kube-api-burst="100" Jul 06 20:08:09.952153 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945641 2551 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 06 20:08:09.952153 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945644 2551 flags.go:64] FLAG: --kube-api-qps="50" Jul 06 20:08:09.952153 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945647 2551 flags.go:64] FLAG: --kube-reserved="" Jul 06 20:08:09.952786 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945650 2551 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 06 20:08:09.952786 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945653 2551 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 06 20:08:09.952786 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945656 2551 flags.go:64] FLAG: --kubelet-cgroups="" Jul 06 20:08:09.952786 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945659 2551 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 06 20:08:09.952786 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945661 2551 flags.go:64] FLAG: --lock-file="" Jul 06 20:08:09.952786 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945664 2551 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 06 20:08:09.952786 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945667 2551 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 06 20:08:09.952786 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945670 2551 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 06 20:08:09.952786 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945675 2551 flags.go:64] FLAG: --log-json-split-stream="false" Jul 06 20:08:09.952786 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945679 2551 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 06 20:08:09.952786 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945681 2551 flags.go:64] FLAG: --log-text-split-stream="false" Jul 06 20:08:09.952786 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945684 2551 flags.go:64] FLAG: --logging-format="text" Jul 06 20:08:09.952786 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945687 2551 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 06 20:08:09.952786 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945690 2551 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 06 20:08:09.952786 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945693 2551 flags.go:64] FLAG: --manifest-url="" Jul 06 20:08:09.952786 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945695 2551 flags.go:64] FLAG: --manifest-url-header="" Jul 06 20:08:09.952786 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945699 2551 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 06 20:08:09.952786 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945702 2551 flags.go:64] FLAG: --max-open-files="1000000" Jul 06 20:08:09.952786 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945706 2551 flags.go:64] FLAG: --max-pods="110" Jul 06 20:08:09.952786 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945709 2551 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 06 20:08:09.952786 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945712 2551 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 06 20:08:09.952786 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945717 2551 flags.go:64] FLAG: --memory-manager-policy="None" Jul 06 20:08:09.952786 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945719 2551 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 06 20:08:09.952786 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945722 2551 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 06 20:08:09.952786 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945726 2551 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 06 20:08:09.953473 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945728 2551 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 06 20:08:09.953473 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945735 2551 flags.go:64] FLAG: --node-status-max-images="50" Jul 06 20:08:09.953473 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945738 2551 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 06 20:08:09.953473 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945742 2551 flags.go:64] FLAG: --oom-score-adj="-999" Jul 06 20:08:09.953473 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945746 2551 flags.go:64] FLAG: --pod-cidr="" Jul 06 20:08:09.953473 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945749 2551 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 06 20:08:09.953473 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945754 2551 flags.go:64] FLAG: --pod-manifest-path="" Jul 06 20:08:09.953473 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945757 2551 flags.go:64] FLAG: --pod-max-pids="-1" Jul 06 20:08:09.953473 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945760 2551 flags.go:64] FLAG: --pods-per-core="0" Jul 06 20:08:09.953473 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945763 2551 flags.go:64] FLAG: --port="10250" Jul 06 20:08:09.953473 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945766 2551 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 06 20:08:09.953473 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945769 2551 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-077e6114ac11a9857" Jul 06 20:08:09.953473 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945772 2551 flags.go:64] FLAG: --qos-reserved="" Jul 06 20:08:09.953473 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945775 2551 flags.go:64] FLAG: --read-only-port="10255" Jul 06 20:08:09.953473 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945778 2551 flags.go:64] FLAG: --register-node="true" Jul 06 20:08:09.953473 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945781 2551 flags.go:64] FLAG: --register-with-taints="" Jul 06 20:08:09.953473 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945784 2551 flags.go:64] FLAG: --registry-burst="10" Jul 06 20:08:09.953473 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945787 2551 flags.go:64] FLAG: --registry-qps="5" Jul 06 20:08:09.953473 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945790 2551 flags.go:64] FLAG: --reserved-cpus="" Jul 06 20:08:09.953473 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945793 2551 flags.go:64] FLAG: --reserved-memory="" Jul 06 20:08:09.953473 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945796 2551 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 06 20:08:09.953473 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945799 2551 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 06 20:08:09.953473 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945802 2551 flags.go:64] FLAG: --rotate-certificates="false" Jul 06 20:08:09.953473 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945805 2551 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 06 20:08:09.953473 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945809 2551 flags.go:64] FLAG: --runonce="false" Jul 06 20:08:09.954101 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945811 2551 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 06 20:08:09.954101 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945814 2551 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 06 20:08:09.954101 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945817 2551 flags.go:64] FLAG: --seccomp-default="false" Jul 06 20:08:09.954101 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945821 2551 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 06 20:08:09.954101 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945824 2551 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 06 20:08:09.954101 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945827 2551 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 06 20:08:09.954101 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945830 2551 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 06 20:08:09.954101 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945833 2551 flags.go:64] FLAG: --storage-driver-password="root" Jul 06 20:08:09.954101 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945836 2551 flags.go:64] FLAG: --storage-driver-secure="false" Jul 06 20:08:09.954101 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945839 2551 flags.go:64] FLAG: --storage-driver-table="stats" Jul 06 20:08:09.954101 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945842 2551 flags.go:64] FLAG: --storage-driver-user="root" Jul 06 20:08:09.954101 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945847 2551 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 06 20:08:09.954101 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945850 2551 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 06 20:08:09.954101 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945853 2551 flags.go:64] FLAG: --system-cgroups="" Jul 06 20:08:09.954101 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945856 2551 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 06 20:08:09.954101 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945861 2551 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 06 20:08:09.954101 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945864 2551 flags.go:64] FLAG: --tls-cert-file="" Jul 06 20:08:09.954101 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945867 2551 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 06 20:08:09.954101 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945871 2551 flags.go:64] FLAG: --tls-min-version="" Jul 06 20:08:09.954101 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945874 2551 flags.go:64] FLAG: --tls-private-key-file="" Jul 06 20:08:09.954101 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945877 2551 flags.go:64] FLAG: --topology-manager-policy="none" Jul 06 20:08:09.954101 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945880 2551 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 06 20:08:09.954101 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945882 2551 flags.go:64] FLAG: --topology-manager-scope="container" Jul 06 20:08:09.954101 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945885 2551 flags.go:64] FLAG: --v="2" Jul 06 20:08:09.954101 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945903 2551 flags.go:64] FLAG: --version="false" Jul 06 20:08:09.954715 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945907 2551 flags.go:64] FLAG: --vmodule="" Jul 06 20:08:09.954715 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945911 2551 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 06 20:08:09.954715 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.945914 2551 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 06 20:08:09.954715 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946044 2551 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 20:08:09.954715 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946048 2551 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 20:08:09.954715 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946051 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 20:08:09.954715 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946054 2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 20:08:09.954715 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946057 2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 20:08:09.954715 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946061 2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 20:08:09.954715 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946065 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 20:08:09.954715 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946068 2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 20:08:09.954715 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946071 2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 20:08:09.954715 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946074 2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 20:08:09.954715 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946077 2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 20:08:09.954715 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946079 2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 20:08:09.954715 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946082 2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 20:08:09.954715 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946085 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 20:08:09.954715 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946087 2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 20:08:09.954715 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946092 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 20:08:09.954715 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946094 2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 20:08:09.955221 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946097 2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 20:08:09.955221 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946100 2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 20:08:09.955221 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946102 2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 20:08:09.955221 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946105 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 20:08:09.955221 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946108 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 20:08:09.955221 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946110 2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 20:08:09.955221 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946113 2551 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 20:08:09.955221 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946115 2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 20:08:09.955221 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946118 2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 20:08:09.955221 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946121 2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 20:08:09.955221 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946124 2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 20:08:09.955221 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946126 2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 20:08:09.955221 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946129 2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 20:08:09.955221 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946131 2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 20:08:09.955221 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946134 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 20:08:09.955221 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946136 2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 20:08:09.955221 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946139 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 20:08:09.955221 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946141 2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 20:08:09.955221 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946144 2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 20:08:09.955679 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946146 2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 20:08:09.955679 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946149 2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 20:08:09.955679 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946152 2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 20:08:09.955679 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946155 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 20:08:09.955679 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946157 2551 feature_gate.go:328] unrecognized feature gate: Example Jul 06 20:08:09.955679 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946160 2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 20:08:09.955679 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946163 2551 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 20:08:09.955679 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946165 2551 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 20:08:09.955679 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946168 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 20:08:09.955679 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946171 2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 20:08:09.955679 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946173 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 20:08:09.955679 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946177 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 20:08:09.955679 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946179 2551 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 20:08:09.955679 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946182 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 20:08:09.955679 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946184 2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 20:08:09.955679 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946188 2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 20:08:09.955679 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946192 2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 20:08:09.955679 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946195 2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 20:08:09.955679 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946198 2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 20:08:09.955679 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946201 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 20:08:09.956207 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946203 2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 20:08:09.956207 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946206 2551 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 20:08:09.956207 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946208 2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 20:08:09.956207 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946210 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 20:08:09.956207 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946213 2551 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 20:08:09.956207 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946215 2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 20:08:09.956207 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946218 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 20:08:09.956207 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946221 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 20:08:09.956207 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946223 2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 20:08:09.956207 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946226 2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 20:08:09.956207 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946228 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 20:08:09.956207 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946231 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 20:08:09.956207 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946233 2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 20:08:09.956207 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946236 2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 20:08:09.956207 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946238 2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 20:08:09.956207 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946241 2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 20:08:09.956207 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946243 2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 20:08:09.956207 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946246 2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 20:08:09.956207 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946248 2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 20:08:09.956207 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946251 2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 20:08:09.956691 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946253 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 20:08:09.956691 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946256 2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 20:08:09.956691 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946258 2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 20:08:09.956691 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946262 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 20:08:09.956691 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946264 2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 20:08:09.956691 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946267 2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 20:08:09.956691 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946269 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 20:08:09.956691 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946272 2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 20:08:09.956691 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946274 2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 20:08:09.956691 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946277 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 20:08:09.956691 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946279 2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 20:08:09.956691 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946282 2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 20:08:09.956691 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946286 2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 20:08:09.956691 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946289 2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 20:08:09.956691 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946291 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 20:08:09.956691 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946294 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 20:08:09.956691 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946297 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 20:08:09.956691 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946300 2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 20:08:09.956691 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946303 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 20:08:09.957176 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946305 2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 20:08:09.957176 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946307 2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 20:08:09.957176 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946310 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 20:08:09.957176 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.946313 2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 20:08:09.957176 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.946947 2551 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 20:08:09.957176 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.954008 2551 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 06 20:08:09.957176 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.954023 2551 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 06 20:08:09.957176 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.954044 2551 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 06 20:08:09.957176 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.954053 2551 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 06 20:08:09.957176 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954115 2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 20:08:09.957176 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954121 2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 20:08:09.957176 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954125 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 20:08:09.957176 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954128 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 20:08:09.957176 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954131 2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 20:08:09.957560 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954134 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 20:08:09.957560 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954137 2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 20:08:09.957560 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954139 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 20:08:09.957560 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954142 2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 20:08:09.957560 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954145 2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 20:08:09.957560 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954148 2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 20:08:09.957560 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954150 2551 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 20:08:09.957560 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954153 2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 20:08:09.957560 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954155 2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 20:08:09.957560 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954159 2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 20:08:09.957560 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954163 2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 20:08:09.957560 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954166 2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 20:08:09.957560 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954169 2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 20:08:09.957560 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954172 2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 20:08:09.957560 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954174 2551 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 20:08:09.957560 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954176 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 20:08:09.957560 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954179 2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 20:08:09.957560 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954181 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 20:08:09.957560 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954184 2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 20:08:09.958045 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954187 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 20:08:09.958045 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954189 2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 20:08:09.958045 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954192 2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 20:08:09.958045 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954194 2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 20:08:09.958045 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954197 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 20:08:09.958045 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954201 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 20:08:09.958045 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954204 2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 20:08:09.958045 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954207 2551 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 20:08:09.958045 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954209 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 20:08:09.958045 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954212 2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 20:08:09.958045 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954215 2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 20:08:09.958045 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954217 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 20:08:09.958045 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954220 2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 20:08:09.958045 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954222 2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 20:08:09.958045 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954225 2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 20:08:09.958045 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954228 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 20:08:09.958045 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954230 2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 20:08:09.958045 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954233 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 20:08:09.958045 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954235 2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 20:08:09.958566 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954237 2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 20:08:09.958566 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954240 2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 20:08:09.958566 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954242 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 20:08:09.958566 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954245 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 20:08:09.958566 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954247 2551 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 20:08:09.958566 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954250 2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 20:08:09.958566 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954253 2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 20:08:09.958566 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954255 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 20:08:09.958566 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954258 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 20:08:09.958566 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954261 2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 20:08:09.958566 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954263 2551 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 20:08:09.958566 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954267 2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 20:08:09.958566 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954271 2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 20:08:09.958566 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954273 2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 20:08:09.958566 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954276 2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 20:08:09.958566 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954278 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 20:08:09.958566 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954281 2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 20:08:09.958566 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954283 2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 20:08:09.958566 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954287 2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 20:08:09.958566 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954290 2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 20:08:09.959227 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954293 2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 20:08:09.959227 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954296 2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 20:08:09.959227 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954299 2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 20:08:09.959227 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954302 2551 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 20:08:09.959227 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954304 2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 20:08:09.959227 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954307 2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 20:08:09.959227 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954309 2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 20:08:09.959227 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954312 2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 20:08:09.959227 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954315 2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 20:08:09.959227 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954317 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 20:08:09.959227 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954320 2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 20:08:09.959227 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954322 2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 20:08:09.959227 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954325 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 20:08:09.959227 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954328 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 20:08:09.959227 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954330 2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 20:08:09.959227 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954333 2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 20:08:09.959227 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954335 2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 20:08:09.959227 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954338 2551 feature_gate.go:328] unrecognized feature gate: Example Jul 06 20:08:09.959227 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954341 2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 20:08:09.959227 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954343 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 20:08:09.959724 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954346 2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 20:08:09.959724 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954348 2551 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 20:08:09.959724 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954351 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 20:08:09.959724 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954355 2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 20:08:09.959724 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954358 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 20:08:09.959724 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954360 2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 20:08:09.959724 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954363 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 20:08:09.959724 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954366 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 20:08:09.959724 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954369 2551 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 20:08:09.959724 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954372 2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 20:08:09.959724 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954376 2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 20:08:09.959724 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954379 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 20:08:09.959724 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954382 2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 20:08:09.959724 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954384 2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 20:08:09.959724 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954387 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 20:08:09.959724 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954389 2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 20:08:09.960125 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.954394 2551 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 20:08:09.960125 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954523 2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 20:08:09.960125 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954528 2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 20:08:09.960125 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954531 2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 20:08:09.960125 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954534 2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 20:08:09.960125 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954537 2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 20:08:09.960125 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954540 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 20:08:09.960125 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954543 2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 20:08:09.960125 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954545 2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 20:08:09.960125 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954548 2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 20:08:09.960125 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954551 2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 20:08:09.960125 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954554 2551 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 20:08:09.960125 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954557 2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 20:08:09.960125 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954559 2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 20:08:09.960125 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954562 2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 20:08:09.960507 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954564 2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 20:08:09.960507 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954567 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 20:08:09.960507 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954569 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 20:08:09.960507 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954571 2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 20:08:09.960507 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954574 2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 20:08:09.960507 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954576 2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 20:08:09.960507 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954579 2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 20:08:09.960507 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954581 2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 20:08:09.960507 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954584 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 20:08:09.960507 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954586 2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 20:08:09.960507 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954589 2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 20:08:09.960507 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954592 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 20:08:09.960507 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954595 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 20:08:09.960507 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954597 2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 20:08:09.960507 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954602 2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 20:08:09.960507 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954605 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 20:08:09.960507 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954608 2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 20:08:09.960507 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954611 2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 20:08:09.960507 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954614 2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 20:08:09.960996 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954617 2551 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 20:08:09.960996 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954619 2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 20:08:09.960996 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954622 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 20:08:09.960996 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954626 2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 20:08:09.960996 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954629 2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 20:08:09.960996 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954632 2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 20:08:09.960996 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954635 2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 20:08:09.960996 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954639 2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 20:08:09.960996 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954642 2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 20:08:09.960996 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954645 2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 20:08:09.960996 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954647 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 20:08:09.960996 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954650 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 20:08:09.960996 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954652 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 20:08:09.960996 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954655 2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 20:08:09.960996 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954658 2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 20:08:09.960996 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954660 2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 20:08:09.960996 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954663 2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 20:08:09.960996 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954665 2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 20:08:09.961457 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954668 2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 20:08:09.961457 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954670 2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 20:08:09.961457 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954673 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 20:08:09.961457 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954675 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 20:08:09.961457 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954678 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 20:08:09.961457 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954680 2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 20:08:09.961457 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954683 2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 20:08:09.961457 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954686 2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 20:08:09.961457 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954688 2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 20:08:09.961457 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954691 2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 20:08:09.961457 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954693 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 20:08:09.961457 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954696 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 20:08:09.961457 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954698 2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 20:08:09.961457 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954701 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 20:08:09.961457 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954704 2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 20:08:09.961457 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954706 2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 20:08:09.961457 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954709 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 20:08:09.961457 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954711 2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 20:08:09.961457 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954713 2551 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 20:08:09.961457 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954716 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 20:08:09.962054 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954719 2551 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 20:08:09.962054 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954721 2551 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 20:08:09.962054 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954724 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 20:08:09.962054 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954727 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 20:08:09.962054 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954730 2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 20:08:09.962054 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954732 2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 20:08:09.962054 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954735 2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 20:08:09.962054 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954737 2551 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 20:08:09.962054 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954740 2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 20:08:09.962054 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954742 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 20:08:09.962054 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954744 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 20:08:09.962054 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954747 2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 20:08:09.962054 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954749 2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 20:08:09.962054 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954751 2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 20:08:09.962054 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954754 2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 20:08:09.962054 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954756 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 20:08:09.962054 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954759 2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 20:08:09.962054 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954761 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 20:08:09.962054 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954764 2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 20:08:09.962054 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954767 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 20:08:09.962963 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954769 2551 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 20:08:09.962963 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954772 2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 20:08:09.962963 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954775 2551 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 20:08:09.962963 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954777 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 20:08:09.962963 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954780 2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 20:08:09.962963 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954782 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 20:08:09.962963 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954785 2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 20:08:09.962963 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:09.954787 2551 feature_gate.go:328] unrecognized feature gate: Example Jul 06 20:08:09.962963 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.954792 2551 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 20:08:09.962963 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.954929 2551 server.go:962] "Client rotation is on, will bootstrap in background" Jul 06 20:08:09.962963 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.957410 2551 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 06 20:08:09.962963 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.958928 2551 server.go:1019] "Starting client certificate rotation" Jul 06 20:08:09.962963 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.959022 2551 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 20:08:09.962963 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.959058 2551 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 20:08:09.963497 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.961595 2551 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 20:08:09.965370 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.965342 2551 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 20:08:09.978088 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.978063 2551 log.go:25] "Validated CRI v1 runtime API" Jul 06 20:08:09.980228 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.980210 2551 log.go:25] "Validated CRI v1 image API" Jul 06 20:08:09.981143 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.981125 2551 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 06 20:08:09.983552 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.983518 2551 fs.go:135] Filesystem UUIDs: map[2a60281f-9b3c-44f0-a4d2-ec6f0990f48a:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 f725124c-74b2-45c7-82aa-ba234f56848f:/dev/nvme0n1p4] Jul 06 20:08:09.983635 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.983551 2551 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 06 20:08:09.988976 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.988835 2551 manager.go:216] Machine: {Timestamp:2026-07-06 20:08:09.98744656 +0000 UTC m=+0.402646371 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3108540 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec204be8d50699f8d1b8ed6d0b923684 SystemUUID:ec204be8-d506-99f8-d1b8-ed6d0b923684 BootID:5f0c637c-ad95-4c84-87d9-1bba7a3f704b Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:e0:81:cd:77:c5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:e0:81:cd:77:c5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ae:e4:1e:a8:59:0f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 06 20:08:09.988976 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.988967 2551 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 06 20:08:09.989154 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.989087 2551 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 06 20:08:09.992110 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.992081 2551 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 06 20:08:09.992268 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.992116 2551 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-178.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 06 20:08:09.992934 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.992922 2551 topology_manager.go:138] "Creating topology manager with none policy" Jul 06 20:08:09.992973 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.992937 2551 container_manager_linux.go:309] "Creating device plugin manager" Jul 06 20:08:09.992973 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.992945 2551 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 20:08:09.993476 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.993466 2551 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 20:08:09.995210 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.995199 2551 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 06 20:08:09.998095 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.998078 2551 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 20:08:09.998771 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.998760 2551 state_mem.go:36] "Initialized new in-memory state store" Jul 06 20:08:09.998904 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.998883 2551 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 06 20:08:09.998941 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.998932 2551 kubelet.go:486] "Attempting to sync node with API server" Jul 06 20:08:09.998976 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.998954 2551 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 06 20:08:09.998976 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.998966 2551 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 06 20:08:09.998976 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.998976 2551 kubelet.go:398] "Adding apiserver pod source" Jul 06 20:08:09.999058 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.998986 2551 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 06 20:08:09.999358 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.999345 2551 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 20:08:09.999404 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:09.999363 2551 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 20:08:10.001831 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.001808 2551 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 06 20:08:10.004833 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.004808 2551 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 06 20:08:10.004933 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.004859 2551 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 06 20:08:10.005942 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.005928 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 06 20:08:10.006006 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.005947 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 06 20:08:10.006006 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.005953 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 06 20:08:10.006006 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.005959 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 06 20:08:10.006006 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.005968 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 06 20:08:10.006006 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.005974 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 06 20:08:10.006006 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.005982 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 06 20:08:10.006006 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.005987 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 06 20:08:10.006006 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.005993 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 06 20:08:10.006006 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.005999 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 06 20:08:10.006006 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.006007 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 06 20:08:10.006265 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.006013 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 06 20:08:10.006764 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.006755 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 06 20:08:10.006764 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.006763 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 06 20:08:10.009924 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.009909 2551 server.go:1268] "Started kubelet" Jul 06 20:08:10.010078 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.010022 2551 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 06 20:08:10.010583 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.010560 2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-178.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 06 20:08:10.010697 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.009998 2551 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 06 20:08:10.010752 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.010721 2551 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 06 20:08:10.010845 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:10.010823 2551 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 06 20:08:10.010886 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:10.010821 2551 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-178.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 06 20:08:10.010853 ip-10-0-137-178 systemd[1]: Started Kubernetes Kubelet. Jul 06 20:08:10.012002 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.011987 2551 server.go:310] "Adding debug handlers to kubelet server" Jul 06 20:08:10.014168 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.014152 2551 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 06 20:08:10.016682 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:10.015653 2551 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-178.ec2.internal.18bfcc3166a99e13 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-178.ec2.internal,UID:ip-10-0-137-178.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-178.ec2.internal,},FirstTimestamp:2026-07-06 20:08:10.009861651 +0000 UTC m=+0.425061448,LastTimestamp:2026-07-06 20:08:10.009861651 +0000 UTC m=+0.425061448,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-178.ec2.internal,}" Jul 06 20:08:10.016880 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:10.016865 2551 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 06 20:08:10.022470 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.022457 2551 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 06 20:08:10.022537 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.022478 2551 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 06 20:08:10.022587 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:10.022549 2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-178.ec2.internal\" not found" Jul 06 20:08:10.023135 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.023116 2551 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 06 20:08:10.023135 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.023119 2551 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 06 20:08:10.023270 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.023147 2551 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 06 20:08:10.023270 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.023228 2551 reconstruct.go:97] "Volume reconstruction finished" Jul 06 20:08:10.023270 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.023236 2551 reconciler.go:29] "Reconciler: start to sync state" Jul 06 20:08:10.023393 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.023350 2551 factory.go:55] Registering systemd factory Jul 06 20:08:10.023393 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.023365 2551 factory.go:223] Registration of the systemd container factory successfully Jul 06 20:08:10.023664 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.023644 2551 factory.go:153] Registering CRI-O factory Jul 06 20:08:10.023664 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.023661 2551 factory.go:223] Registration of the crio container factory successfully Jul 06 20:08:10.023800 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.023705 2551 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 06 20:08:10.023800 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.023723 2551 factory.go:103] Registering Raw factory Jul 06 20:08:10.023800 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.023733 2551 manager.go:1196] Started watching for new ooms in manager Jul 06 20:08:10.024120 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.024105 2551 manager.go:349] Starting recovery of all containers Jul 06 20:08:10.024570 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:10.024539 2551 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 06 20:08:10.024672 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:10.024646 2551 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-137-178.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 06 20:08:10.035660 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.035632 2551 manager.go:354] Recovery completed Jul 06 20:08:10.040882 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.040866 2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 20:08:10.042576 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.042559 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-178.ec2.internal" event="NodeHasSufficientMemory" Jul 06 20:08:10.042636 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.042593 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-178.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 20:08:10.042636 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.042603 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-178.ec2.internal" event="NodeHasSufficientPID" Jul 06 20:08:10.043122 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.043106 2551 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 06 20:08:10.043122 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.043120 2551 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 06 20:08:10.043221 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.043126 2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 20:08:10.043221 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.043136 2551 state_mem.go:36] "Initialized new in-memory state store" Jul 06 20:08:10.043722 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.043708 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-178.ec2.internal" event="NodeHasSufficientMemory" Jul 06 20:08:10.043769 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.043734 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-178.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 20:08:10.043769 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.043746 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-178.ec2.internal" event="NodeHasSufficientPID" Jul 06 20:08:10.044405 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.044389 2551 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4b6fd" Jul 06 20:08:10.044463 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.044414 2551 policy_none.go:49] "None policy: Start" Jul 06 20:08:10.044463 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.044427 2551 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 06 20:08:10.044463 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.044438 2551 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 06 20:08:10.045456 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.045445 2551 policy_none.go:47] "Start" Jul 06 20:08:10.051577 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:10.051405 2551 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-178.ec2.internal.18bfcc31689cd3f0 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-178.ec2.internal,UID:ip-10-0-137-178.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-137-178.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-137-178.ec2.internal,},FirstTimestamp:2026-07-06 20:08:10.042577904 +0000 UTC m=+0.457777703,LastTimestamp:2026-07-06 20:08:10.042577904 +0000 UTC m=+0.457777703,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-178.ec2.internal,}" Jul 06 20:08:10.053570 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.053554 2551 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4b6fd" Jul 06 20:08:10.095017 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.081424 2551 manager.go:337] "Starting Device Plugin manager" Jul 06 20:08:10.095017 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:10.081452 2551 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 06 20:08:10.095017 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.081462 2551 server.go:87] "Starting device plugin registration server" Jul 06 20:08:10.095017 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.081711 2551 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 06 20:08:10.095017 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.081723 2551 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 06 20:08:10.095017 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.081797 2551 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 06 20:08:10.095017 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.081872 2551 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 06 20:08:10.095017 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.081880 2551 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 06 20:08:10.095017 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:10.082541 2551 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 06 20:08:10.095017 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:10.082622 2551 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-178.ec2.internal\" not found" Jul 06 20:08:10.156674 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.156635 2551 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 06 20:08:10.157783 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.157765 2551 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 06 20:08:10.157848 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.157808 2551 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 06 20:08:10.157848 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.157832 2551 kubelet.go:2448] "Starting kubelet main sync loop" Jul 06 20:08:10.157949 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:10.157870 2551 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 06 20:08:10.160217 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.160196 2551 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 20:08:10.182118 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.182093 2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 20:08:10.182979 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.182964 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-178.ec2.internal" event="NodeHasSufficientMemory" Jul 06 20:08:10.183046 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.182994 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-178.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 20:08:10.183046 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.183007 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-178.ec2.internal" event="NodeHasSufficientPID" Jul 06 20:08:10.183046 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.183030 2551 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-178.ec2.internal" Jul 06 20:08:10.192259 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.192206 2551 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-178.ec2.internal" Jul 06 20:08:10.192259 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:10.192228 2551 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-178.ec2.internal\": node \"ip-10-0-137-178.ec2.internal\" not found" Jul 06 20:08:10.208014 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:10.207991 2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-178.ec2.internal\" not found" Jul 06 20:08:10.258953 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.258916 2551 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-178.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-178.ec2.internal"] Jul 06 20:08:10.259058 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.259004 2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 20:08:10.260050 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.260033 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-178.ec2.internal" event="NodeHasSufficientMemory" Jul 06 20:08:10.260144 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.260065 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-178.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 20:08:10.260144 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.260081 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-178.ec2.internal" event="NodeHasSufficientPID" Jul 06 20:08:10.261457 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.261442 2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 20:08:10.261553 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.261538 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-178.ec2.internal" Jul 06 20:08:10.261594 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.261568 2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 20:08:10.262130 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.262116 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-178.ec2.internal" event="NodeHasSufficientMemory" Jul 06 20:08:10.262188 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.262132 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-178.ec2.internal" event="NodeHasSufficientMemory" Jul 06 20:08:10.262188 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.262142 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-178.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 20:08:10.262188 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.262153 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-178.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 20:08:10.262188 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.262155 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-178.ec2.internal" event="NodeHasSufficientPID" Jul 06 20:08:10.262188 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.262164 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-178.ec2.internal" event="NodeHasSufficientPID" Jul 06 20:08:10.263363 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.263348 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-178.ec2.internal" Jul 06 20:08:10.263408 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.263384 2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 20:08:10.263990 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.263975 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-178.ec2.internal" event="NodeHasSufficientMemory" Jul 06 20:08:10.264065 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.264006 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-178.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 20:08:10.264065 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.264029 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-178.ec2.internal" event="NodeHasSufficientPID" Jul 06 20:08:10.287523 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:10.287501 2551 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-178.ec2.internal\" not found" node="ip-10-0-137-178.ec2.internal" Jul 06 20:08:10.290882 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:10.290865 2551 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-178.ec2.internal\" not found" node="ip-10-0-137-178.ec2.internal" Jul 06 20:08:10.308746 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:10.308729 2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-178.ec2.internal\" not found" Jul 06 20:08:10.324963 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.324930 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e99ab74f56459d2f57aab6c6fbd79614-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-178.ec2.internal\" (UID: \"e99ab74f56459d2f57aab6c6fbd79614\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-178.ec2.internal" Jul 06 20:08:10.324963 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.324968 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ac36f75d0ec38f2ed1aabc8706b8a395-config\") pod \"kube-apiserver-proxy-ip-10-0-137-178.ec2.internal\" (UID: \"ac36f75d0ec38f2ed1aabc8706b8a395\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-178.ec2.internal" Jul 06 20:08:10.325081 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.324985 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e99ab74f56459d2f57aab6c6fbd79614-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-178.ec2.internal\" (UID: \"e99ab74f56459d2f57aab6c6fbd79614\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-178.ec2.internal" Jul 06 20:08:10.409630 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:10.409594 2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-178.ec2.internal\" not found" Jul 06 20:08:10.426014 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.425987 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ac36f75d0ec38f2ed1aabc8706b8a395-config\") pod \"kube-apiserver-proxy-ip-10-0-137-178.ec2.internal\" (UID: \"ac36f75d0ec38f2ed1aabc8706b8a395\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-178.ec2.internal" Jul 06 20:08:10.426104 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.426018 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e99ab74f56459d2f57aab6c6fbd79614-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-178.ec2.internal\" (UID: \"e99ab74f56459d2f57aab6c6fbd79614\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-178.ec2.internal" Jul 06 20:08:10.426104 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.426036 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e99ab74f56459d2f57aab6c6fbd79614-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-178.ec2.internal\" (UID: \"e99ab74f56459d2f57aab6c6fbd79614\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-178.ec2.internal" Jul 06 20:08:10.426104 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.426085 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ac36f75d0ec38f2ed1aabc8706b8a395-config\") pod \"kube-apiserver-proxy-ip-10-0-137-178.ec2.internal\" (UID: \"ac36f75d0ec38f2ed1aabc8706b8a395\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-178.ec2.internal" Jul 06 20:08:10.426198 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.426132 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e99ab74f56459d2f57aab6c6fbd79614-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-178.ec2.internal\" (UID: \"e99ab74f56459d2f57aab6c6fbd79614\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-178.ec2.internal" Jul 06 20:08:10.426198 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.426158 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e99ab74f56459d2f57aab6c6fbd79614-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-178.ec2.internal\" (UID: \"e99ab74f56459d2f57aab6c6fbd79614\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-178.ec2.internal" Jul 06 20:08:10.510509 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:10.510431 2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-178.ec2.internal\" not found" Jul 06 20:08:10.591946 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.591912 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-178.ec2.internal" Jul 06 20:08:10.594961 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.594938 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-178.ec2.internal" Jul 06 20:08:10.610769 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:10.610747 2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-178.ec2.internal\" not found" Jul 06 20:08:10.711388 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:10.711343 2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-178.ec2.internal\" not found" Jul 06 20:08:10.811879 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:10.811815 2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-178.ec2.internal\" not found" Jul 06 20:08:10.912435 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:10.912399 2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-178.ec2.internal\" not found" Jul 06 20:08:10.957930 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.957882 2551 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 06 20:08:10.958412 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:10.958043 2551 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 20:08:11.012966 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:11.012945 2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-178.ec2.internal\" not found" Jul 06 20:08:11.055824 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:11.055783 2551 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 20:03:10 +0000 UTC" deadline="2028-03-01 10:39:04.818966851 +0000 UTC" Jul 06 20:08:11.055824 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:11.055815 2551 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14486h30m53.763155132s" Jul 06 20:08:11.106195 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:11.106163 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac36f75d0ec38f2ed1aabc8706b8a395.slice/crio-1a34dfddd2ee90e67ab14146f6eeaa8ab68a31eb5b7caa9516b6cc808a808e78 WatchSource:0}: Error finding container 1a34dfddd2ee90e67ab14146f6eeaa8ab68a31eb5b7caa9516b6cc808a808e78: Status 404 returned error can't find the container with id 1a34dfddd2ee90e67ab14146f6eeaa8ab68a31eb5b7caa9516b6cc808a808e78 Jul 06 20:08:11.106627 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:11.106608 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode99ab74f56459d2f57aab6c6fbd79614.slice/crio-209b149d397e2a575f3878d8b4099f82aace0bda85e7b4816501e874ad4b7ce6 WatchSource:0}: Error finding container 209b149d397e2a575f3878d8b4099f82aace0bda85e7b4816501e874ad4b7ce6: Status 404 returned error can't find the container with id 209b149d397e2a575f3878d8b4099f82aace0bda85e7b4816501e874ad4b7ce6 Jul 06 20:08:11.111079 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:11.111065 2551 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 20:08:11.113755 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:11.113730 2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-178.ec2.internal\" not found" Jul 06 20:08:11.160721 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:11.160671 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-178.ec2.internal" event={"ID":"e99ab74f56459d2f57aab6c6fbd79614","Type":"ContainerStarted","Data":"209b149d397e2a575f3878d8b4099f82aace0bda85e7b4816501e874ad4b7ce6"} Jul 06 20:08:11.161696 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:11.161674 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-178.ec2.internal" event={"ID":"ac36f75d0ec38f2ed1aabc8706b8a395","Type":"ContainerStarted","Data":"1a34dfddd2ee90e67ab14146f6eeaa8ab68a31eb5b7caa9516b6cc808a808e78"} Jul 06 20:08:11.183796 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:11.183771 2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 20:08:11.184926 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:11.184911 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-178.ec2.internal" event="NodeHasSufficientMemory" Jul 06 20:08:11.185001 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:11.184942 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-178.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 20:08:11.185001 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:11.184953 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-178.ec2.internal" event="NodeHasSufficientPID" Jul 06 20:08:11.214241 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:11.214217 2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-178.ec2.internal\" not found" Jul 06 20:08:11.221017 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:11.220998 2551 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 20:08:11.314525 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:11.314495 2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-178.ec2.internal\" not found" Jul 06 20:08:11.319308 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:11.319291 2551 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 20:08:11.323367 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:11.323351 2551 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-178.ec2.internal" Jul 06 20:08:11.335931 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:11.335913 2551 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 20:08:11.337542 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:11.337529 2551 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-178.ec2.internal" Jul 06 20:08:11.347680 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:11.347630 2551 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 20:08:11.572875 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:11.572846 2551 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 20:08:12.001271 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.001239 2551 apiserver.go:52] "Watching apiserver" Jul 06 20:08:12.009765 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.009533 2551 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 06 20:08:12.010186 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.010157 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-g6t4k","kube-system/konnectivity-agent-5b2wn","kube-system/kube-apiserver-proxy-ip-10-0-137-178.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9cb25","openshift-cluster-node-tuning-operator/tuned-b6fnm","openshift-multus/multus-jsbkg","openshift-multus/network-metrics-daemon-v648r","openshift-network-diagnostics/network-check-target-6x889","openshift-dns/node-resolver-477lt","openshift-image-registry/node-ca-mv9hq","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-178.ec2.internal","openshift-multus/multus-additional-cni-plugins-vgxs4","openshift-network-operator/iptables-alerter-55nk7","openshift-ovn-kubernetes/ovnkube-node-r7s8d"] Jul 06 20:08:12.012148 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.012121 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v648r" Jul 06 20:08:12.012264 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:12.012224 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v648r" podUID="ef16cb88-f1e2-49c7-a470-47a23730758d" Jul 06 20:08:12.013656 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.013625 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-55nk7" Jul 06 20:08:12.015541 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.015190 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.016374 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.016349 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 06 20:08:12.016490 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.016417 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 06 20:08:12.016490 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.016449 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-9pkm6\"" Jul 06 20:08:12.016575 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.016352 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 06 20:08:12.017821 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.017645 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9cb25" Jul 06 20:08:12.018008 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.017987 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 06 20:08:12.018101 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.018015 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 06 20:08:12.018459 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.018441 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 06 20:08:12.018567 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.018507 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 06 20:08:12.018632 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.018593 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 06 20:08:12.018742 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.018725 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-h7s2w\"" Jul 06 20:08:12.019446 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.019429 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-b6fnm" Jul 06 20:08:12.019544 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.019472 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 06 20:08:12.020552 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.020521 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 06 20:08:12.020941 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.020914 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 06 20:08:12.021388 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.021091 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-qzrks\"" Jul 06 20:08:12.021388 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.021223 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 06 20:08:12.021388 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.021354 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jsbkg" Jul 06 20:08:12.021996 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.021959 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 06 20:08:12.022180 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.022162 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-mqpmj\"" Jul 06 20:08:12.022548 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.022530 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 06 20:08:12.023971 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.023669 2551 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 06 20:08:12.023971 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.023967 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 06 20:08:12.024119 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.023996 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 06 20:08:12.024237 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.024217 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-477lt" Jul 06 20:08:12.024492 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.024471 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 06 20:08:12.024571 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.024548 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 06 20:08:12.024663 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.024638 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-hcl2k\"" Jul 06 20:08:12.025273 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.025254 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6t4k" Jul 06 20:08:12.025360 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:12.025325 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6t4k" podUID="d6909872-c920-4e38-a17a-11ad51d50379" Jul 06 20:08:12.026723 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.026699 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-bkbwh\"" Jul 06 20:08:12.026816 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.026732 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 06 20:08:12.027064 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.027045 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 06 20:08:12.028685 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.028665 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5b2wn" Jul 06 20:08:12.030163 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.030133 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mv9hq" Jul 06 20:08:12.031787 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.031765 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6x889" Jul 06 20:08:12.031892 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:12.031864 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6x889" podUID="0899bbed-8629-4dd1-a6d0-be26f7870c7f" Jul 06 20:08:12.032796 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.032774 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 06 20:08:12.032915 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.032899 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 06 20:08:12.033164 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.033145 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 06 20:08:12.033798 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.033772 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 06 20:08:12.034018 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.034001 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vgxs4" Jul 06 20:08:12.034018 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.034013 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 06 20:08:12.034170 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.034111 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-npbdf\"" Jul 06 20:08:12.034584 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.034565 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-dcpj7\"" Jul 06 20:08:12.035375 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.035338 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e85c458d-f954-41bc-a06d-1b84854762ee-multus-conf-dir\") pod \"multus-jsbkg\" (UID: \"e85c458d-f954-41bc-a06d-1b84854762ee\") " pod="openshift-multus/multus-jsbkg" Jul 06 20:08:12.035484 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.035394 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/953dca0f-091c-4aa1-90bc-3424c89859af-etc-sysconfig\") pod \"tuned-b6fnm\" (UID: \"953dca0f-091c-4aa1-90bc-3424c89859af\") " pod="openshift-cluster-node-tuning-operator/tuned-b6fnm" Jul 06 20:08:12.035484 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.035429 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/953dca0f-091c-4aa1-90bc-3424c89859af-host\") pod \"tuned-b6fnm\" (UID: \"953dca0f-091c-4aa1-90bc-3424c89859af\") " pod="openshift-cluster-node-tuning-operator/tuned-b6fnm" Jul 06 20:08:12.035484 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.035464 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e85c458d-f954-41bc-a06d-1b84854762ee-system-cni-dir\") pod \"multus-jsbkg\" (UID: \"e85c458d-f954-41bc-a06d-1b84854762ee\") " pod="openshift-multus/multus-jsbkg" Jul 06 20:08:12.035675 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.035496 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/854b635c-f1e7-47d5-bf88-b5a80b572b1b-tmp-dir\") pod \"node-resolver-477lt\" (UID: \"854b635c-f1e7-47d5-bf88-b5a80b572b1b\") " pod="openshift-dns/node-resolver-477lt" Jul 06 20:08:12.035675 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.035532 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f3415158-a586-4fd7-bf83-bbd81db28b49-sys-fs\") pod \"aws-ebs-csi-driver-node-9cb25\" (UID: \"f3415158-a586-4fd7-bf83-bbd81db28b49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9cb25" Jul 06 20:08:12.035675 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.035571 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/953dca0f-091c-4aa1-90bc-3424c89859af-run\") pod \"tuned-b6fnm\" (UID: \"953dca0f-091c-4aa1-90bc-3424c89859af\") " pod="openshift-cluster-node-tuning-operator/tuned-b6fnm" Jul 06 20:08:12.035675 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.035609 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/953dca0f-091c-4aa1-90bc-3424c89859af-sys\") pod \"tuned-b6fnm\" (UID: \"953dca0f-091c-4aa1-90bc-3424c89859af\") " pod="openshift-cluster-node-tuning-operator/tuned-b6fnm" Jul 06 20:08:12.035675 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.035641 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/953dca0f-091c-4aa1-90bc-3424c89859af-etc-tuned\") pod \"tuned-b6fnm\" (UID: \"953dca0f-091c-4aa1-90bc-3424c89859af\") " pod="openshift-cluster-node-tuning-operator/tuned-b6fnm" Jul 06 20:08:12.036039 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.035693 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e85c458d-f954-41bc-a06d-1b84854762ee-multus-cni-dir\") pod \"multus-jsbkg\" (UID: \"e85c458d-f954-41bc-a06d-1b84854762ee\") " pod="openshift-multus/multus-jsbkg" Jul 06 20:08:12.036039 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.035724 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e85c458d-f954-41bc-a06d-1b84854762ee-os-release\") pod \"multus-jsbkg\" (UID: \"e85c458d-f954-41bc-a06d-1b84854762ee\") " pod="openshift-multus/multus-jsbkg" Jul 06 20:08:12.036039 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.035802 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e85c458d-f954-41bc-a06d-1b84854762ee-host-run-k8s-cni-cncf-io\") pod \"multus-jsbkg\" (UID: \"e85c458d-f954-41bc-a06d-1b84854762ee\") " pod="openshift-multus/multus-jsbkg" Jul 06 20:08:12.036039 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.035842 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b7b05003-ad5f-4f11-bc35-476e28673d12-iptables-alerter-script\") pod \"iptables-alerter-55nk7\" (UID: \"b7b05003-ad5f-4f11-bc35-476e28673d12\") " pod="openshift-network-operator/iptables-alerter-55nk7" Jul 06 20:08:12.036039 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.035871 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/645eb1ca-38b9-4599-80d5-cf4314f701c9-run-systemd\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.036039 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.035921 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmdgp\" (UniqueName: \"kubernetes.io/projected/f3415158-a586-4fd7-bf83-bbd81db28b49-kube-api-access-hmdgp\") pod \"aws-ebs-csi-driver-node-9cb25\" (UID: \"f3415158-a586-4fd7-bf83-bbd81db28b49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9cb25" Jul 06 20:08:12.036039 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.035983 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e85c458d-f954-41bc-a06d-1b84854762ee-host-var-lib-kubelet\") pod \"multus-jsbkg\" (UID: \"e85c458d-f954-41bc-a06d-1b84854762ee\") " pod="openshift-multus/multus-jsbkg" Jul 06 20:08:12.036039 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.036014 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e85c458d-f954-41bc-a06d-1b84854762ee-host-run-multus-certs\") pod \"multus-jsbkg\" (UID: \"e85c458d-f954-41bc-a06d-1b84854762ee\") " pod="openshift-multus/multus-jsbkg" Jul 06 20:08:12.036430 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.036046 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2l8xc\" (UniqueName: \"kubernetes.io/projected/e85c458d-f954-41bc-a06d-1b84854762ee-kube-api-access-2l8xc\") pod \"multus-jsbkg\" (UID: \"e85c458d-f954-41bc-a06d-1b84854762ee\") " pod="openshift-multus/multus-jsbkg" Jul 06 20:08:12.036430 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.036076 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6909872-c920-4e38-a17a-11ad51d50379-original-pull-secret\") pod \"global-pull-secret-syncer-g6t4k\" (UID: \"d6909872-c920-4e38-a17a-11ad51d50379\") " pod="kube-system/global-pull-secret-syncer-g6t4k" Jul 06 20:08:12.036430 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.036104 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/645eb1ca-38b9-4599-80d5-cf4314f701c9-systemd-units\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.036430 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.036137 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/645eb1ca-38b9-4599-80d5-cf4314f701c9-host-cni-bin\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.036430 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.036166 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f3415158-a586-4fd7-bf83-bbd81db28b49-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9cb25\" (UID: \"f3415158-a586-4fd7-bf83-bbd81db28b49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9cb25" Jul 06 20:08:12.036430 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.036197 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d6909872-c920-4e38-a17a-11ad51d50379-dbus\") pod \"global-pull-secret-syncer-g6t4k\" (UID: \"d6909872-c920-4e38-a17a-11ad51d50379\") " pod="kube-system/global-pull-secret-syncer-g6t4k" Jul 06 20:08:12.036430 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.036254 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/645eb1ca-38b9-4599-80d5-cf4314f701c9-host-run-netns\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.036430 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.036304 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/645eb1ca-38b9-4599-80d5-cf4314f701c9-etc-openvswitch\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.036430 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.036346 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/645eb1ca-38b9-4599-80d5-cf4314f701c9-run-openvswitch\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.036430 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.036410 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f3415158-a586-4fd7-bf83-bbd81db28b49-registration-dir\") pod \"aws-ebs-csi-driver-node-9cb25\" (UID: \"f3415158-a586-4fd7-bf83-bbd81db28b49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9cb25" Jul 06 20:08:12.036896 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.036475 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7vjq\" (UniqueName: \"kubernetes.io/projected/953dca0f-091c-4aa1-90bc-3424c89859af-kube-api-access-k7vjq\") pod \"tuned-b6fnm\" (UID: \"953dca0f-091c-4aa1-90bc-3424c89859af\") " pod="openshift-cluster-node-tuning-operator/tuned-b6fnm" Jul 06 20:08:12.036896 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.036522 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/645eb1ca-38b9-4599-80d5-cf4314f701c9-host-slash\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.036896 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.036556 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/645eb1ca-38b9-4599-80d5-cf4314f701c9-var-lib-openvswitch\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.036896 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.036602 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/645eb1ca-38b9-4599-80d5-cf4314f701c9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.036896 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.036640 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/645eb1ca-38b9-4599-80d5-cf4314f701c9-env-overrides\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.036896 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.036672 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/953dca0f-091c-4aa1-90bc-3424c89859af-tmp\") pod \"tuned-b6fnm\" (UID: \"953dca0f-091c-4aa1-90bc-3424c89859af\") " pod="openshift-cluster-node-tuning-operator/tuned-b6fnm" Jul 06 20:08:12.036896 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.036733 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/645eb1ca-38b9-4599-80d5-cf4314f701c9-host-cni-netd\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.036896 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.036775 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/953dca0f-091c-4aa1-90bc-3424c89859af-etc-modprobe-d\") pod \"tuned-b6fnm\" (UID: \"953dca0f-091c-4aa1-90bc-3424c89859af\") " pod="openshift-cluster-node-tuning-operator/tuned-b6fnm" Jul 06 20:08:12.036896 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.036814 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d6909872-c920-4e38-a17a-11ad51d50379-kubelet-config\") pod \"global-pull-secret-syncer-g6t4k\" (UID: \"d6909872-c920-4e38-a17a-11ad51d50379\") " pod="kube-system/global-pull-secret-syncer-g6t4k" Jul 06 20:08:12.036896 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.036847 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/953dca0f-091c-4aa1-90bc-3424c89859af-lib-modules\") pod \"tuned-b6fnm\" (UID: \"953dca0f-091c-4aa1-90bc-3424c89859af\") " pod="openshift-cluster-node-tuning-operator/tuned-b6fnm" Jul 06 20:08:12.036896 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.036885 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/953dca0f-091c-4aa1-90bc-3424c89859af-var-lib-kubelet\") pod \"tuned-b6fnm\" (UID: \"953dca0f-091c-4aa1-90bc-3424c89859af\") " pod="openshift-cluster-node-tuning-operator/tuned-b6fnm" Jul 06 20:08:12.037441 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.036917 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/854b635c-f1e7-47d5-bf88-b5a80b572b1b-hosts-file\") pod \"node-resolver-477lt\" (UID: \"854b635c-f1e7-47d5-bf88-b5a80b572b1b\") " pod="openshift-dns/node-resolver-477lt" Jul 06 20:08:12.037441 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.037029 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/645eb1ca-38b9-4599-80d5-cf4314f701c9-run-ovn\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.037441 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.037092 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/645eb1ca-38b9-4599-80d5-cf4314f701c9-host-run-ovn-kubernetes\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.037441 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.037212 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/645eb1ca-38b9-4599-80d5-cf4314f701c9-ovnkube-script-lib\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.037441 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.037270 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbscn\" (UniqueName: \"kubernetes.io/projected/645eb1ca-38b9-4599-80d5-cf4314f701c9-kube-api-access-jbscn\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.037441 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.037321 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/953dca0f-091c-4aa1-90bc-3424c89859af-etc-systemd\") pod \"tuned-b6fnm\" (UID: \"953dca0f-091c-4aa1-90bc-3424c89859af\") " pod="openshift-cluster-node-tuning-operator/tuned-b6fnm" Jul 06 20:08:12.037441 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.037371 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f3415158-a586-4fd7-bf83-bbd81db28b49-device-dir\") pod \"aws-ebs-csi-driver-node-9cb25\" (UID: \"f3415158-a586-4fd7-bf83-bbd81db28b49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9cb25" Jul 06 20:08:12.037441 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.037421 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f3415158-a586-4fd7-bf83-bbd81db28b49-etc-selinux\") pod \"aws-ebs-csi-driver-node-9cb25\" (UID: \"f3415158-a586-4fd7-bf83-bbd81db28b49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9cb25" Jul 06 20:08:12.037819 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.037458 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/953dca0f-091c-4aa1-90bc-3424c89859af-etc-sysctl-conf\") pod \"tuned-b6fnm\" (UID: \"953dca0f-091c-4aa1-90bc-3424c89859af\") " pod="openshift-cluster-node-tuning-operator/tuned-b6fnm" Jul 06 20:08:12.037819 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.037501 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e85c458d-f954-41bc-a06d-1b84854762ee-multus-socket-dir-parent\") pod \"multus-jsbkg\" (UID: \"e85c458d-f954-41bc-a06d-1b84854762ee\") " pod="openshift-multus/multus-jsbkg" Jul 06 20:08:12.037819 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.037537 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e85c458d-f954-41bc-a06d-1b84854762ee-host-run-netns\") pod \"multus-jsbkg\" (UID: \"e85c458d-f954-41bc-a06d-1b84854762ee\") " pod="openshift-multus/multus-jsbkg" Jul 06 20:08:12.037819 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.037607 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e85c458d-f954-41bc-a06d-1b84854762ee-host-var-lib-cni-multus\") pod \"multus-jsbkg\" (UID: \"e85c458d-f954-41bc-a06d-1b84854762ee\") " pod="openshift-multus/multus-jsbkg" Jul 06 20:08:12.037819 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.037665 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e85c458d-f954-41bc-a06d-1b84854762ee-etc-kubernetes\") pod \"multus-jsbkg\" (UID: \"e85c458d-f954-41bc-a06d-1b84854762ee\") " pod="openshift-multus/multus-jsbkg" Jul 06 20:08:12.037819 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.037703 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b7b05003-ad5f-4f11-bc35-476e28673d12-host-slash\") pod \"iptables-alerter-55nk7\" (UID: \"b7b05003-ad5f-4f11-bc35-476e28673d12\") " pod="openshift-network-operator/iptables-alerter-55nk7" Jul 06 20:08:12.038182 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.037945 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lf8r\" (UniqueName: \"kubernetes.io/projected/854b635c-f1e7-47d5-bf88-b5a80b572b1b-kube-api-access-2lf8r\") pod \"node-resolver-477lt\" (UID: \"854b635c-f1e7-47d5-bf88-b5a80b572b1b\") " pod="openshift-dns/node-resolver-477lt" Jul 06 20:08:12.038182 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.038017 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/645eb1ca-38b9-4599-80d5-cf4314f701c9-node-log\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.038182 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.038057 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/953dca0f-091c-4aa1-90bc-3424c89859af-etc-kubernetes\") pod \"tuned-b6fnm\" (UID: \"953dca0f-091c-4aa1-90bc-3424c89859af\") " pod="openshift-cluster-node-tuning-operator/tuned-b6fnm" Jul 06 20:08:12.038182 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.038088 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/645eb1ca-38b9-4599-80d5-cf4314f701c9-host-kubelet\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.038182 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.038121 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e85c458d-f954-41bc-a06d-1b84854762ee-cnibin\") pod \"multus-jsbkg\" (UID: \"e85c458d-f954-41bc-a06d-1b84854762ee\") " pod="openshift-multus/multus-jsbkg" Jul 06 20:08:12.038425 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.038280 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-s5mmd\"" Jul 06 20:08:12.038425 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.038295 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 06 20:08:12.038425 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.038393 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 06 20:08:12.038743 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.038720 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctl7x\" (UniqueName: \"kubernetes.io/projected/b7b05003-ad5f-4f11-bc35-476e28673d12-kube-api-access-ctl7x\") pod \"iptables-alerter-55nk7\" (UID: \"b7b05003-ad5f-4f11-bc35-476e28673d12\") " pod="openshift-network-operator/iptables-alerter-55nk7" Jul 06 20:08:12.038840 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.038768 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8cns\" (UniqueName: \"kubernetes.io/projected/ef16cb88-f1e2-49c7-a470-47a23730758d-kube-api-access-w8cns\") pod \"network-metrics-daemon-v648r\" (UID: \"ef16cb88-f1e2-49c7-a470-47a23730758d\") " pod="openshift-multus/network-metrics-daemon-v648r" Jul 06 20:08:12.038840 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.038804 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/645eb1ca-38b9-4599-80d5-cf4314f701c9-log-socket\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.038967 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.038845 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/645eb1ca-38b9-4599-80d5-cf4314f701c9-ovn-node-metrics-cert\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.038967 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.038912 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e85c458d-f954-41bc-a06d-1b84854762ee-cni-binary-copy\") pod \"multus-jsbkg\" (UID: \"e85c458d-f954-41bc-a06d-1b84854762ee\") " pod="openshift-multus/multus-jsbkg" Jul 06 20:08:12.038967 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.038947 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e85c458d-f954-41bc-a06d-1b84854762ee-host-var-lib-cni-bin\") pod \"multus-jsbkg\" (UID: \"e85c458d-f954-41bc-a06d-1b84854762ee\") " pod="openshift-multus/multus-jsbkg" Jul 06 20:08:12.039124 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.038999 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f3415158-a586-4fd7-bf83-bbd81db28b49-socket-dir\") pod \"aws-ebs-csi-driver-node-9cb25\" (UID: \"f3415158-a586-4fd7-bf83-bbd81db28b49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9cb25" Jul 06 20:08:12.039124 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.039026 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/953dca0f-091c-4aa1-90bc-3424c89859af-etc-sysctl-d\") pod \"tuned-b6fnm\" (UID: \"953dca0f-091c-4aa1-90bc-3424c89859af\") " pod="openshift-cluster-node-tuning-operator/tuned-b6fnm" Jul 06 20:08:12.039124 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.039042 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e85c458d-f954-41bc-a06d-1b84854762ee-hostroot\") pod \"multus-jsbkg\" (UID: \"e85c458d-f954-41bc-a06d-1b84854762ee\") " pod="openshift-multus/multus-jsbkg" Jul 06 20:08:12.039124 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.039060 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e85c458d-f954-41bc-a06d-1b84854762ee-multus-daemon-config\") pod \"multus-jsbkg\" (UID: \"e85c458d-f954-41bc-a06d-1b84854762ee\") " pod="openshift-multus/multus-jsbkg" Jul 06 20:08:12.039124 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.039089 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef16cb88-f1e2-49c7-a470-47a23730758d-metrics-certs\") pod \"network-metrics-daemon-v648r\" (UID: \"ef16cb88-f1e2-49c7-a470-47a23730758d\") " pod="openshift-multus/network-metrics-daemon-v648r" Jul 06 20:08:12.039124 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.039120 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/645eb1ca-38b9-4599-80d5-cf4314f701c9-ovnkube-config\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.041289 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.041267 2551 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 20:08:12.069220 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.069192 2551 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-lvprh" Jul 06 20:08:12.081110 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.081085 2551 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-lvprh" Jul 06 20:08:12.124692 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.124662 2551 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 06 20:08:12.139588 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.139545 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/953dca0f-091c-4aa1-90bc-3424c89859af-etc-sysconfig\") pod \"tuned-b6fnm\" (UID: \"953dca0f-091c-4aa1-90bc-3424c89859af\") " pod="openshift-cluster-node-tuning-operator/tuned-b6fnm" Jul 06 20:08:12.139794 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.139586 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/953dca0f-091c-4aa1-90bc-3424c89859af-host\") pod \"tuned-b6fnm\" (UID: \"953dca0f-091c-4aa1-90bc-3424c89859af\") " pod="openshift-cluster-node-tuning-operator/tuned-b6fnm" Jul 06 20:08:12.139794 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.139644 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e85c458d-f954-41bc-a06d-1b84854762ee-system-cni-dir\") pod \"multus-jsbkg\" (UID: \"e85c458d-f954-41bc-a06d-1b84854762ee\") " pod="openshift-multus/multus-jsbkg" Jul 06 20:08:12.139794 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.139680 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4c726ad1-d957-471d-95d9-fcaee1f227e8-cnibin\") pod \"multus-additional-cni-plugins-vgxs4\" (UID: \"4c726ad1-d957-471d-95d9-fcaee1f227e8\") " pod="openshift-multus/multus-additional-cni-plugins-vgxs4" Jul 06 20:08:12.139794 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.139706 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4c726ad1-d957-471d-95d9-fcaee1f227e8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vgxs4\" (UID: \"4c726ad1-d957-471d-95d9-fcaee1f227e8\") " pod="openshift-multus/multus-additional-cni-plugins-vgxs4" Jul 06 20:08:12.139794 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.139712 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/953dca0f-091c-4aa1-90bc-3424c89859af-etc-sysconfig\") pod \"tuned-b6fnm\" (UID: \"953dca0f-091c-4aa1-90bc-3424c89859af\") " pod="openshift-cluster-node-tuning-operator/tuned-b6fnm" Jul 06 20:08:12.139794 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.139734 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/854b635c-f1e7-47d5-bf88-b5a80b572b1b-tmp-dir\") pod \"node-resolver-477lt\" (UID: \"854b635c-f1e7-47d5-bf88-b5a80b572b1b\") " pod="openshift-dns/node-resolver-477lt" Jul 06 20:08:12.139794 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.139765 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f3415158-a586-4fd7-bf83-bbd81db28b49-sys-fs\") pod \"aws-ebs-csi-driver-node-9cb25\" (UID: \"f3415158-a586-4fd7-bf83-bbd81db28b49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9cb25" Jul 06 20:08:12.139794 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.139787 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/953dca0f-091c-4aa1-90bc-3424c89859af-run\") pod \"tuned-b6fnm\" (UID: \"953dca0f-091c-4aa1-90bc-3424c89859af\") " pod="openshift-cluster-node-tuning-operator/tuned-b6fnm" Jul 06 20:08:12.139794 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.139789 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e85c458d-f954-41bc-a06d-1b84854762ee-system-cni-dir\") pod \"multus-jsbkg\" (UID: \"e85c458d-f954-41bc-a06d-1b84854762ee\") " pod="openshift-multus/multus-jsbkg" Jul 06 20:08:12.140276 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.139824 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/953dca0f-091c-4aa1-90bc-3424c89859af-sys\") pod \"tuned-b6fnm\" (UID: \"953dca0f-091c-4aa1-90bc-3424c89859af\") " pod="openshift-cluster-node-tuning-operator/tuned-b6fnm" Jul 06 20:08:12.140276 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.139852 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/953dca0f-091c-4aa1-90bc-3424c89859af-etc-tuned\") pod \"tuned-b6fnm\" (UID: \"953dca0f-091c-4aa1-90bc-3424c89859af\") " pod="openshift-cluster-node-tuning-operator/tuned-b6fnm" Jul 06 20:08:12.140276 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.139857 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/953dca0f-091c-4aa1-90bc-3424c89859af-run\") pod \"tuned-b6fnm\" (UID: \"953dca0f-091c-4aa1-90bc-3424c89859af\") " pod="openshift-cluster-node-tuning-operator/tuned-b6fnm" Jul 06 20:08:12.140276 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.139878 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e85c458d-f954-41bc-a06d-1b84854762ee-multus-cni-dir\") pod \"multus-jsbkg\" (UID: \"e85c458d-f954-41bc-a06d-1b84854762ee\") " pod="openshift-multus/multus-jsbkg" Jul 06 20:08:12.140276 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.139953 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/953dca0f-091c-4aa1-90bc-3424c89859af-sys\") pod \"tuned-b6fnm\" (UID: \"953dca0f-091c-4aa1-90bc-3424c89859af\") " pod="openshift-cluster-node-tuning-operator/tuned-b6fnm" Jul 06 20:08:12.140276 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.139997 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/953dca0f-091c-4aa1-90bc-3424c89859af-host\") pod \"tuned-b6fnm\" (UID: \"953dca0f-091c-4aa1-90bc-3424c89859af\") " pod="openshift-cluster-node-tuning-operator/tuned-b6fnm" Jul 06 20:08:12.140541 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.140300 2551 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 06 20:08:12.140541 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.140327 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e85c458d-f954-41bc-a06d-1b84854762ee-multus-cni-dir\") pod \"multus-jsbkg\" (UID: \"e85c458d-f954-41bc-a06d-1b84854762ee\") " pod="openshift-multus/multus-jsbkg" Jul 06 20:08:12.140541 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.140342 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/854b635c-f1e7-47d5-bf88-b5a80b572b1b-tmp-dir\") pod \"node-resolver-477lt\" (UID: \"854b635c-f1e7-47d5-bf88-b5a80b572b1b\") " pod="openshift-dns/node-resolver-477lt" Jul 06 20:08:12.140541 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.140373 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e85c458d-f954-41bc-a06d-1b84854762ee-os-release\") pod \"multus-jsbkg\" (UID: \"e85c458d-f954-41bc-a06d-1b84854762ee\") " pod="openshift-multus/multus-jsbkg" Jul 06 20:08:12.140541 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.140404 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e85c458d-f954-41bc-a06d-1b84854762ee-host-run-k8s-cni-cncf-io\") pod \"multus-jsbkg\" (UID: \"e85c458d-f954-41bc-a06d-1b84854762ee\") " pod="openshift-multus/multus-jsbkg" Jul 06 20:08:12.140541 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.140436 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b7b05003-ad5f-4f11-bc35-476e28673d12-iptables-alerter-script\") pod \"iptables-alerter-55nk7\" (UID: \"b7b05003-ad5f-4f11-bc35-476e28673d12\") " pod="openshift-network-operator/iptables-alerter-55nk7" Jul 06 20:08:12.140541 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.140445 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f3415158-a586-4fd7-bf83-bbd81db28b49-sys-fs\") pod \"aws-ebs-csi-driver-node-9cb25\" (UID: \"f3415158-a586-4fd7-bf83-bbd81db28b49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9cb25" Jul 06 20:08:12.140541 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.140461 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/645eb1ca-38b9-4599-80d5-cf4314f701c9-run-systemd\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.140541 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.140486 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e85c458d-f954-41bc-a06d-1b84854762ee-host-run-k8s-cni-cncf-io\") pod \"multus-jsbkg\" (UID: \"e85c458d-f954-41bc-a06d-1b84854762ee\") " pod="openshift-multus/multus-jsbkg" Jul 06 20:08:12.140541 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.140487 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmdgp\" (UniqueName: \"kubernetes.io/projected/f3415158-a586-4fd7-bf83-bbd81db28b49-kube-api-access-hmdgp\") pod \"aws-ebs-csi-driver-node-9cb25\" (UID: \"f3415158-a586-4fd7-bf83-bbd81db28b49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9cb25" Jul 06 20:08:12.140541 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.140524 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e85c458d-f954-41bc-a06d-1b84854762ee-host-var-lib-kubelet\") pod \"multus-jsbkg\" (UID: \"e85c458d-f954-41bc-a06d-1b84854762ee\") " pod="openshift-multus/multus-jsbkg" Jul 06 20:08:12.141103 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.140550 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e85c458d-f954-41bc-a06d-1b84854762ee-host-run-multus-certs\") pod \"multus-jsbkg\" (UID: \"e85c458d-f954-41bc-a06d-1b84854762ee\") " pod="openshift-multus/multus-jsbkg" Jul 06 20:08:12.141103 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.140594 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2l8xc\" (UniqueName: \"kubernetes.io/projected/e85c458d-f954-41bc-a06d-1b84854762ee-kube-api-access-2l8xc\") pod \"multus-jsbkg\" (UID: \"e85c458d-f954-41bc-a06d-1b84854762ee\") " pod="openshift-multus/multus-jsbkg" Jul 06 20:08:12.141103 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.140622 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6909872-c920-4e38-a17a-11ad51d50379-original-pull-secret\") pod \"global-pull-secret-syncer-g6t4k\" (UID: \"d6909872-c920-4e38-a17a-11ad51d50379\") " pod="kube-system/global-pull-secret-syncer-g6t4k" Jul 06 20:08:12.141103 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.140655 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ds27q\" (UniqueName: \"kubernetes.io/projected/0899bbed-8629-4dd1-a6d0-be26f7870c7f-kube-api-access-ds27q\") pod \"network-check-target-6x889\" (UID: \"0899bbed-8629-4dd1-a6d0-be26f7870c7f\") " pod="openshift-network-diagnostics/network-check-target-6x889" Jul 06 20:08:12.141103 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.140687 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/645eb1ca-38b9-4599-80d5-cf4314f701c9-systemd-units\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.141103 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.140714 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/645eb1ca-38b9-4599-80d5-cf4314f701c9-host-cni-bin\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.141103 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.140756 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f3415158-a586-4fd7-bf83-bbd81db28b49-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9cb25\" (UID: \"f3415158-a586-4fd7-bf83-bbd81db28b49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9cb25" Jul 06 20:08:12.141103 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.140792 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d6909872-c920-4e38-a17a-11ad51d50379-dbus\") pod \"global-pull-secret-syncer-g6t4k\" (UID: \"d6909872-c920-4e38-a17a-11ad51d50379\") " pod="kube-system/global-pull-secret-syncer-g6t4k" Jul 06 20:08:12.141103 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:12.140800 2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 20:08:12.141103 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.140823 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/645eb1ca-38b9-4599-80d5-cf4314f701c9-host-run-netns\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.141103 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.140850 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/645eb1ca-38b9-4599-80d5-cf4314f701c9-etc-openvswitch\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.141103 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:12.140871 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6909872-c920-4e38-a17a-11ad51d50379-original-pull-secret podName:d6909872-c920-4e38-a17a-11ad51d50379 nodeName:}" failed. No retries permitted until 2026-07-06 20:08:12.640846518 +0000 UTC m=+3.056046307 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d6909872-c920-4e38-a17a-11ad51d50379-original-pull-secret") pod "global-pull-secret-syncer-g6t4k" (UID: "d6909872-c920-4e38-a17a-11ad51d50379") : object "kube-system"/"original-pull-secret" not registered Jul 06 20:08:12.141103 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.140902 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/645eb1ca-38b9-4599-80d5-cf4314f701c9-etc-openvswitch\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.141103 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.140909 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/645eb1ca-38b9-4599-80d5-cf4314f701c9-run-openvswitch\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.141103 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.140935 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f3415158-a586-4fd7-bf83-bbd81db28b49-registration-dir\") pod \"aws-ebs-csi-driver-node-9cb25\" (UID: \"f3415158-a586-4fd7-bf83-bbd81db28b49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9cb25" Jul 06 20:08:12.141831 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.141145 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/645eb1ca-38b9-4599-80d5-cf4314f701c9-host-cni-bin\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.141831 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.141189 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/645eb1ca-38b9-4599-80d5-cf4314f701c9-run-openvswitch\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.141831 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.141210 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e85c458d-f954-41bc-a06d-1b84854762ee-os-release\") pod \"multus-jsbkg\" (UID: \"e85c458d-f954-41bc-a06d-1b84854762ee\") " pod="openshift-multus/multus-jsbkg" Jul 06 20:08:12.141831 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.141237 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/645eb1ca-38b9-4599-80d5-cf4314f701c9-run-systemd\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.141831 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.141254 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e85c458d-f954-41bc-a06d-1b84854762ee-host-var-lib-kubelet\") pod \"multus-jsbkg\" (UID: \"e85c458d-f954-41bc-a06d-1b84854762ee\") " pod="openshift-multus/multus-jsbkg" Jul 06 20:08:12.141831 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.141279 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/645eb1ca-38b9-4599-80d5-cf4314f701c9-systemd-units\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.141831 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.141282 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e85c458d-f954-41bc-a06d-1b84854762ee-host-run-multus-certs\") pod \"multus-jsbkg\" (UID: \"e85c458d-f954-41bc-a06d-1b84854762ee\") " pod="openshift-multus/multus-jsbkg" Jul 06 20:08:12.141831 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.141300 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f3415158-a586-4fd7-bf83-bbd81db28b49-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9cb25\" (UID: \"f3415158-a586-4fd7-bf83-bbd81db28b49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9cb25" Jul 06 20:08:12.141831 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.141317 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7vjq\" (UniqueName: \"kubernetes.io/projected/953dca0f-091c-4aa1-90bc-3424c89859af-kube-api-access-k7vjq\") pod \"tuned-b6fnm\" (UID: \"953dca0f-091c-4aa1-90bc-3424c89859af\") " pod="openshift-cluster-node-tuning-operator/tuned-b6fnm" Jul 06 20:08:12.141831 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.141332 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d6909872-c920-4e38-a17a-11ad51d50379-dbus\") pod \"global-pull-secret-syncer-g6t4k\" (UID: \"d6909872-c920-4e38-a17a-11ad51d50379\") " pod="kube-system/global-pull-secret-syncer-g6t4k" Jul 06 20:08:12.141831 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.141347 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9cd8d42e-db03-477c-8ef9-bfed943eadb3-serviceca\") pod \"node-ca-mv9hq\" (UID: \"9cd8d42e-db03-477c-8ef9-bfed943eadb3\") " pod="openshift-image-registry/node-ca-mv9hq" Jul 06 20:08:12.141831 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.141345 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b7b05003-ad5f-4f11-bc35-476e28673d12-iptables-alerter-script\") pod \"iptables-alerter-55nk7\" (UID: \"b7b05003-ad5f-4f11-bc35-476e28673d12\") " pod="openshift-network-operator/iptables-alerter-55nk7" Jul 06 20:08:12.141831 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.141402 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4c726ad1-d957-471d-95d9-fcaee1f227e8-system-cni-dir\") pod \"multus-additional-cni-plugins-vgxs4\" (UID: \"4c726ad1-d957-471d-95d9-fcaee1f227e8\") " pod="openshift-multus/multus-additional-cni-plugins-vgxs4" Jul 06 20:08:12.141831 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.141438 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f3415158-a586-4fd7-bf83-bbd81db28b49-registration-dir\") pod \"aws-ebs-csi-driver-node-9cb25\" (UID: \"f3415158-a586-4fd7-bf83-bbd81db28b49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9cb25" Jul 06 20:08:12.141831 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.141477 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/645eb1ca-38b9-4599-80d5-cf4314f701c9-host-slash\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.141831 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.141507 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/645eb1ca-38b9-4599-80d5-cf4314f701c9-var-lib-openvswitch\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.141831 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.141537 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/645eb1ca-38b9-4599-80d5-cf4314f701c9-host-run-netns\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.142744 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.141565 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/645eb1ca-38b9-4599-80d5-cf4314f701c9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.142744 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.141585 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/645eb1ca-38b9-4599-80d5-cf4314f701c9-host-slash\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.142744 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.141588 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/645eb1ca-38b9-4599-80d5-cf4314f701c9-var-lib-openvswitch\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.142744 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.141626 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/645eb1ca-38b9-4599-80d5-cf4314f701c9-env-overrides\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.142744 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.141654 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/645eb1ca-38b9-4599-80d5-cf4314f701c9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.142744 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.141694 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/953dca0f-091c-4aa1-90bc-3424c89859af-tmp\") pod \"tuned-b6fnm\" (UID: \"953dca0f-091c-4aa1-90bc-3424c89859af\") " pod="openshift-cluster-node-tuning-operator/tuned-b6fnm" Jul 06 20:08:12.142744 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.141769 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1bbf9b1e-36f5-487b-a66e-55dc425fd1a0-agent-certs\") pod \"konnectivity-agent-5b2wn\" (UID: \"1bbf9b1e-36f5-487b-a66e-55dc425fd1a0\") " pod="kube-system/konnectivity-agent-5b2wn" Jul 06 20:08:12.142744 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.141814 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1bbf9b1e-36f5-487b-a66e-55dc425fd1a0-konnectivity-ca\") pod \"konnectivity-agent-5b2wn\" (UID: \"1bbf9b1e-36f5-487b-a66e-55dc425fd1a0\") " pod="kube-system/konnectivity-agent-5b2wn" Jul 06 20:08:12.142744 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.141853 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9cd8d42e-db03-477c-8ef9-bfed943eadb3-host\") pod \"node-ca-mv9hq\" (UID: \"9cd8d42e-db03-477c-8ef9-bfed943eadb3\") " pod="openshift-image-registry/node-ca-mv9hq" Jul 06 20:08:12.142744 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.141897 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/645eb1ca-38b9-4599-80d5-cf4314f701c9-host-cni-netd\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.142744 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.141953 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/953dca0f-091c-4aa1-90bc-3424c89859af-etc-modprobe-d\") pod \"tuned-b6fnm\" (UID: \"953dca0f-091c-4aa1-90bc-3424c89859af\") " pod="openshift-cluster-node-tuning-operator/tuned-b6fnm" Jul 06 20:08:12.142744 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.141956 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/645eb1ca-38b9-4599-80d5-cf4314f701c9-host-cni-netd\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.142744 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.141993 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d6909872-c920-4e38-a17a-11ad51d50379-kubelet-config\") pod \"global-pull-secret-syncer-g6t4k\" (UID: \"d6909872-c920-4e38-a17a-11ad51d50379\") " pod="kube-system/global-pull-secret-syncer-g6t4k" Jul 06 20:08:12.142744 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.142036 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4c726ad1-d957-471d-95d9-fcaee1f227e8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vgxs4\" (UID: \"4c726ad1-d957-471d-95d9-fcaee1f227e8\") " pod="openshift-multus/multus-additional-cni-plugins-vgxs4" Jul 06 20:08:12.142744 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.142096 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/953dca0f-091c-4aa1-90bc-3424c89859af-etc-modprobe-d\") pod \"tuned-b6fnm\" (UID: \"953dca0f-091c-4aa1-90bc-3424c89859af\") " pod="openshift-cluster-node-tuning-operator/tuned-b6fnm" Jul 06 20:08:12.142744 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.142100 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5tzx\" (UniqueName: \"kubernetes.io/projected/4c726ad1-d957-471d-95d9-fcaee1f227e8-kube-api-access-l5tzx\") pod \"multus-additional-cni-plugins-vgxs4\" (UID: \"4c726ad1-d957-471d-95d9-fcaee1f227e8\") " pod="openshift-multus/multus-additional-cni-plugins-vgxs4" Jul 06 20:08:12.142744 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.142105 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d6909872-c920-4e38-a17a-11ad51d50379-kubelet-config\") pod \"global-pull-secret-syncer-g6t4k\" (UID: \"d6909872-c920-4e38-a17a-11ad51d50379\") " pod="kube-system/global-pull-secret-syncer-g6t4k" Jul 06 20:08:12.143614 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.142139 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/953dca0f-091c-4aa1-90bc-3424c89859af-lib-modules\") pod \"tuned-b6fnm\" (UID: \"953dca0f-091c-4aa1-90bc-3424c89859af\") " pod="openshift-cluster-node-tuning-operator/tuned-b6fnm" Jul 06 20:08:12.143614 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.142154 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/645eb1ca-38b9-4599-80d5-cf4314f701c9-env-overrides\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.143614 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.142172 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/953dca0f-091c-4aa1-90bc-3424c89859af-var-lib-kubelet\") pod \"tuned-b6fnm\" (UID: \"953dca0f-091c-4aa1-90bc-3424c89859af\") " pod="openshift-cluster-node-tuning-operator/tuned-b6fnm" Jul 06 20:08:12.143614 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.142200 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/854b635c-f1e7-47d5-bf88-b5a80b572b1b-hosts-file\") pod \"node-resolver-477lt\" (UID: \"854b635c-f1e7-47d5-bf88-b5a80b572b1b\") " pod="openshift-dns/node-resolver-477lt" Jul 06 20:08:12.143614 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.142225 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/645eb1ca-38b9-4599-80d5-cf4314f701c9-run-ovn\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.143614 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.142254 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/953dca0f-091c-4aa1-90bc-3424c89859af-var-lib-kubelet\") pod \"tuned-b6fnm\" (UID: \"953dca0f-091c-4aa1-90bc-3424c89859af\") " pod="openshift-cluster-node-tuning-operator/tuned-b6fnm" Jul 06 20:08:12.143614 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.142271 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/645eb1ca-38b9-4599-80d5-cf4314f701c9-host-run-ovn-kubernetes\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.143614 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.142290 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/953dca0f-091c-4aa1-90bc-3424c89859af-lib-modules\") pod \"tuned-b6fnm\" (UID: \"953dca0f-091c-4aa1-90bc-3424c89859af\") " pod="openshift-cluster-node-tuning-operator/tuned-b6fnm" Jul 06 20:08:12.143614 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.142299 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/645eb1ca-38b9-4599-80d5-cf4314f701c9-ovnkube-script-lib\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.143614 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.142317 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/645eb1ca-38b9-4599-80d5-cf4314f701c9-run-ovn\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.143614 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.142324 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/854b635c-f1e7-47d5-bf88-b5a80b572b1b-hosts-file\") pod \"node-resolver-477lt\" (UID: \"854b635c-f1e7-47d5-bf88-b5a80b572b1b\") " pod="openshift-dns/node-resolver-477lt" Jul 06 20:08:12.143614 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.142329 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbscn\" (UniqueName: \"kubernetes.io/projected/645eb1ca-38b9-4599-80d5-cf4314f701c9-kube-api-access-jbscn\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.143614 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.142339 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/645eb1ca-38b9-4599-80d5-cf4314f701c9-host-run-ovn-kubernetes\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.143614 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.142366 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/953dca0f-091c-4aa1-90bc-3424c89859af-etc-systemd\") pod \"tuned-b6fnm\" (UID: \"953dca0f-091c-4aa1-90bc-3424c89859af\") " pod="openshift-cluster-node-tuning-operator/tuned-b6fnm" Jul 06 20:08:12.143614 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.142398 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f3415158-a586-4fd7-bf83-bbd81db28b49-device-dir\") pod \"aws-ebs-csi-driver-node-9cb25\" (UID: \"f3415158-a586-4fd7-bf83-bbd81db28b49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9cb25" Jul 06 20:08:12.143614 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.142445 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f3415158-a586-4fd7-bf83-bbd81db28b49-etc-selinux\") pod \"aws-ebs-csi-driver-node-9cb25\" (UID: \"f3415158-a586-4fd7-bf83-bbd81db28b49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9cb25" Jul 06 20:08:12.143614 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.142452 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/953dca0f-091c-4aa1-90bc-3424c89859af-etc-systemd\") pod \"tuned-b6fnm\" (UID: \"953dca0f-091c-4aa1-90bc-3424c89859af\") " pod="openshift-cluster-node-tuning-operator/tuned-b6fnm" Jul 06 20:08:12.144543 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.142478 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/953dca0f-091c-4aa1-90bc-3424c89859af-etc-sysctl-conf\") pod \"tuned-b6fnm\" (UID: \"953dca0f-091c-4aa1-90bc-3424c89859af\") " pod="openshift-cluster-node-tuning-operator/tuned-b6fnm" Jul 06 20:08:12.144543 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.142492 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f3415158-a586-4fd7-bf83-bbd81db28b49-device-dir\") pod \"aws-ebs-csi-driver-node-9cb25\" (UID: \"f3415158-a586-4fd7-bf83-bbd81db28b49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9cb25" Jul 06 20:08:12.144543 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.142504 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e85c458d-f954-41bc-a06d-1b84854762ee-multus-socket-dir-parent\") pod \"multus-jsbkg\" (UID: \"e85c458d-f954-41bc-a06d-1b84854762ee\") " pod="openshift-multus/multus-jsbkg" Jul 06 20:08:12.144543 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.142531 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e85c458d-f954-41bc-a06d-1b84854762ee-host-run-netns\") pod \"multus-jsbkg\" (UID: \"e85c458d-f954-41bc-a06d-1b84854762ee\") " pod="openshift-multus/multus-jsbkg" Jul 06 20:08:12.144543 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.142544 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f3415158-a586-4fd7-bf83-bbd81db28b49-etc-selinux\") pod \"aws-ebs-csi-driver-node-9cb25\" (UID: \"f3415158-a586-4fd7-bf83-bbd81db28b49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9cb25" Jul 06 20:08:12.144543 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.142573 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e85c458d-f954-41bc-a06d-1b84854762ee-host-var-lib-cni-multus\") pod \"multus-jsbkg\" (UID: \"e85c458d-f954-41bc-a06d-1b84854762ee\") " pod="openshift-multus/multus-jsbkg" Jul 06 20:08:12.144543 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.142587 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e85c458d-f954-41bc-a06d-1b84854762ee-host-run-netns\") pod \"multus-jsbkg\" (UID: \"e85c458d-f954-41bc-a06d-1b84854762ee\") " pod="openshift-multus/multus-jsbkg" Jul 06 20:08:12.144543 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.142592 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e85c458d-f954-41bc-a06d-1b84854762ee-multus-socket-dir-parent\") pod \"multus-jsbkg\" (UID: \"e85c458d-f954-41bc-a06d-1b84854762ee\") " pod="openshift-multus/multus-jsbkg" Jul 06 20:08:12.144543 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.142598 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e85c458d-f954-41bc-a06d-1b84854762ee-etc-kubernetes\") pod \"multus-jsbkg\" (UID: \"e85c458d-f954-41bc-a06d-1b84854762ee\") " pod="openshift-multus/multus-jsbkg" Jul 06 20:08:12.144543 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.142624 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/953dca0f-091c-4aa1-90bc-3424c89859af-etc-sysctl-conf\") pod \"tuned-b6fnm\" (UID: \"953dca0f-091c-4aa1-90bc-3424c89859af\") " pod="openshift-cluster-node-tuning-operator/tuned-b6fnm" Jul 06 20:08:12.144543 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.142635 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b7b05003-ad5f-4f11-bc35-476e28673d12-host-slash\") pod \"iptables-alerter-55nk7\" (UID: \"b7b05003-ad5f-4f11-bc35-476e28673d12\") " pod="openshift-network-operator/iptables-alerter-55nk7" Jul 06 20:08:12.144543 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.142643 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e85c458d-f954-41bc-a06d-1b84854762ee-etc-kubernetes\") pod \"multus-jsbkg\" (UID: \"e85c458d-f954-41bc-a06d-1b84854762ee\") " pod="openshift-multus/multus-jsbkg" Jul 06 20:08:12.144543 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.142634 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e85c458d-f954-41bc-a06d-1b84854762ee-host-var-lib-cni-multus\") pod \"multus-jsbkg\" (UID: \"e85c458d-f954-41bc-a06d-1b84854762ee\") " pod="openshift-multus/multus-jsbkg" Jul 06 20:08:12.144543 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.142662 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lf8r\" (UniqueName: \"kubernetes.io/projected/854b635c-f1e7-47d5-bf88-b5a80b572b1b-kube-api-access-2lf8r\") pod \"node-resolver-477lt\" (UID: \"854b635c-f1e7-47d5-bf88-b5a80b572b1b\") " pod="openshift-dns/node-resolver-477lt" Jul 06 20:08:12.144543 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.142674 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b7b05003-ad5f-4f11-bc35-476e28673d12-host-slash\") pod \"iptables-alerter-55nk7\" (UID: \"b7b05003-ad5f-4f11-bc35-476e28673d12\") " pod="openshift-network-operator/iptables-alerter-55nk7" Jul 06 20:08:12.144543 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.142701 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlm8g\" (UniqueName: \"kubernetes.io/projected/9cd8d42e-db03-477c-8ef9-bfed943eadb3-kube-api-access-hlm8g\") pod \"node-ca-mv9hq\" (UID: \"9cd8d42e-db03-477c-8ef9-bfed943eadb3\") " pod="openshift-image-registry/node-ca-mv9hq" Jul 06 20:08:12.144543 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.142741 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/645eb1ca-38b9-4599-80d5-cf4314f701c9-node-log\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.144543 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.142766 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/953dca0f-091c-4aa1-90bc-3424c89859af-etc-kubernetes\") pod \"tuned-b6fnm\" (UID: \"953dca0f-091c-4aa1-90bc-3424c89859af\") " pod="openshift-cluster-node-tuning-operator/tuned-b6fnm" Jul 06 20:08:12.145467 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.142797 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4c726ad1-d957-471d-95d9-fcaee1f227e8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vgxs4\" (UID: \"4c726ad1-d957-471d-95d9-fcaee1f227e8\") " pod="openshift-multus/multus-additional-cni-plugins-vgxs4" Jul 06 20:08:12.145467 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.142816 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/645eb1ca-38b9-4599-80d5-cf4314f701c9-node-log\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.145467 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.142825 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/645eb1ca-38b9-4599-80d5-cf4314f701c9-host-kubelet\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.145467 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.142830 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/953dca0f-091c-4aa1-90bc-3424c89859af-etc-kubernetes\") pod \"tuned-b6fnm\" (UID: \"953dca0f-091c-4aa1-90bc-3424c89859af\") " pod="openshift-cluster-node-tuning-operator/tuned-b6fnm" Jul 06 20:08:12.145467 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.142837 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/645eb1ca-38b9-4599-80d5-cf4314f701c9-ovnkube-script-lib\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.145467 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.142873 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e85c458d-f954-41bc-a06d-1b84854762ee-cnibin\") pod \"multus-jsbkg\" (UID: \"e85c458d-f954-41bc-a06d-1b84854762ee\") " pod="openshift-multus/multus-jsbkg" Jul 06 20:08:12.145467 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.142874 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/645eb1ca-38b9-4599-80d5-cf4314f701c9-host-kubelet\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.145467 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.142918 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4c726ad1-d957-471d-95d9-fcaee1f227e8-os-release\") pod \"multus-additional-cni-plugins-vgxs4\" (UID: \"4c726ad1-d957-471d-95d9-fcaee1f227e8\") " pod="openshift-multus/multus-additional-cni-plugins-vgxs4" Jul 06 20:08:12.145467 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.142968 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctl7x\" (UniqueName: \"kubernetes.io/projected/b7b05003-ad5f-4f11-bc35-476e28673d12-kube-api-access-ctl7x\") pod \"iptables-alerter-55nk7\" (UID: \"b7b05003-ad5f-4f11-bc35-476e28673d12\") " pod="openshift-network-operator/iptables-alerter-55nk7" Jul 06 20:08:12.145467 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.142974 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e85c458d-f954-41bc-a06d-1b84854762ee-cnibin\") pod \"multus-jsbkg\" (UID: \"e85c458d-f954-41bc-a06d-1b84854762ee\") " pod="openshift-multus/multus-jsbkg" Jul 06 20:08:12.145467 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.142999 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8cns\" (UniqueName: \"kubernetes.io/projected/ef16cb88-f1e2-49c7-a470-47a23730758d-kube-api-access-w8cns\") pod \"network-metrics-daemon-v648r\" (UID: \"ef16cb88-f1e2-49c7-a470-47a23730758d\") " pod="openshift-multus/network-metrics-daemon-v648r" Jul 06 20:08:12.145467 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.143024 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/645eb1ca-38b9-4599-80d5-cf4314f701c9-log-socket\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.145467 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.143059 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/645eb1ca-38b9-4599-80d5-cf4314f701c9-ovn-node-metrics-cert\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.145467 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.143089 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e85c458d-f954-41bc-a06d-1b84854762ee-cni-binary-copy\") pod \"multus-jsbkg\" (UID: \"e85c458d-f954-41bc-a06d-1b84854762ee\") " pod="openshift-multus/multus-jsbkg" Jul 06 20:08:12.145467 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.143113 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e85c458d-f954-41bc-a06d-1b84854762ee-host-var-lib-cni-bin\") pod \"multus-jsbkg\" (UID: \"e85c458d-f954-41bc-a06d-1b84854762ee\") " pod="openshift-multus/multus-jsbkg" Jul 06 20:08:12.145467 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.143164 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e85c458d-f954-41bc-a06d-1b84854762ee-host-var-lib-cni-bin\") pod \"multus-jsbkg\" (UID: \"e85c458d-f954-41bc-a06d-1b84854762ee\") " pod="openshift-multus/multus-jsbkg" Jul 06 20:08:12.145467 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.143210 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/645eb1ca-38b9-4599-80d5-cf4314f701c9-log-socket\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.146376 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.143238 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f3415158-a586-4fd7-bf83-bbd81db28b49-socket-dir\") pod \"aws-ebs-csi-driver-node-9cb25\" (UID: \"f3415158-a586-4fd7-bf83-bbd81db28b49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9cb25" Jul 06 20:08:12.146376 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.143264 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/953dca0f-091c-4aa1-90bc-3424c89859af-etc-sysctl-d\") pod \"tuned-b6fnm\" (UID: \"953dca0f-091c-4aa1-90bc-3424c89859af\") " pod="openshift-cluster-node-tuning-operator/tuned-b6fnm" Jul 06 20:08:12.146376 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.143289 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e85c458d-f954-41bc-a06d-1b84854762ee-hostroot\") pod \"multus-jsbkg\" (UID: \"e85c458d-f954-41bc-a06d-1b84854762ee\") " pod="openshift-multus/multus-jsbkg" Jul 06 20:08:12.146376 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.143315 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e85c458d-f954-41bc-a06d-1b84854762ee-multus-daemon-config\") pod \"multus-jsbkg\" (UID: \"e85c458d-f954-41bc-a06d-1b84854762ee\") " pod="openshift-multus/multus-jsbkg" Jul 06 20:08:12.146376 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.143430 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/953dca0f-091c-4aa1-90bc-3424c89859af-etc-sysctl-d\") pod \"tuned-b6fnm\" (UID: \"953dca0f-091c-4aa1-90bc-3424c89859af\") " pod="openshift-cluster-node-tuning-operator/tuned-b6fnm" Jul 06 20:08:12.146376 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.143451 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e85c458d-f954-41bc-a06d-1b84854762ee-hostroot\") pod \"multus-jsbkg\" (UID: \"e85c458d-f954-41bc-a06d-1b84854762ee\") " pod="openshift-multus/multus-jsbkg" Jul 06 20:08:12.146376 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.143474 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4c726ad1-d957-471d-95d9-fcaee1f227e8-cni-binary-copy\") pod \"multus-additional-cni-plugins-vgxs4\" (UID: \"4c726ad1-d957-471d-95d9-fcaee1f227e8\") " pod="openshift-multus/multus-additional-cni-plugins-vgxs4" Jul 06 20:08:12.146376 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.143507 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f3415158-a586-4fd7-bf83-bbd81db28b49-socket-dir\") pod \"aws-ebs-csi-driver-node-9cb25\" (UID: \"f3415158-a586-4fd7-bf83-bbd81db28b49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9cb25" Jul 06 20:08:12.146376 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.143515 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef16cb88-f1e2-49c7-a470-47a23730758d-metrics-certs\") pod \"network-metrics-daemon-v648r\" (UID: \"ef16cb88-f1e2-49c7-a470-47a23730758d\") " pod="openshift-multus/network-metrics-daemon-v648r" Jul 06 20:08:12.146376 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.143554 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/645eb1ca-38b9-4599-80d5-cf4314f701c9-ovnkube-config\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.146376 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.143586 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e85c458d-f954-41bc-a06d-1b84854762ee-multus-conf-dir\") pod \"multus-jsbkg\" (UID: \"e85c458d-f954-41bc-a06d-1b84854762ee\") " pod="openshift-multus/multus-jsbkg" Jul 06 20:08:12.146376 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:12.143624 2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 20:08:12.146376 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.143664 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e85c458d-f954-41bc-a06d-1b84854762ee-multus-conf-dir\") pod \"multus-jsbkg\" (UID: \"e85c458d-f954-41bc-a06d-1b84854762ee\") " pod="openshift-multus/multus-jsbkg" Jul 06 20:08:12.146376 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:12.143678 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef16cb88-f1e2-49c7-a470-47a23730758d-metrics-certs podName:ef16cb88-f1e2-49c7-a470-47a23730758d nodeName:}" failed. No retries permitted until 2026-07-06 20:08:12.643661908 +0000 UTC m=+3.058861708 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef16cb88-f1e2-49c7-a470-47a23730758d-metrics-certs") pod "network-metrics-daemon-v648r" (UID: "ef16cb88-f1e2-49c7-a470-47a23730758d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 20:08:12.146376 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.143804 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e85c458d-f954-41bc-a06d-1b84854762ee-cni-binary-copy\") pod \"multus-jsbkg\" (UID: \"e85c458d-f954-41bc-a06d-1b84854762ee\") " pod="openshift-multus/multus-jsbkg" Jul 06 20:08:12.146376 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.144009 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e85c458d-f954-41bc-a06d-1b84854762ee-multus-daemon-config\") pod \"multus-jsbkg\" (UID: \"e85c458d-f954-41bc-a06d-1b84854762ee\") " pod="openshift-multus/multus-jsbkg" Jul 06 20:08:12.146376 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.144250 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/645eb1ca-38b9-4599-80d5-cf4314f701c9-ovnkube-config\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.147137 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.144603 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/953dca0f-091c-4aa1-90bc-3424c89859af-etc-tuned\") pod \"tuned-b6fnm\" (UID: \"953dca0f-091c-4aa1-90bc-3424c89859af\") " pod="openshift-cluster-node-tuning-operator/tuned-b6fnm" Jul 06 20:08:12.147137 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.144777 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/953dca0f-091c-4aa1-90bc-3424c89859af-tmp\") pod \"tuned-b6fnm\" (UID: \"953dca0f-091c-4aa1-90bc-3424c89859af\") " pod="openshift-cluster-node-tuning-operator/tuned-b6fnm" Jul 06 20:08:12.147137 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.146505 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/645eb1ca-38b9-4599-80d5-cf4314f701c9-ovn-node-metrics-cert\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.156944 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.156913 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7vjq\" (UniqueName: \"kubernetes.io/projected/953dca0f-091c-4aa1-90bc-3424c89859af-kube-api-access-k7vjq\") pod \"tuned-b6fnm\" (UID: \"953dca0f-091c-4aa1-90bc-3424c89859af\") " pod="openshift-cluster-node-tuning-operator/tuned-b6fnm" Jul 06 20:08:12.157119 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.157054 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8cns\" (UniqueName: \"kubernetes.io/projected/ef16cb88-f1e2-49c7-a470-47a23730758d-kube-api-access-w8cns\") pod \"network-metrics-daemon-v648r\" (UID: \"ef16cb88-f1e2-49c7-a470-47a23730758d\") " pod="openshift-multus/network-metrics-daemon-v648r" Jul 06 20:08:12.157604 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.157561 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbscn\" (UniqueName: \"kubernetes.io/projected/645eb1ca-38b9-4599-80d5-cf4314f701c9-kube-api-access-jbscn\") pod \"ovnkube-node-r7s8d\" (UID: \"645eb1ca-38b9-4599-80d5-cf4314f701c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.157937 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.157914 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lf8r\" (UniqueName: \"kubernetes.io/projected/854b635c-f1e7-47d5-bf88-b5a80b572b1b-kube-api-access-2lf8r\") pod \"node-resolver-477lt\" (UID: \"854b635c-f1e7-47d5-bf88-b5a80b572b1b\") " pod="openshift-dns/node-resolver-477lt" Jul 06 20:08:12.158395 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.158353 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctl7x\" (UniqueName: \"kubernetes.io/projected/b7b05003-ad5f-4f11-bc35-476e28673d12-kube-api-access-ctl7x\") pod \"iptables-alerter-55nk7\" (UID: \"b7b05003-ad5f-4f11-bc35-476e28673d12\") " pod="openshift-network-operator/iptables-alerter-55nk7" Jul 06 20:08:12.158475 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.158379 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2l8xc\" (UniqueName: \"kubernetes.io/projected/e85c458d-f954-41bc-a06d-1b84854762ee-kube-api-access-2l8xc\") pod \"multus-jsbkg\" (UID: \"e85c458d-f954-41bc-a06d-1b84854762ee\") " pod="openshift-multus/multus-jsbkg" Jul 06 20:08:12.159036 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.159008 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmdgp\" (UniqueName: \"kubernetes.io/projected/f3415158-a586-4fd7-bf83-bbd81db28b49-kube-api-access-hmdgp\") pod \"aws-ebs-csi-driver-node-9cb25\" (UID: \"f3415158-a586-4fd7-bf83-bbd81db28b49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9cb25" Jul 06 20:08:12.234980 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.234946 2551 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 20:08:12.244020 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.243984 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9cd8d42e-db03-477c-8ef9-bfed943eadb3-serviceca\") pod \"node-ca-mv9hq\" (UID: \"9cd8d42e-db03-477c-8ef9-bfed943eadb3\") " pod="openshift-image-registry/node-ca-mv9hq" Jul 06 20:08:12.244221 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.244031 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4c726ad1-d957-471d-95d9-fcaee1f227e8-system-cni-dir\") pod \"multus-additional-cni-plugins-vgxs4\" (UID: \"4c726ad1-d957-471d-95d9-fcaee1f227e8\") " pod="openshift-multus/multus-additional-cni-plugins-vgxs4" Jul 06 20:08:12.244221 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.244064 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1bbf9b1e-36f5-487b-a66e-55dc425fd1a0-agent-certs\") pod \"konnectivity-agent-5b2wn\" (UID: \"1bbf9b1e-36f5-487b-a66e-55dc425fd1a0\") " pod="kube-system/konnectivity-agent-5b2wn" Jul 06 20:08:12.244221 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.244090 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1bbf9b1e-36f5-487b-a66e-55dc425fd1a0-konnectivity-ca\") pod \"konnectivity-agent-5b2wn\" (UID: \"1bbf9b1e-36f5-487b-a66e-55dc425fd1a0\") " pod="kube-system/konnectivity-agent-5b2wn" Jul 06 20:08:12.244221 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.244118 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9cd8d42e-db03-477c-8ef9-bfed943eadb3-host\") pod \"node-ca-mv9hq\" (UID: \"9cd8d42e-db03-477c-8ef9-bfed943eadb3\") " pod="openshift-image-registry/node-ca-mv9hq" Jul 06 20:08:12.244221 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.244139 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4c726ad1-d957-471d-95d9-fcaee1f227e8-system-cni-dir\") pod \"multus-additional-cni-plugins-vgxs4\" (UID: \"4c726ad1-d957-471d-95d9-fcaee1f227e8\") " pod="openshift-multus/multus-additional-cni-plugins-vgxs4" Jul 06 20:08:12.244221 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.244145 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4c726ad1-d957-471d-95d9-fcaee1f227e8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vgxs4\" (UID: \"4c726ad1-d957-471d-95d9-fcaee1f227e8\") " pod="openshift-multus/multus-additional-cni-plugins-vgxs4" Jul 06 20:08:12.244221 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.244208 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5tzx\" (UniqueName: \"kubernetes.io/projected/4c726ad1-d957-471d-95d9-fcaee1f227e8-kube-api-access-l5tzx\") pod \"multus-additional-cni-plugins-vgxs4\" (UID: \"4c726ad1-d957-471d-95d9-fcaee1f227e8\") " pod="openshift-multus/multus-additional-cni-plugins-vgxs4" Jul 06 20:08:12.244591 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.244256 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlm8g\" (UniqueName: \"kubernetes.io/projected/9cd8d42e-db03-477c-8ef9-bfed943eadb3-kube-api-access-hlm8g\") pod \"node-ca-mv9hq\" (UID: \"9cd8d42e-db03-477c-8ef9-bfed943eadb3\") " pod="openshift-image-registry/node-ca-mv9hq" Jul 06 20:08:12.244591 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.244286 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4c726ad1-d957-471d-95d9-fcaee1f227e8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vgxs4\" (UID: \"4c726ad1-d957-471d-95d9-fcaee1f227e8\") " pod="openshift-multus/multus-additional-cni-plugins-vgxs4" Jul 06 20:08:12.244591 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.244313 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4c726ad1-d957-471d-95d9-fcaee1f227e8-os-release\") pod \"multus-additional-cni-plugins-vgxs4\" (UID: \"4c726ad1-d957-471d-95d9-fcaee1f227e8\") " pod="openshift-multus/multus-additional-cni-plugins-vgxs4" Jul 06 20:08:12.244591 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.244349 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4c726ad1-d957-471d-95d9-fcaee1f227e8-cni-binary-copy\") pod \"multus-additional-cni-plugins-vgxs4\" (UID: \"4c726ad1-d957-471d-95d9-fcaee1f227e8\") " pod="openshift-multus/multus-additional-cni-plugins-vgxs4" Jul 06 20:08:12.244591 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.244395 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4c726ad1-d957-471d-95d9-fcaee1f227e8-cnibin\") pod \"multus-additional-cni-plugins-vgxs4\" (UID: \"4c726ad1-d957-471d-95d9-fcaee1f227e8\") " pod="openshift-multus/multus-additional-cni-plugins-vgxs4" Jul 06 20:08:12.244591 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.244422 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4c726ad1-d957-471d-95d9-fcaee1f227e8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vgxs4\" (UID: \"4c726ad1-d957-471d-95d9-fcaee1f227e8\") " pod="openshift-multus/multus-additional-cni-plugins-vgxs4" Jul 06 20:08:12.244591 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.244472 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ds27q\" (UniqueName: \"kubernetes.io/projected/0899bbed-8629-4dd1-a6d0-be26f7870c7f-kube-api-access-ds27q\") pod \"network-check-target-6x889\" (UID: \"0899bbed-8629-4dd1-a6d0-be26f7870c7f\") " pod="openshift-network-diagnostics/network-check-target-6x889" Jul 06 20:08:12.244944 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.244638 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9cd8d42e-db03-477c-8ef9-bfed943eadb3-serviceca\") pod \"node-ca-mv9hq\" (UID: \"9cd8d42e-db03-477c-8ef9-bfed943eadb3\") " pod="openshift-image-registry/node-ca-mv9hq" Jul 06 20:08:12.244944 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.244756 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4c726ad1-d957-471d-95d9-fcaee1f227e8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vgxs4\" (UID: \"4c726ad1-d957-471d-95d9-fcaee1f227e8\") " pod="openshift-multus/multus-additional-cni-plugins-vgxs4" Jul 06 20:08:12.244944 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.244930 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4c726ad1-d957-471d-95d9-fcaee1f227e8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vgxs4\" (UID: \"4c726ad1-d957-471d-95d9-fcaee1f227e8\") " pod="openshift-multus/multus-additional-cni-plugins-vgxs4" Jul 06 20:08:12.245153 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.245024 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4c726ad1-d957-471d-95d9-fcaee1f227e8-os-release\") pod \"multus-additional-cni-plugins-vgxs4\" (UID: \"4c726ad1-d957-471d-95d9-fcaee1f227e8\") " pod="openshift-multus/multus-additional-cni-plugins-vgxs4" Jul 06 20:08:12.245153 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.245051 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9cd8d42e-db03-477c-8ef9-bfed943eadb3-host\") pod \"node-ca-mv9hq\" (UID: \"9cd8d42e-db03-477c-8ef9-bfed943eadb3\") " pod="openshift-image-registry/node-ca-mv9hq" Jul 06 20:08:12.245153 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.245093 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4c726ad1-d957-471d-95d9-fcaee1f227e8-cnibin\") pod \"multus-additional-cni-plugins-vgxs4\" (UID: \"4c726ad1-d957-471d-95d9-fcaee1f227e8\") " pod="openshift-multus/multus-additional-cni-plugins-vgxs4" Jul 06 20:08:12.245321 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.245256 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1bbf9b1e-36f5-487b-a66e-55dc425fd1a0-konnectivity-ca\") pod \"konnectivity-agent-5b2wn\" (UID: \"1bbf9b1e-36f5-487b-a66e-55dc425fd1a0\") " pod="kube-system/konnectivity-agent-5b2wn" Jul 06 20:08:12.245545 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.245519 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4c726ad1-d957-471d-95d9-fcaee1f227e8-cni-binary-copy\") pod \"multus-additional-cni-plugins-vgxs4\" (UID: \"4c726ad1-d957-471d-95d9-fcaee1f227e8\") " pod="openshift-multus/multus-additional-cni-plugins-vgxs4" Jul 06 20:08:12.245856 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.245833 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4c726ad1-d957-471d-95d9-fcaee1f227e8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vgxs4\" (UID: \"4c726ad1-d957-471d-95d9-fcaee1f227e8\") " pod="openshift-multus/multus-additional-cni-plugins-vgxs4" Jul 06 20:08:12.248233 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.248211 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1bbf9b1e-36f5-487b-a66e-55dc425fd1a0-agent-certs\") pod \"konnectivity-agent-5b2wn\" (UID: \"1bbf9b1e-36f5-487b-a66e-55dc425fd1a0\") " pod="kube-system/konnectivity-agent-5b2wn" Jul 06 20:08:12.251846 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:12.251786 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 20:08:12.251846 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:12.251818 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 20:08:12.251846 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:12.251835 2551 projected.go:196] Error preparing data for projected volume kube-api-access-ds27q for pod openshift-network-diagnostics/network-check-target-6x889: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 20:08:12.252081 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:12.251920 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0899bbed-8629-4dd1-a6d0-be26f7870c7f-kube-api-access-ds27q podName:0899bbed-8629-4dd1-a6d0-be26f7870c7f nodeName:}" failed. No retries permitted until 2026-07-06 20:08:12.751899279 +0000 UTC m=+3.167099086 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-ds27q" (UniqueName: "kubernetes.io/projected/0899bbed-8629-4dd1-a6d0-be26f7870c7f-kube-api-access-ds27q") pod "network-check-target-6x889" (UID: "0899bbed-8629-4dd1-a6d0-be26f7870c7f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 20:08:12.254874 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.254846 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlm8g\" (UniqueName: \"kubernetes.io/projected/9cd8d42e-db03-477c-8ef9-bfed943eadb3-kube-api-access-hlm8g\") pod \"node-ca-mv9hq\" (UID: \"9cd8d42e-db03-477c-8ef9-bfed943eadb3\") " pod="openshift-image-registry/node-ca-mv9hq" Jul 06 20:08:12.255830 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.255799 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5tzx\" (UniqueName: \"kubernetes.io/projected/4c726ad1-d957-471d-95d9-fcaee1f227e8-kube-api-access-l5tzx\") pod \"multus-additional-cni-plugins-vgxs4\" (UID: \"4c726ad1-d957-471d-95d9-fcaee1f227e8\") " pod="openshift-multus/multus-additional-cni-plugins-vgxs4" Jul 06 20:08:12.328683 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.328651 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-55nk7" Jul 06 20:08:12.340606 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.340580 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:12.351912 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.351876 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9cb25" Jul 06 20:08:12.356758 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.356732 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-b6fnm" Jul 06 20:08:12.364790 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.364766 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jsbkg" Jul 06 20:08:12.373779 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.373754 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-477lt" Jul 06 20:08:12.383855 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.383830 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5b2wn" Jul 06 20:08:12.389987 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.389961 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mv9hq" Jul 06 20:08:12.396080 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.396054 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vgxs4" Jul 06 20:08:12.648359 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.648275 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef16cb88-f1e2-49c7-a470-47a23730758d-metrics-certs\") pod \"network-metrics-daemon-v648r\" (UID: \"ef16cb88-f1e2-49c7-a470-47a23730758d\") " pod="openshift-multus/network-metrics-daemon-v648r" Jul 06 20:08:12.648359 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.648322 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6909872-c920-4e38-a17a-11ad51d50379-original-pull-secret\") pod \"global-pull-secret-syncer-g6t4k\" (UID: \"d6909872-c920-4e38-a17a-11ad51d50379\") " pod="kube-system/global-pull-secret-syncer-g6t4k" Jul 06 20:08:12.648559 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:12.648434 2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 20:08:12.648559 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:12.648443 2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 20:08:12.648559 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:12.648503 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6909872-c920-4e38-a17a-11ad51d50379-original-pull-secret podName:d6909872-c920-4e38-a17a-11ad51d50379 nodeName:}" failed. No retries permitted until 2026-07-06 20:08:13.648483603 +0000 UTC m=+4.063683409 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d6909872-c920-4e38-a17a-11ad51d50379-original-pull-secret") pod "global-pull-secret-syncer-g6t4k" (UID: "d6909872-c920-4e38-a17a-11ad51d50379") : object "kube-system"/"original-pull-secret" not registered Jul 06 20:08:12.648559 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:12.648519 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef16cb88-f1e2-49c7-a470-47a23730758d-metrics-certs podName:ef16cb88-f1e2-49c7-a470-47a23730758d nodeName:}" failed. No retries permitted until 2026-07-06 20:08:13.648512375 +0000 UTC m=+4.063712160 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef16cb88-f1e2-49c7-a470-47a23730758d-metrics-certs") pod "network-metrics-daemon-v648r" (UID: "ef16cb88-f1e2-49c7-a470-47a23730758d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 20:08:12.849857 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:12.849823 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ds27q\" (UniqueName: \"kubernetes.io/projected/0899bbed-8629-4dd1-a6d0-be26f7870c7f-kube-api-access-ds27q\") pod \"network-check-target-6x889\" (UID: \"0899bbed-8629-4dd1-a6d0-be26f7870c7f\") " pod="openshift-network-diagnostics/network-check-target-6x889" Jul 06 20:08:12.850033 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:12.849991 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 20:08:12.850033 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:12.850010 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 20:08:12.850033 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:12.850019 2551 projected.go:196] Error preparing data for projected volume kube-api-access-ds27q for pod openshift-network-diagnostics/network-check-target-6x889: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 20:08:12.850143 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:12.850067 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0899bbed-8629-4dd1-a6d0-be26f7870c7f-kube-api-access-ds27q podName:0899bbed-8629-4dd1-a6d0-be26f7870c7f nodeName:}" failed. No retries permitted until 2026-07-06 20:08:13.85005208 +0000 UTC m=+4.265251865 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-ds27q" (UniqueName: "kubernetes.io/projected/0899bbed-8629-4dd1-a6d0-be26f7870c7f-kube-api-access-ds27q") pod "network-check-target-6x889" (UID: "0899bbed-8629-4dd1-a6d0-be26f7870c7f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 20:08:13.084151 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:13.084100 2551 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 20:03:12 +0000 UTC" deadline="2028-01-10 21:58:01.700261344 +0000 UTC" Jul 06 20:08:13.084151 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:13.084143 2551 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13273h49m48.616122957s" Jul 06 20:08:13.159141 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:13.159111 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v648r" Jul 06 20:08:13.159320 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:13.159250 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v648r" podUID="ef16cb88-f1e2-49c7-a470-47a23730758d" Jul 06 20:08:13.409517 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:13.409489 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode85c458d_f954_41bc_a06d_1b84854762ee.slice/crio-f32c9624dac6040a09d656ae0a4dafa2d33902c543dee9d612cb8409ba270170 WatchSource:0}: Error finding container f32c9624dac6040a09d656ae0a4dafa2d33902c543dee9d612cb8409ba270170: Status 404 returned error can't find the container with id f32c9624dac6040a09d656ae0a4dafa2d33902c543dee9d612cb8409ba270170 Jul 06 20:08:13.410947 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:13.410875 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bbf9b1e_36f5_487b_a66e_55dc425fd1a0.slice/crio-2ec534fed041e19595ca0900cb40d8693ae5c13a8bdbbafc3599137f8f7108e9 WatchSource:0}: Error finding container 2ec534fed041e19595ca0900cb40d8693ae5c13a8bdbbafc3599137f8f7108e9: Status 404 returned error can't find the container with id 2ec534fed041e19595ca0900cb40d8693ae5c13a8bdbbafc3599137f8f7108e9 Jul 06 20:08:13.412988 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:13.412725 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c726ad1_d957_471d_95d9_fcaee1f227e8.slice/crio-31e223c3ac82e102c3cf3be2a7587e2e08b94e1e964dda8fc8420de43a11f3c4 WatchSource:0}: Error finding container 31e223c3ac82e102c3cf3be2a7587e2e08b94e1e964dda8fc8420de43a11f3c4: Status 404 returned error can't find the container with id 31e223c3ac82e102c3cf3be2a7587e2e08b94e1e964dda8fc8420de43a11f3c4 Jul 06 20:08:13.413371 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:13.413341 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cd8d42e_db03_477c_8ef9_bfed943eadb3.slice/crio-eef224e055a043a2b05eca8cb00a47d05bb95fc7ffb6b2e807c484374434845a WatchSource:0}: Error finding container eef224e055a043a2b05eca8cb00a47d05bb95fc7ffb6b2e807c484374434845a: Status 404 returned error can't find the container with id eef224e055a043a2b05eca8cb00a47d05bb95fc7ffb6b2e807c484374434845a Jul 06 20:08:13.414925 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:13.414880 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3415158_a586_4fd7_bf83_bbd81db28b49.slice/crio-59d006dc07c90faf9a69108353b4e306fc76b1254a1cbb1781c8276490924c09 WatchSource:0}: Error finding container 59d006dc07c90faf9a69108353b4e306fc76b1254a1cbb1781c8276490924c09: Status 404 returned error can't find the container with id 59d006dc07c90faf9a69108353b4e306fc76b1254a1cbb1781c8276490924c09 Jul 06 20:08:13.416250 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:13.416225 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod953dca0f_091c_4aa1_90bc_3424c89859af.slice/crio-447883034ae5de8bf970d456e768018dc839a70da503f75cb0ceb00f3044a312 WatchSource:0}: Error finding container 447883034ae5de8bf970d456e768018dc839a70da503f75cb0ceb00f3044a312: Status 404 returned error can't find the container with id 447883034ae5de8bf970d456e768018dc839a70da503f75cb0ceb00f3044a312 Jul 06 20:08:13.420911 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:13.420809 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7b05003_ad5f_4f11_bc35_476e28673d12.slice/crio-97539fcc4cab2580a59223112fadc637208c32ccc08c686558c926c9827fa7f4 WatchSource:0}: Error finding container 97539fcc4cab2580a59223112fadc637208c32ccc08c686558c926c9827fa7f4: Status 404 returned error can't find the container with id 97539fcc4cab2580a59223112fadc637208c32ccc08c686558c926c9827fa7f4 Jul 06 20:08:13.421951 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:13.421464 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod645eb1ca_38b9_4599_80d5_cf4314f701c9.slice/crio-2f9d61cd9aa4ce39cfb27c606b529ac412077ccea648ebab08e990678eb2dee1 WatchSource:0}: Error finding container 2f9d61cd9aa4ce39cfb27c606b529ac412077ccea648ebab08e990678eb2dee1: Status 404 returned error can't find the container with id 2f9d61cd9aa4ce39cfb27c606b529ac412077ccea648ebab08e990678eb2dee1 Jul 06 20:08:13.657390 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:13.657023 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef16cb88-f1e2-49c7-a470-47a23730758d-metrics-certs\") pod \"network-metrics-daemon-v648r\" (UID: \"ef16cb88-f1e2-49c7-a470-47a23730758d\") " pod="openshift-multus/network-metrics-daemon-v648r" Jul 06 20:08:13.657390 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:13.657360 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6909872-c920-4e38-a17a-11ad51d50379-original-pull-secret\") pod \"global-pull-secret-syncer-g6t4k\" (UID: \"d6909872-c920-4e38-a17a-11ad51d50379\") " pod="kube-system/global-pull-secret-syncer-g6t4k" Jul 06 20:08:13.657390 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:13.657207 2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 20:08:13.657614 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:13.657451 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef16cb88-f1e2-49c7-a470-47a23730758d-metrics-certs podName:ef16cb88-f1e2-49c7-a470-47a23730758d nodeName:}" failed. No retries permitted until 2026-07-06 20:08:15.657429626 +0000 UTC m=+6.072629416 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef16cb88-f1e2-49c7-a470-47a23730758d-metrics-certs") pod "network-metrics-daemon-v648r" (UID: "ef16cb88-f1e2-49c7-a470-47a23730758d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 20:08:13.657614 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:13.657498 2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 20:08:13.657614 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:13.657545 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6909872-c920-4e38-a17a-11ad51d50379-original-pull-secret podName:d6909872-c920-4e38-a17a-11ad51d50379 nodeName:}" failed. No retries permitted until 2026-07-06 20:08:15.657530851 +0000 UTC m=+6.072730640 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d6909872-c920-4e38-a17a-11ad51d50379-original-pull-secret") pod "global-pull-secret-syncer-g6t4k" (UID: "d6909872-c920-4e38-a17a-11ad51d50379") : object "kube-system"/"original-pull-secret" not registered Jul 06 20:08:13.860406 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:13.860372 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ds27q\" (UniqueName: \"kubernetes.io/projected/0899bbed-8629-4dd1-a6d0-be26f7870c7f-kube-api-access-ds27q\") pod \"network-check-target-6x889\" (UID: \"0899bbed-8629-4dd1-a6d0-be26f7870c7f\") " pod="openshift-network-diagnostics/network-check-target-6x889" Jul 06 20:08:13.860606 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:13.860580 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 20:08:13.860658 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:13.860617 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 20:08:13.860658 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:13.860631 2551 projected.go:196] Error preparing data for projected volume kube-api-access-ds27q for pod openshift-network-diagnostics/network-check-target-6x889: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 20:08:13.863640 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:13.861150 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0899bbed-8629-4dd1-a6d0-be26f7870c7f-kube-api-access-ds27q podName:0899bbed-8629-4dd1-a6d0-be26f7870c7f nodeName:}" failed. No retries permitted until 2026-07-06 20:08:15.860680782 +0000 UTC m=+6.275880570 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-ds27q" (UniqueName: "kubernetes.io/projected/0899bbed-8629-4dd1-a6d0-be26f7870c7f-kube-api-access-ds27q") pod "network-check-target-6x889" (UID: "0899bbed-8629-4dd1-a6d0-be26f7870c7f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 20:08:14.084780 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:14.084712 2551 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 20:03:12 +0000 UTC" deadline="2028-02-17 10:28:17.23827575 +0000 UTC" Jul 06 20:08:14.084780 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:14.084777 2551 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14174h20m3.153502957s" Jul 06 20:08:14.175352 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:14.174320 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6x889" Jul 06 20:08:14.175352 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:14.174440 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6x889" podUID="0899bbed-8629-4dd1-a6d0-be26f7870c7f" Jul 06 20:08:14.175352 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:14.174838 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6t4k" Jul 06 20:08:14.175352 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:14.174930 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6t4k" podUID="d6909872-c920-4e38-a17a-11ad51d50379" Jul 06 20:08:14.179809 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:14.178968 2551 generic.go:356] "Generic (PLEG): container finished" podID="e99ab74f56459d2f57aab6c6fbd79614" containerID="f5a318ff8429143a3d3875e32b19262ea8fcc54003417d087a02bdfee599b5cd" exitCode=0 Jul 06 20:08:14.179809 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:14.179061 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-178.ec2.internal" event={"ID":"e99ab74f56459d2f57aab6c6fbd79614","Type":"ContainerDied","Data":"f5a318ff8429143a3d3875e32b19262ea8fcc54003417d087a02bdfee599b5cd"} Jul 06 20:08:14.190795 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:14.190701 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-178.ec2.internal" event={"ID":"ac36f75d0ec38f2ed1aabc8706b8a395","Type":"ContainerStarted","Data":"d87d9a9e552be2b7e89a5f7b15610f634b99de70ebdb7d10e92b096a4e51595d"} Jul 06 20:08:14.200559 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:14.200436 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" event={"ID":"645eb1ca-38b9-4599-80d5-cf4314f701c9","Type":"ContainerStarted","Data":"2f9d61cd9aa4ce39cfb27c606b529ac412077ccea648ebab08e990678eb2dee1"} Jul 06 20:08:14.208302 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:14.208244 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-477lt" event={"ID":"854b635c-f1e7-47d5-bf88-b5a80b572b1b","Type":"ContainerStarted","Data":"4064f486904155a4bde25851ad1f42a26118ef0a3ac46b840dc0bb8194490269"} Jul 06 20:08:14.216866 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:14.216835 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-b6fnm" event={"ID":"953dca0f-091c-4aa1-90bc-3424c89859af","Type":"ContainerStarted","Data":"447883034ae5de8bf970d456e768018dc839a70da503f75cb0ceb00f3044a312"} Jul 06 20:08:14.222634 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:14.222582 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-178.ec2.internal" podStartSLOduration=3.222563019 podStartE2EDuration="3.222563019s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:08:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:08:14.222545273 +0000 UTC m=+4.637745083" watchObservedRunningTime="2026-07-06 20:08:14.222563019 +0000 UTC m=+4.637762826" Jul 06 20:08:14.226041 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:14.226011 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mv9hq" event={"ID":"9cd8d42e-db03-477c-8ef9-bfed943eadb3","Type":"ContainerStarted","Data":"eef224e055a043a2b05eca8cb00a47d05bb95fc7ffb6b2e807c484374434845a"} Jul 06 20:08:14.240448 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:14.240410 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5b2wn" event={"ID":"1bbf9b1e-36f5-487b-a66e-55dc425fd1a0","Type":"ContainerStarted","Data":"2ec534fed041e19595ca0900cb40d8693ae5c13a8bdbbafc3599137f8f7108e9"} Jul 06 20:08:14.257819 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:14.257743 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vgxs4" event={"ID":"4c726ad1-d957-471d-95d9-fcaee1f227e8","Type":"ContainerStarted","Data":"31e223c3ac82e102c3cf3be2a7587e2e08b94e1e964dda8fc8420de43a11f3c4"} Jul 06 20:08:14.269623 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:14.269532 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-55nk7" event={"ID":"b7b05003-ad5f-4f11-bc35-476e28673d12","Type":"ContainerStarted","Data":"97539fcc4cab2580a59223112fadc637208c32ccc08c686558c926c9827fa7f4"} Jul 06 20:08:14.278400 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:14.278359 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9cb25" event={"ID":"f3415158-a586-4fd7-bf83-bbd81db28b49","Type":"ContainerStarted","Data":"59d006dc07c90faf9a69108353b4e306fc76b1254a1cbb1781c8276490924c09"} Jul 06 20:08:14.301269 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:14.301228 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jsbkg" event={"ID":"e85c458d-f954-41bc-a06d-1b84854762ee","Type":"ContainerStarted","Data":"f32c9624dac6040a09d656ae0a4dafa2d33902c543dee9d612cb8409ba270170"} Jul 06 20:08:15.158570 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:15.158530 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v648r" Jul 06 20:08:15.159110 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:15.158701 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v648r" podUID="ef16cb88-f1e2-49c7-a470-47a23730758d" Jul 06 20:08:15.320112 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:15.320033 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-178.ec2.internal" event={"ID":"e99ab74f56459d2f57aab6c6fbd79614","Type":"ContainerStarted","Data":"292aaa6bc0471bb629e90a01d9a0c2f0d8045ad304050ea3ab0a368ca35b8270"} Jul 06 20:08:15.677600 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:15.677557 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef16cb88-f1e2-49c7-a470-47a23730758d-metrics-certs\") pod \"network-metrics-daemon-v648r\" (UID: \"ef16cb88-f1e2-49c7-a470-47a23730758d\") " pod="openshift-multus/network-metrics-daemon-v648r" Jul 06 20:08:15.677803 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:15.677639 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6909872-c920-4e38-a17a-11ad51d50379-original-pull-secret\") pod \"global-pull-secret-syncer-g6t4k\" (UID: \"d6909872-c920-4e38-a17a-11ad51d50379\") " pod="kube-system/global-pull-secret-syncer-g6t4k" Jul 06 20:08:15.677803 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:15.677793 2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 20:08:15.677919 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:15.677857 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6909872-c920-4e38-a17a-11ad51d50379-original-pull-secret podName:d6909872-c920-4e38-a17a-11ad51d50379 nodeName:}" failed. No retries permitted until 2026-07-06 20:08:19.677838093 +0000 UTC m=+10.093037881 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d6909872-c920-4e38-a17a-11ad51d50379-original-pull-secret") pod "global-pull-secret-syncer-g6t4k" (UID: "d6909872-c920-4e38-a17a-11ad51d50379") : object "kube-system"/"original-pull-secret" not registered Jul 06 20:08:15.678131 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:15.678104 2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 20:08:15.678231 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:15.678183 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef16cb88-f1e2-49c7-a470-47a23730758d-metrics-certs podName:ef16cb88-f1e2-49c7-a470-47a23730758d nodeName:}" failed. No retries permitted until 2026-07-06 20:08:19.67816563 +0000 UTC m=+10.093365418 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef16cb88-f1e2-49c7-a470-47a23730758d-metrics-certs") pod "network-metrics-daemon-v648r" (UID: "ef16cb88-f1e2-49c7-a470-47a23730758d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 20:08:15.880727 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:15.880084 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ds27q\" (UniqueName: \"kubernetes.io/projected/0899bbed-8629-4dd1-a6d0-be26f7870c7f-kube-api-access-ds27q\") pod \"network-check-target-6x889\" (UID: \"0899bbed-8629-4dd1-a6d0-be26f7870c7f\") " pod="openshift-network-diagnostics/network-check-target-6x889" Jul 06 20:08:15.880727 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:15.880250 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 20:08:15.880727 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:15.880269 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 20:08:15.880727 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:15.880281 2551 projected.go:196] Error preparing data for projected volume kube-api-access-ds27q for pod openshift-network-diagnostics/network-check-target-6x889: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 20:08:15.880727 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:15.880342 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0899bbed-8629-4dd1-a6d0-be26f7870c7f-kube-api-access-ds27q podName:0899bbed-8629-4dd1-a6d0-be26f7870c7f nodeName:}" failed. No retries permitted until 2026-07-06 20:08:19.880322751 +0000 UTC m=+10.295522555 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-ds27q" (UniqueName: "kubernetes.io/projected/0899bbed-8629-4dd1-a6d0-be26f7870c7f-kube-api-access-ds27q") pod "network-check-target-6x889" (UID: "0899bbed-8629-4dd1-a6d0-be26f7870c7f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 20:08:16.160000 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:16.158781 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6t4k" Jul 06 20:08:16.160000 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:16.158941 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6t4k" podUID="d6909872-c920-4e38-a17a-11ad51d50379" Jul 06 20:08:16.160000 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:16.159365 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6x889" Jul 06 20:08:16.160000 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:16.159470 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6x889" podUID="0899bbed-8629-4dd1-a6d0-be26f7870c7f" Jul 06 20:08:17.159023 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:17.158990 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v648r" Jul 06 20:08:17.159270 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:17.159121 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v648r" podUID="ef16cb88-f1e2-49c7-a470-47a23730758d" Jul 06 20:08:18.161662 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:18.161489 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6t4k" Jul 06 20:08:18.161662 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:18.161582 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6t4k" podUID="d6909872-c920-4e38-a17a-11ad51d50379" Jul 06 20:08:18.161662 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:18.161634 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6x889" Jul 06 20:08:18.162249 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:18.161727 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6x889" podUID="0899bbed-8629-4dd1-a6d0-be26f7870c7f" Jul 06 20:08:19.158617 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:19.158506 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v648r" Jul 06 20:08:19.158816 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:19.158638 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v648r" podUID="ef16cb88-f1e2-49c7-a470-47a23730758d" Jul 06 20:08:19.715350 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:19.715312 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef16cb88-f1e2-49c7-a470-47a23730758d-metrics-certs\") pod \"network-metrics-daemon-v648r\" (UID: \"ef16cb88-f1e2-49c7-a470-47a23730758d\") " pod="openshift-multus/network-metrics-daemon-v648r" Jul 06 20:08:19.715836 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:19.715378 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6909872-c920-4e38-a17a-11ad51d50379-original-pull-secret\") pod \"global-pull-secret-syncer-g6t4k\" (UID: \"d6909872-c920-4e38-a17a-11ad51d50379\") " pod="kube-system/global-pull-secret-syncer-g6t4k" Jul 06 20:08:19.715836 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:19.715490 2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 20:08:19.715836 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:19.715573 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6909872-c920-4e38-a17a-11ad51d50379-original-pull-secret podName:d6909872-c920-4e38-a17a-11ad51d50379 nodeName:}" failed. No retries permitted until 2026-07-06 20:08:27.715549747 +0000 UTC m=+18.130749532 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d6909872-c920-4e38-a17a-11ad51d50379-original-pull-secret") pod "global-pull-secret-syncer-g6t4k" (UID: "d6909872-c920-4e38-a17a-11ad51d50379") : object "kube-system"/"original-pull-secret" not registered Jul 06 20:08:19.715836 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:19.715490 2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 20:08:19.715836 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:19.715670 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef16cb88-f1e2-49c7-a470-47a23730758d-metrics-certs podName:ef16cb88-f1e2-49c7-a470-47a23730758d nodeName:}" failed. No retries permitted until 2026-07-06 20:08:27.715635662 +0000 UTC m=+18.130835465 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef16cb88-f1e2-49c7-a470-47a23730758d-metrics-certs") pod "network-metrics-daemon-v648r" (UID: "ef16cb88-f1e2-49c7-a470-47a23730758d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 20:08:19.917079 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:19.917043 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ds27q\" (UniqueName: \"kubernetes.io/projected/0899bbed-8629-4dd1-a6d0-be26f7870c7f-kube-api-access-ds27q\") pod \"network-check-target-6x889\" (UID: \"0899bbed-8629-4dd1-a6d0-be26f7870c7f\") " pod="openshift-network-diagnostics/network-check-target-6x889" Jul 06 20:08:19.917251 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:19.917235 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 20:08:19.917326 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:19.917260 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 20:08:19.917326 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:19.917274 2551 projected.go:196] Error preparing data for projected volume kube-api-access-ds27q for pod openshift-network-diagnostics/network-check-target-6x889: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 20:08:19.917410 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:19.917343 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0899bbed-8629-4dd1-a6d0-be26f7870c7f-kube-api-access-ds27q podName:0899bbed-8629-4dd1-a6d0-be26f7870c7f nodeName:}" failed. No retries permitted until 2026-07-06 20:08:27.917322778 +0000 UTC m=+18.332522574 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-ds27q" (UniqueName: "kubernetes.io/projected/0899bbed-8629-4dd1-a6d0-be26f7870c7f-kube-api-access-ds27q") pod "network-check-target-6x889" (UID: "0899bbed-8629-4dd1-a6d0-be26f7870c7f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 20:08:20.158838 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:20.158791 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6x889" Jul 06 20:08:20.159024 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:20.158918 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6x889" podUID="0899bbed-8629-4dd1-a6d0-be26f7870c7f" Jul 06 20:08:20.159024 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:20.158926 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6t4k" Jul 06 20:08:20.159024 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:20.159008 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6t4k" podUID="d6909872-c920-4e38-a17a-11ad51d50379" Jul 06 20:08:21.158349 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:21.158312 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v648r" Jul 06 20:08:21.158923 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:21.158455 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v648r" podUID="ef16cb88-f1e2-49c7-a470-47a23730758d" Jul 06 20:08:22.158873 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:22.158838 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6x889" Jul 06 20:08:22.159333 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:22.158983 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6x889" podUID="0899bbed-8629-4dd1-a6d0-be26f7870c7f" Jul 06 20:08:22.159333 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:22.159058 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6t4k" Jul 06 20:08:22.159333 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:22.159163 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6t4k" podUID="d6909872-c920-4e38-a17a-11ad51d50379" Jul 06 20:08:23.158380 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:23.158350 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v648r" Jul 06 20:08:23.158542 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:23.158487 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v648r" podUID="ef16cb88-f1e2-49c7-a470-47a23730758d" Jul 06 20:08:24.158555 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:24.158515 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6t4k" Jul 06 20:08:24.159025 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:24.158642 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6t4k" podUID="d6909872-c920-4e38-a17a-11ad51d50379" Jul 06 20:08:24.159025 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:24.158693 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6x889" Jul 06 20:08:24.159025 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:24.158780 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6x889" podUID="0899bbed-8629-4dd1-a6d0-be26f7870c7f" Jul 06 20:08:25.158107 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:25.158065 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v648r" Jul 06 20:08:25.158277 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:25.158219 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v648r" podUID="ef16cb88-f1e2-49c7-a470-47a23730758d" Jul 06 20:08:26.161378 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:26.161344 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6t4k" Jul 06 20:08:26.161750 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:26.161476 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6t4k" podUID="d6909872-c920-4e38-a17a-11ad51d50379" Jul 06 20:08:26.161750 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:26.161512 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6x889" Jul 06 20:08:26.161750 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:26.161580 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6x889" podUID="0899bbed-8629-4dd1-a6d0-be26f7870c7f" Jul 06 20:08:27.158469 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:27.158437 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v648r" Jul 06 20:08:27.158656 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:27.158593 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v648r" podUID="ef16cb88-f1e2-49c7-a470-47a23730758d" Jul 06 20:08:27.779145 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:27.779106 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6909872-c920-4e38-a17a-11ad51d50379-original-pull-secret\") pod \"global-pull-secret-syncer-g6t4k\" (UID: \"d6909872-c920-4e38-a17a-11ad51d50379\") " pod="kube-system/global-pull-secret-syncer-g6t4k" Jul 06 20:08:27.779145 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:27.779175 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef16cb88-f1e2-49c7-a470-47a23730758d-metrics-certs\") pod \"network-metrics-daemon-v648r\" (UID: \"ef16cb88-f1e2-49c7-a470-47a23730758d\") " pod="openshift-multus/network-metrics-daemon-v648r" Jul 06 20:08:27.779542 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:27.779250 2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 20:08:27.779542 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:27.779259 2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 20:08:27.779542 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:27.779311 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6909872-c920-4e38-a17a-11ad51d50379-original-pull-secret podName:d6909872-c920-4e38-a17a-11ad51d50379 nodeName:}" failed. No retries permitted until 2026-07-06 20:08:43.779295844 +0000 UTC m=+34.194495627 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d6909872-c920-4e38-a17a-11ad51d50379-original-pull-secret") pod "global-pull-secret-syncer-g6t4k" (UID: "d6909872-c920-4e38-a17a-11ad51d50379") : object "kube-system"/"original-pull-secret" not registered Jul 06 20:08:27.779542 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:27.779328 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef16cb88-f1e2-49c7-a470-47a23730758d-metrics-certs podName:ef16cb88-f1e2-49c7-a470-47a23730758d nodeName:}" failed. No retries permitted until 2026-07-06 20:08:43.77931966 +0000 UTC m=+34.194519443 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef16cb88-f1e2-49c7-a470-47a23730758d-metrics-certs") pod "network-metrics-daemon-v648r" (UID: "ef16cb88-f1e2-49c7-a470-47a23730758d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 20:08:27.980250 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:27.980164 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ds27q\" (UniqueName: \"kubernetes.io/projected/0899bbed-8629-4dd1-a6d0-be26f7870c7f-kube-api-access-ds27q\") pod \"network-check-target-6x889\" (UID: \"0899bbed-8629-4dd1-a6d0-be26f7870c7f\") " pod="openshift-network-diagnostics/network-check-target-6x889" Jul 06 20:08:27.980412 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:27.980299 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 20:08:27.980412 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:27.980327 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 20:08:27.980412 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:27.980344 2551 projected.go:196] Error preparing data for projected volume kube-api-access-ds27q for pod openshift-network-diagnostics/network-check-target-6x889: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 20:08:27.980412 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:27.980405 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0899bbed-8629-4dd1-a6d0-be26f7870c7f-kube-api-access-ds27q podName:0899bbed-8629-4dd1-a6d0-be26f7870c7f nodeName:}" failed. No retries permitted until 2026-07-06 20:08:43.980390174 +0000 UTC m=+34.395589957 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-ds27q" (UniqueName: "kubernetes.io/projected/0899bbed-8629-4dd1-a6d0-be26f7870c7f-kube-api-access-ds27q") pod "network-check-target-6x889" (UID: "0899bbed-8629-4dd1-a6d0-be26f7870c7f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 20:08:28.161784 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:28.161755 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6x889" Jul 06 20:08:28.161976 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:28.161884 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6x889" podUID="0899bbed-8629-4dd1-a6d0-be26f7870c7f" Jul 06 20:08:28.161976 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:28.161923 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6t4k" Jul 06 20:08:28.162069 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:28.162030 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6t4k" podUID="d6909872-c920-4e38-a17a-11ad51d50379" Jul 06 20:08:29.158451 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:29.158420 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v648r" Jul 06 20:08:29.158826 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:29.158549 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v648r" podUID="ef16cb88-f1e2-49c7-a470-47a23730758d" Jul 06 20:08:30.161553 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:30.161525 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6x889" Jul 06 20:08:30.161553 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:30.161550 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6t4k" Jul 06 20:08:30.162053 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:30.161630 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6x889" podUID="0899bbed-8629-4dd1-a6d0-be26f7870c7f" Jul 06 20:08:30.162053 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:30.161782 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6t4k" podUID="d6909872-c920-4e38-a17a-11ad51d50379" Jul 06 20:08:31.158634 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:31.158598 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v648r" Jul 06 20:08:31.158828 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:31.158732 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v648r" podUID="ef16cb88-f1e2-49c7-a470-47a23730758d" Jul 06 20:08:32.161207 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:32.161178 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6x889" Jul 06 20:08:32.161652 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:32.161304 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6x889" podUID="0899bbed-8629-4dd1-a6d0-be26f7870c7f" Jul 06 20:08:32.161652 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:32.161348 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6t4k" Jul 06 20:08:32.161652 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:32.161433 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6t4k" podUID="d6909872-c920-4e38-a17a-11ad51d50379" Jul 06 20:08:33.158386 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:33.158118 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v648r" Jul 06 20:08:33.158543 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:33.158473 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v648r" podUID="ef16cb88-f1e2-49c7-a470-47a23730758d" Jul 06 20:08:33.360000 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:33.359968 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" event={"ID":"645eb1ca-38b9-4599-80d5-cf4314f701c9","Type":"ContainerStarted","Data":"42c5a9aded34e1b0cc6ec65fce7617a1c8d703a14fa293b82a4b6ba0887d24c2"} Jul 06 20:08:33.360701 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:33.360062 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" event={"ID":"645eb1ca-38b9-4599-80d5-cf4314f701c9","Type":"ContainerStarted","Data":"ddedc26e44ce63b759a5151c8a731b7f665f0abfa31899eeb47d09cdf3a74893"} Jul 06 20:08:33.360701 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:33.360117 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" event={"ID":"645eb1ca-38b9-4599-80d5-cf4314f701c9","Type":"ContainerStarted","Data":"2d63544a7852980ca81de34807fc556dc665bda3a3c08e6876d5cfa0e16e2776"} Jul 06 20:08:33.360701 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:33.360166 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" event={"ID":"645eb1ca-38b9-4599-80d5-cf4314f701c9","Type":"ContainerStarted","Data":"2457160b293cf657b512989aacfde8164178e9f9c6551dd347dc390e68604181"} Jul 06 20:08:33.361548 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:33.361519 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-477lt" event={"ID":"854b635c-f1e7-47d5-bf88-b5a80b572b1b","Type":"ContainerStarted","Data":"082307934e7f753da7e68dd32f5891e1f652508968735d830386ef6334d84dcb"} Jul 06 20:08:33.363168 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:33.363142 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-b6fnm" event={"ID":"953dca0f-091c-4aa1-90bc-3424c89859af","Type":"ContainerStarted","Data":"85daa3c17e2d2d5bf20007be22f6c577a34a002f7a737e87482e308362338ea1"} Jul 06 20:08:33.364582 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:33.364560 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mv9hq" event={"ID":"9cd8d42e-db03-477c-8ef9-bfed943eadb3","Type":"ContainerStarted","Data":"d1194b33ba0fd2c9979aa2be9172283d8d7819e53f2ca2f94ffb7b365a35e498"} Jul 06 20:08:33.365908 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:33.365873 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5b2wn" event={"ID":"1bbf9b1e-36f5-487b-a66e-55dc425fd1a0","Type":"ContainerStarted","Data":"a2eb658ac2b552f4b4f825e0b60985f039fa72877d6decd923b89d4ff8dde7ef"} Jul 06 20:08:33.367416 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:33.367394 2551 generic.go:356] "Generic (PLEG): container finished" podID="4c726ad1-d957-471d-95d9-fcaee1f227e8" containerID="22c98b9bd7126c9a105f3ac93f348309ad470028d5e6eca8a650ab977b2fc541" exitCode=0 Jul 06 20:08:33.367502 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:33.367442 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vgxs4" event={"ID":"4c726ad1-d957-471d-95d9-fcaee1f227e8","Type":"ContainerDied","Data":"22c98b9bd7126c9a105f3ac93f348309ad470028d5e6eca8a650ab977b2fc541"} Jul 06 20:08:33.368993 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:33.368970 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9cb25" event={"ID":"f3415158-a586-4fd7-bf83-bbd81db28b49","Type":"ContainerStarted","Data":"8b92c9ea56fb8ee5d8d84256da1cbfb417b74b12ae4dcaab00cdc2017549ca12"} Jul 06 20:08:33.372463 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:33.372439 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jsbkg" event={"ID":"e85c458d-f954-41bc-a06d-1b84854762ee","Type":"ContainerStarted","Data":"2ecc095d775db747824019c61bac640aa15fd633a004bcb448a1c8d4279c9dbc"} Jul 06 20:08:33.386578 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:33.386542 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-178.ec2.internal" podStartSLOduration=22.386530892 podStartE2EDuration="22.386530892s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:08:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:08:15.339743774 +0000 UTC m=+5.754943583" watchObservedRunningTime="2026-07-06 20:08:33.386530892 +0000 UTC m=+23.801730697" Jul 06 20:08:33.432276 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:33.432223 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-477lt" podStartSLOduration=4.276616943 podStartE2EDuration="23.432202909s" totalImagesPullingTime="19.155585966s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:08:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:08:33.387843731 +0000 UTC m=+23.803043537" watchObservedRunningTime="2026-07-06 20:08:33.432202909 +0000 UTC m=+23.847402711" Jul 06 20:08:33.471912 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:33.471829 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-mv9hq" podStartSLOduration=4.346199279 podStartE2EDuration="23.471803904s" totalImagesPullingTime="19.125604625s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:08:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:08:33.432355238 +0000 UTC m=+23.847555044" watchObservedRunningTime="2026-07-06 20:08:33.471803904 +0000 UTC m=+23.887003707" Jul 06 20:08:33.504251 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:33.504198 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-b6fnm" podStartSLOduration=4.3447716849999996 podStartE2EDuration="23.504180715s" totalImagesPullingTime="19.15940903s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:08:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:08:33.503879599 +0000 UTC m=+23.919079405" watchObservedRunningTime="2026-07-06 20:08:33.504180715 +0000 UTC m=+23.919380520" Jul 06 20:08:33.539933 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:33.539826 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-jsbkg" podStartSLOduration=4.371317617 podStartE2EDuration="23.539808364s" totalImagesPullingTime="19.168490747s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:08:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:08:33.539554494 +0000 UTC m=+23.954754302" watchObservedRunningTime="2026-07-06 20:08:33.539808364 +0000 UTC m=+23.955008184" Jul 06 20:08:33.565551 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:33.565492 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-5b2wn" podStartSLOduration=9.264449928 podStartE2EDuration="23.565473502s" totalImagesPullingTime="14.301023574s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:08:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:08:33.56497621 +0000 UTC m=+23.980176036" watchObservedRunningTime="2026-07-06 20:08:33.565473502 +0000 UTC m=+23.980673306" Jul 06 20:08:34.158809 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:34.158776 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6t4k" Jul 06 20:08:34.158991 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:34.158888 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6t4k" podUID="d6909872-c920-4e38-a17a-11ad51d50379" Jul 06 20:08:34.158991 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:34.158915 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6x889" Jul 06 20:08:34.159084 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:34.159002 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6x889" podUID="0899bbed-8629-4dd1-a6d0-be26f7870c7f" Jul 06 20:08:34.378196 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:34.378149 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" event={"ID":"645eb1ca-38b9-4599-80d5-cf4314f701c9","Type":"ContainerStarted","Data":"7c507b4895776b55a3ecccb35ece98b74d261b6c9ac49f1fd7303891b3659a39"} Jul 06 20:08:34.378660 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:34.378251 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" event={"ID":"645eb1ca-38b9-4599-80d5-cf4314f701c9","Type":"ContainerStarted","Data":"7a066097744edd83dce33a10a4080c55fcd9186caf03e57799ef0d60aac6a7a4"} Jul 06 20:08:34.379950 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:34.379920 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-55nk7" event={"ID":"b7b05003-ad5f-4f11-bc35-476e28673d12","Type":"ContainerStarted","Data":"6e85f47820965d2f530a613424997f40127c07c1adb47439cd0fb0c6623a069f"} Jul 06 20:08:34.507932 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:34.507885 2551 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 06 20:08:35.097915 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:35.097791 2551 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T20:08:34.50792669Z","UUID":"dcf545cb-c0cd-4bd3-a259-1d91ab8917ce","Handler":null,"Name":"","Endpoint":""} Jul 06 20:08:35.099671 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:35.099647 2551 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 06 20:08:35.099815 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:35.099692 2551 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 06 20:08:35.158356 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:35.158319 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v648r" Jul 06 20:08:35.158532 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:35.158454 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v648r" podUID="ef16cb88-f1e2-49c7-a470-47a23730758d" Jul 06 20:08:35.384222 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:35.384138 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9cb25" event={"ID":"f3415158-a586-4fd7-bf83-bbd81db28b49","Type":"ContainerStarted","Data":"7fe485e2a6b13b2f270cf76af98bcfcae5438ed82268731284d00256820085c9"} Jul 06 20:08:35.671852 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:35.671818 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-5b2wn" Jul 06 20:08:35.672376 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:35.672355 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-5b2wn" Jul 06 20:08:35.689528 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:35.689457 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-55nk7" podStartSLOduration=6.536446233 podStartE2EDuration="25.68944098s" totalImagesPullingTime="19.152994747s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:08:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:08:34.401850382 +0000 UTC m=+24.817050188" watchObservedRunningTime="2026-07-06 20:08:35.68944098 +0000 UTC m=+26.104640784" Jul 06 20:08:36.159150 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:36.158978 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6t4k" Jul 06 20:08:36.159319 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:36.159049 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6x889" Jul 06 20:08:36.159319 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:36.159230 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6t4k" podUID="d6909872-c920-4e38-a17a-11ad51d50379" Jul 06 20:08:36.159319 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:36.159295 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6x889" podUID="0899bbed-8629-4dd1-a6d0-be26f7870c7f" Jul 06 20:08:36.390715 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:36.390681 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" event={"ID":"645eb1ca-38b9-4599-80d5-cf4314f701c9","Type":"ContainerStarted","Data":"465ce0f990a1d62e0fe7e5eff2cd30c401683d0026605f54005d2e4c284e8d32"} Jul 06 20:08:36.393152 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:36.393117 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9cb25" event={"ID":"f3415158-a586-4fd7-bf83-bbd81db28b49","Type":"ContainerStarted","Data":"7f63d0bea05384201e901998d57089464a9bf1dc9477f59209445465563d4171"} Jul 06 20:08:36.393367 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:36.393343 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-5b2wn" Jul 06 20:08:36.393843 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:36.393825 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-5b2wn" Jul 06 20:08:36.431699 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:36.431597 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9cb25" podStartSLOduration=4.905509565 podStartE2EDuration="26.431574351s" totalImagesPullingTime="21.526064786s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:08:10 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:08:36.415651635 +0000 UTC m=+26.830851439" watchObservedRunningTime="2026-07-06 20:08:36.431574351 +0000 UTC m=+26.846774159" Jul 06 20:08:37.158714 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:37.158674 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v648r" Jul 06 20:08:37.158969 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:37.158806 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v648r" podUID="ef16cb88-f1e2-49c7-a470-47a23730758d" Jul 06 20:08:37.598461 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:37.598421 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5b2wn_1bbf9b1e-36f5-487b-a66e-55dc425fd1a0/konnectivity-agent/0.log" limit=1024 Jul 06 20:08:37.695316 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:37.695252 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5b2wn_1bbf9b1e-36f5-487b-a66e-55dc425fd1a0/konnectivity-agent/0.log" limit=1024 Jul 06 20:08:38.158871 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:38.158831 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6t4k" Jul 06 20:08:38.159057 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:38.158995 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6t4k" podUID="d6909872-c920-4e38-a17a-11ad51d50379" Jul 06 20:08:38.159057 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:38.159042 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6x889" Jul 06 20:08:38.159177 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:38.159153 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6x889" podUID="0899bbed-8629-4dd1-a6d0-be26f7870c7f" Jul 06 20:08:38.404656 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:38.404609 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" event={"ID":"645eb1ca-38b9-4599-80d5-cf4314f701c9","Type":"ContainerStarted","Data":"abc27685b2291124396c06c4439e724370c658a3c508c84610f9b64f0a7f0589"} Jul 06 20:08:39.158779 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:39.158575 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v648r" Jul 06 20:08:39.159595 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:39.158912 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v648r" podUID="ef16cb88-f1e2-49c7-a470-47a23730758d" Jul 06 20:08:39.407083 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:39.407055 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:39.407287 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:39.407144 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:39.407287 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:39.407188 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:39.426850 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:39.426754 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:39.427004 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:39.426936 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:08:39.461448 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:39.461390 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" podStartSLOduration=10.238776526 podStartE2EDuration="29.461365248s" totalImagesPullingTime="19.222588722s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:08:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:08:38.45206685 +0000 UTC m=+28.867266690" watchObservedRunningTime="2026-07-06 20:08:39.461365248 +0000 UTC m=+29.876565041" Jul 06 20:08:39.958467 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:39.958437 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-g6t4k"] Jul 06 20:08:39.958588 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:39.958575 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6t4k" Jul 06 20:08:39.958713 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:39.958692 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6t4k" podUID="d6909872-c920-4e38-a17a-11ad51d50379" Jul 06 20:08:39.961521 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:39.961449 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6x889"] Jul 06 20:08:39.961678 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:39.961580 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6x889" Jul 06 20:08:39.961756 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:39.961677 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6x889" podUID="0899bbed-8629-4dd1-a6d0-be26f7870c7f" Jul 06 20:08:39.962294 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:39.962268 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-v648r"] Jul 06 20:08:39.962379 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:39.962365 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v648r" Jul 06 20:08:39.962508 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:39.962473 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v648r" podUID="ef16cb88-f1e2-49c7-a470-47a23730758d" Jul 06 20:08:40.411005 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:40.410972 2551 generic.go:356] "Generic (PLEG): container finished" podID="4c726ad1-d957-471d-95d9-fcaee1f227e8" containerID="361d0d8a137ca014c08058a5d973cc8ff36a492438ecc24cef4623e8829abe97" exitCode=0 Jul 06 20:08:40.411411 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:40.411051 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vgxs4" event={"ID":"4c726ad1-d957-471d-95d9-fcaee1f227e8","Type":"ContainerDied","Data":"361d0d8a137ca014c08058a5d973cc8ff36a492438ecc24cef4623e8829abe97"} Jul 06 20:08:41.414820 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:41.414795 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vgxs4" event={"ID":"4c726ad1-d957-471d-95d9-fcaee1f227e8","Type":"ContainerStarted","Data":"4c1468c7e061d9ef173710bf700ea2b7b8291dcfb21cf0d79b644ca2a0d4ee27"} Jul 06 20:08:42.158067 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:42.157980 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6t4k" Jul 06 20:08:42.158067 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:42.157984 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6x889" Jul 06 20:08:42.158268 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:42.158093 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6t4k" podUID="d6909872-c920-4e38-a17a-11ad51d50379" Jul 06 20:08:42.158268 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:42.158103 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v648r" Jul 06 20:08:42.158268 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:42.158156 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6x889" podUID="0899bbed-8629-4dd1-a6d0-be26f7870c7f" Jul 06 20:08:42.158268 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:42.158228 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v648r" podUID="ef16cb88-f1e2-49c7-a470-47a23730758d" Jul 06 20:08:42.420087 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:42.420004 2551 generic.go:356] "Generic (PLEG): container finished" podID="4c726ad1-d957-471d-95d9-fcaee1f227e8" containerID="4c1468c7e061d9ef173710bf700ea2b7b8291dcfb21cf0d79b644ca2a0d4ee27" exitCode=0 Jul 06 20:08:42.420087 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:42.420047 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vgxs4" event={"ID":"4c726ad1-d957-471d-95d9-fcaee1f227e8","Type":"ContainerDied","Data":"4c1468c7e061d9ef173710bf700ea2b7b8291dcfb21cf0d79b644ca2a0d4ee27"} Jul 06 20:08:43.425385 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:43.425287 2551 generic.go:356] "Generic (PLEG): container finished" podID="4c726ad1-d957-471d-95d9-fcaee1f227e8" containerID="d05b7c450ee389da7ce0296e03e31f89c6b0e250e63ccec1ab2f24da5af8694f" exitCode=0 Jul 06 20:08:43.425385 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:43.425330 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vgxs4" event={"ID":"4c726ad1-d957-471d-95d9-fcaee1f227e8","Type":"ContainerDied","Data":"d05b7c450ee389da7ce0296e03e31f89c6b0e250e63ccec1ab2f24da5af8694f"} Jul 06 20:08:43.797152 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:43.797117 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6909872-c920-4e38-a17a-11ad51d50379-original-pull-secret\") pod \"global-pull-secret-syncer-g6t4k\" (UID: \"d6909872-c920-4e38-a17a-11ad51d50379\") " pod="kube-system/global-pull-secret-syncer-g6t4k" Jul 06 20:08:43.797323 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:43.797188 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef16cb88-f1e2-49c7-a470-47a23730758d-metrics-certs\") pod \"network-metrics-daemon-v648r\" (UID: \"ef16cb88-f1e2-49c7-a470-47a23730758d\") " pod="openshift-multus/network-metrics-daemon-v648r" Jul 06 20:08:43.797323 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:43.797270 2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 20:08:43.797393 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:43.797331 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6909872-c920-4e38-a17a-11ad51d50379-original-pull-secret podName:d6909872-c920-4e38-a17a-11ad51d50379 nodeName:}" failed. No retries permitted until 2026-07-06 20:09:15.797316084 +0000 UTC m=+66.212515870 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d6909872-c920-4e38-a17a-11ad51d50379-original-pull-secret") pod "global-pull-secret-syncer-g6t4k" (UID: "d6909872-c920-4e38-a17a-11ad51d50379") : object "kube-system"/"original-pull-secret" not registered Jul 06 20:08:43.797393 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:43.797270 2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 20:08:43.797465 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:43.797400 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef16cb88-f1e2-49c7-a470-47a23730758d-metrics-certs podName:ef16cb88-f1e2-49c7-a470-47a23730758d nodeName:}" failed. No retries permitted until 2026-07-06 20:09:15.797388628 +0000 UTC m=+66.212588414 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef16cb88-f1e2-49c7-a470-47a23730758d-metrics-certs") pod "network-metrics-daemon-v648r" (UID: "ef16cb88-f1e2-49c7-a470-47a23730758d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 20:08:43.999100 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:43.999069 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ds27q\" (UniqueName: \"kubernetes.io/projected/0899bbed-8629-4dd1-a6d0-be26f7870c7f-kube-api-access-ds27q\") pod \"network-check-target-6x889\" (UID: \"0899bbed-8629-4dd1-a6d0-be26f7870c7f\") " pod="openshift-network-diagnostics/network-check-target-6x889" Jul 06 20:08:43.999286 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:43.999202 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 20:08:43.999286 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:43.999218 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 20:08:43.999286 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:43.999227 2551 projected.go:196] Error preparing data for projected volume kube-api-access-ds27q for pod openshift-network-diagnostics/network-check-target-6x889: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 20:08:43.999286 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:43.999273 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0899bbed-8629-4dd1-a6d0-be26f7870c7f-kube-api-access-ds27q podName:0899bbed-8629-4dd1-a6d0-be26f7870c7f nodeName:}" failed. No retries permitted until 2026-07-06 20:09:15.999258792 +0000 UTC m=+66.414458574 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-ds27q" (UniqueName: "kubernetes.io/projected/0899bbed-8629-4dd1-a6d0-be26f7870c7f-kube-api-access-ds27q") pod "network-check-target-6x889" (UID: "0899bbed-8629-4dd1-a6d0-be26f7870c7f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 20:08:44.158453 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:44.158370 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6x889" Jul 06 20:08:44.158612 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:44.158369 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6t4k" Jul 06 20:08:44.158612 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:44.158484 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6x889" podUID="0899bbed-8629-4dd1-a6d0-be26f7870c7f" Jul 06 20:08:44.158612 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:44.158380 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v648r" Jul 06 20:08:44.158612 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:44.158579 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6t4k" podUID="d6909872-c920-4e38-a17a-11ad51d50379" Jul 06 20:08:44.158825 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:44.158645 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v648r" podUID="ef16cb88-f1e2-49c7-a470-47a23730758d" Jul 06 20:08:46.158646 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:46.158613 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6t4k" Jul 06 20:08:46.159193 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:46.158613 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v648r" Jul 06 20:08:46.159193 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:46.158726 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6t4k" podUID="d6909872-c920-4e38-a17a-11ad51d50379" Jul 06 20:08:46.159193 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:46.158752 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6x889" Jul 06 20:08:46.159193 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:46.158947 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v648r" podUID="ef16cb88-f1e2-49c7-a470-47a23730758d" Jul 06 20:08:46.159193 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:46.159071 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6x889" podUID="0899bbed-8629-4dd1-a6d0-be26f7870c7f" Jul 06 20:08:46.325025 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:46.324999 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-178.ec2.internal" event="NodeReady" Jul 06 20:08:46.325163 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:46.325120 2551 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 06 20:08:46.377821 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:46.377784 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-qwn7v"] Jul 06 20:08:46.392413 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:46.392368 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-z67pq"] Jul 06 20:08:46.392614 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:46.392507 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qwn7v" Jul 06 20:08:46.397240 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:46.397100 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 06 20:08:46.397383 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:46.397362 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-t7dkq\"" Jul 06 20:08:46.397564 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:46.397491 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 06 20:08:46.405954 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:46.405934 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qwn7v"] Jul 06 20:08:46.406053 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:46.405980 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-z67pq"] Jul 06 20:08:46.406053 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:46.405988 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-z67pq" Jul 06 20:08:46.408500 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:46.408465 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 06 20:08:46.408741 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:46.408699 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-5lqt6\"" Jul 06 20:08:46.408827 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:46.408796 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 06 20:08:46.409205 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:46.409187 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 06 20:08:46.516417 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:46.516378 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/148d8232-818e-46d9-b74b-94f6d03bf080-tmp-dir\") pod \"dns-default-qwn7v\" (UID: \"148d8232-818e-46d9-b74b-94f6d03bf080\") " pod="openshift-dns/dns-default-qwn7v" Jul 06 20:08:46.516613 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:46.516438 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pz4v\" (UniqueName: \"kubernetes.io/projected/148d8232-818e-46d9-b74b-94f6d03bf080-kube-api-access-4pz4v\") pod \"dns-default-qwn7v\" (UID: \"148d8232-818e-46d9-b74b-94f6d03bf080\") " pod="openshift-dns/dns-default-qwn7v" Jul 06 20:08:46.516613 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:46.516467 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/148d8232-818e-46d9-b74b-94f6d03bf080-config-volume\") pod \"dns-default-qwn7v\" (UID: \"148d8232-818e-46d9-b74b-94f6d03bf080\") " pod="openshift-dns/dns-default-qwn7v" Jul 06 20:08:46.516613 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:46.516496 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/148d8232-818e-46d9-b74b-94f6d03bf080-metrics-tls\") pod \"dns-default-qwn7v\" (UID: \"148d8232-818e-46d9-b74b-94f6d03bf080\") " pod="openshift-dns/dns-default-qwn7v" Jul 06 20:08:46.516613 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:46.516515 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cf4f10b3-8b8d-45c2-9f64-8a37aa591b68-cert\") pod \"ingress-canary-z67pq\" (UID: \"cf4f10b3-8b8d-45c2-9f64-8a37aa591b68\") " pod="openshift-ingress-canary/ingress-canary-z67pq" Jul 06 20:08:46.516613 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:46.516589 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5crh\" (UniqueName: \"kubernetes.io/projected/cf4f10b3-8b8d-45c2-9f64-8a37aa591b68-kube-api-access-p5crh\") pod \"ingress-canary-z67pq\" (UID: \"cf4f10b3-8b8d-45c2-9f64-8a37aa591b68\") " pod="openshift-ingress-canary/ingress-canary-z67pq" Jul 06 20:08:46.617310 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:46.617275 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/148d8232-818e-46d9-b74b-94f6d03bf080-tmp-dir\") pod \"dns-default-qwn7v\" (UID: \"148d8232-818e-46d9-b74b-94f6d03bf080\") " pod="openshift-dns/dns-default-qwn7v" Jul 06 20:08:46.617504 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:46.617335 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pz4v\" (UniqueName: \"kubernetes.io/projected/148d8232-818e-46d9-b74b-94f6d03bf080-kube-api-access-4pz4v\") pod \"dns-default-qwn7v\" (UID: \"148d8232-818e-46d9-b74b-94f6d03bf080\") " pod="openshift-dns/dns-default-qwn7v" Jul 06 20:08:46.617504 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:46.617363 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/148d8232-818e-46d9-b74b-94f6d03bf080-config-volume\") pod \"dns-default-qwn7v\" (UID: \"148d8232-818e-46d9-b74b-94f6d03bf080\") " pod="openshift-dns/dns-default-qwn7v" Jul 06 20:08:46.617504 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:46.617388 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/148d8232-818e-46d9-b74b-94f6d03bf080-metrics-tls\") pod \"dns-default-qwn7v\" (UID: \"148d8232-818e-46d9-b74b-94f6d03bf080\") " pod="openshift-dns/dns-default-qwn7v" Jul 06 20:08:46.617504 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:46.617406 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cf4f10b3-8b8d-45c2-9f64-8a37aa591b68-cert\") pod \"ingress-canary-z67pq\" (UID: \"cf4f10b3-8b8d-45c2-9f64-8a37aa591b68\") " pod="openshift-ingress-canary/ingress-canary-z67pq" Jul 06 20:08:46.617504 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:46.617446 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5crh\" (UniqueName: \"kubernetes.io/projected/cf4f10b3-8b8d-45c2-9f64-8a37aa591b68-kube-api-access-p5crh\") pod \"ingress-canary-z67pq\" (UID: \"cf4f10b3-8b8d-45c2-9f64-8a37aa591b68\") " pod="openshift-ingress-canary/ingress-canary-z67pq" Jul 06 20:08:46.617760 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:46.617569 2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 20:08:46.617760 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:46.617631 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/148d8232-818e-46d9-b74b-94f6d03bf080-metrics-tls podName:148d8232-818e-46d9-b74b-94f6d03bf080 nodeName:}" failed. No retries permitted until 2026-07-06 20:08:47.117612001 +0000 UTC m=+37.532811788 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/148d8232-818e-46d9-b74b-94f6d03bf080-metrics-tls") pod "dns-default-qwn7v" (UID: "148d8232-818e-46d9-b74b-94f6d03bf080") : secret "dns-default-metrics-tls" not found Jul 06 20:08:46.617760 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:46.617716 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/148d8232-818e-46d9-b74b-94f6d03bf080-tmp-dir\") pod \"dns-default-qwn7v\" (UID: \"148d8232-818e-46d9-b74b-94f6d03bf080\") " pod="openshift-dns/dns-default-qwn7v" Jul 06 20:08:46.618203 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:46.618178 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/148d8232-818e-46d9-b74b-94f6d03bf080-config-volume\") pod \"dns-default-qwn7v\" (UID: \"148d8232-818e-46d9-b74b-94f6d03bf080\") " pod="openshift-dns/dns-default-qwn7v" Jul 06 20:08:46.622490 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:46.622462 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cf4f10b3-8b8d-45c2-9f64-8a37aa591b68-cert\") pod \"ingress-canary-z67pq\" (UID: \"cf4f10b3-8b8d-45c2-9f64-8a37aa591b68\") " pod="openshift-ingress-canary/ingress-canary-z67pq" Jul 06 20:08:46.628270 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:46.628246 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pz4v\" (UniqueName: \"kubernetes.io/projected/148d8232-818e-46d9-b74b-94f6d03bf080-kube-api-access-4pz4v\") pod \"dns-default-qwn7v\" (UID: \"148d8232-818e-46d9-b74b-94f6d03bf080\") " pod="openshift-dns/dns-default-qwn7v" Jul 06 20:08:46.628732 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:46.628715 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5crh\" (UniqueName: \"kubernetes.io/projected/cf4f10b3-8b8d-45c2-9f64-8a37aa591b68-kube-api-access-p5crh\") pod \"ingress-canary-z67pq\" (UID: \"cf4f10b3-8b8d-45c2-9f64-8a37aa591b68\") " pod="openshift-ingress-canary/ingress-canary-z67pq" Jul 06 20:08:46.715806 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:46.715710 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-z67pq" Jul 06 20:08:46.871871 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:46.871630 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-z67pq"] Jul 06 20:08:46.876602 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:08:46.876577 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf4f10b3_8b8d_45c2_9f64_8a37aa591b68.slice/crio-d0d0788e685099d36e3cefe4ce15da0643678783b98b64c1427d65c07c2fc513 WatchSource:0}: Error finding container d0d0788e685099d36e3cefe4ce15da0643678783b98b64c1427d65c07c2fc513: Status 404 returned error can't find the container with id d0d0788e685099d36e3cefe4ce15da0643678783b98b64c1427d65c07c2fc513 Jul 06 20:08:47.121423 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:47.121392 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/148d8232-818e-46d9-b74b-94f6d03bf080-metrics-tls\") pod \"dns-default-qwn7v\" (UID: \"148d8232-818e-46d9-b74b-94f6d03bf080\") " pod="openshift-dns/dns-default-qwn7v" Jul 06 20:08:47.121602 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:47.121520 2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 20:08:47.121602 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:47.121592 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/148d8232-818e-46d9-b74b-94f6d03bf080-metrics-tls podName:148d8232-818e-46d9-b74b-94f6d03bf080 nodeName:}" failed. No retries permitted until 2026-07-06 20:08:48.121575881 +0000 UTC m=+38.536775666 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/148d8232-818e-46d9-b74b-94f6d03bf080-metrics-tls") pod "dns-default-qwn7v" (UID: "148d8232-818e-46d9-b74b-94f6d03bf080") : secret "dns-default-metrics-tls" not found Jul 06 20:08:47.438057 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:47.437909 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-z67pq" event={"ID":"cf4f10b3-8b8d-45c2-9f64-8a37aa591b68","Type":"ContainerStarted","Data":"d0d0788e685099d36e3cefe4ce15da0643678783b98b64c1427d65c07c2fc513"} Jul 06 20:08:48.128557 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:48.128516 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/148d8232-818e-46d9-b74b-94f6d03bf080-metrics-tls\") pod \"dns-default-qwn7v\" (UID: \"148d8232-818e-46d9-b74b-94f6d03bf080\") " pod="openshift-dns/dns-default-qwn7v" Jul 06 20:08:48.128771 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:48.128709 2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 20:08:48.128843 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:48.128784 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/148d8232-818e-46d9-b74b-94f6d03bf080-metrics-tls podName:148d8232-818e-46d9-b74b-94f6d03bf080 nodeName:}" failed. No retries permitted until 2026-07-06 20:08:50.128764521 +0000 UTC m=+40.543964319 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/148d8232-818e-46d9-b74b-94f6d03bf080-metrics-tls") pod "dns-default-qwn7v" (UID: "148d8232-818e-46d9-b74b-94f6d03bf080") : secret "dns-default-metrics-tls" not found Jul 06 20:08:48.158941 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:48.158902 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6x889" Jul 06 20:08:48.159261 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:48.158907 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v648r" Jul 06 20:08:48.159601 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:48.158906 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6t4k" Jul 06 20:08:48.163140 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:48.163117 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 20:08:48.163312 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:48.163291 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 20:08:48.163404 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:48.163387 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7lmbq\"" Jul 06 20:08:48.163517 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:48.163499 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 20:08:48.163671 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:48.163650 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-rczn2\"" Jul 06 20:08:48.163775 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:48.163743 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 20:08:50.145773 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:50.145727 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/148d8232-818e-46d9-b74b-94f6d03bf080-metrics-tls\") pod \"dns-default-qwn7v\" (UID: \"148d8232-818e-46d9-b74b-94f6d03bf080\") " pod="openshift-dns/dns-default-qwn7v" Jul 06 20:08:50.146176 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:50.145838 2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 20:08:50.146176 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:50.145905 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/148d8232-818e-46d9-b74b-94f6d03bf080-metrics-tls podName:148d8232-818e-46d9-b74b-94f6d03bf080 nodeName:}" failed. No retries permitted until 2026-07-06 20:08:54.145874191 +0000 UTC m=+44.561073989 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/148d8232-818e-46d9-b74b-94f6d03bf080-metrics-tls") pod "dns-default-qwn7v" (UID: "148d8232-818e-46d9-b74b-94f6d03bf080") : secret "dns-default-metrics-tls" not found Jul 06 20:08:50.448362 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:50.448123 2551 generic.go:356] "Generic (PLEG): container finished" podID="4c726ad1-d957-471d-95d9-fcaee1f227e8" containerID="23a8d2a459b6175fef7963af836e077817d5fbd06538b7d84e2bc3258967ab66" exitCode=0 Jul 06 20:08:50.448362 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:50.448202 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vgxs4" event={"ID":"4c726ad1-d957-471d-95d9-fcaee1f227e8","Type":"ContainerDied","Data":"23a8d2a459b6175fef7963af836e077817d5fbd06538b7d84e2bc3258967ab66"} Jul 06 20:08:50.449882 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:50.449853 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-z67pq" event={"ID":"cf4f10b3-8b8d-45c2-9f64-8a37aa591b68","Type":"ContainerStarted","Data":"a318bb9ce1667c4019b040b1fae7cf3f88735bf4c267fb7b140ffe368b3d1646"} Jul 06 20:08:50.492723 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:50.492679 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-z67pq" podStartSLOduration=1.4227431689999999 podStartE2EDuration="4.492663193s" totalImagesPullingTime="3.069920024s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:08:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:08:50.492098286 +0000 UTC m=+40.907298092" watchObservedRunningTime="2026-07-06 20:08:50.492663193 +0000 UTC m=+40.907862998" Jul 06 20:08:51.455493 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:51.455462 2551 generic.go:356] "Generic (PLEG): container finished" podID="4c726ad1-d957-471d-95d9-fcaee1f227e8" containerID="10043689c428db6962f94aed036303fe980865ba49ba0012e4a6460d34a84a63" exitCode=0 Jul 06 20:08:51.455937 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:51.455539 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vgxs4" event={"ID":"4c726ad1-d957-471d-95d9-fcaee1f227e8","Type":"ContainerDied","Data":"10043689c428db6962f94aed036303fe980865ba49ba0012e4a6460d34a84a63"} Jul 06 20:08:52.460778 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:52.460739 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vgxs4" event={"ID":"4c726ad1-d957-471d-95d9-fcaee1f227e8","Type":"ContainerStarted","Data":"5936811686aa3ce52ebbf15a0dba14c7c055d8832dee2823329ba5da53d8709e"} Jul 06 20:08:52.486383 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:52.486335 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-vgxs4" podStartSLOduration=8.813749652 podStartE2EDuration="42.486318537s" totalImagesPullingTime="33.672568885s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:08:10 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:08:52.485375418 +0000 UTC m=+42.900575224" watchObservedRunningTime="2026-07-06 20:08:52.486318537 +0000 UTC m=+42.901518336" Jul 06 20:08:54.171473 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:08:54.171436 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/148d8232-818e-46d9-b74b-94f6d03bf080-metrics-tls\") pod \"dns-default-qwn7v\" (UID: \"148d8232-818e-46d9-b74b-94f6d03bf080\") " pod="openshift-dns/dns-default-qwn7v" Jul 06 20:08:54.171832 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:54.171577 2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 20:08:54.171832 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:08:54.171636 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/148d8232-818e-46d9-b74b-94f6d03bf080-metrics-tls podName:148d8232-818e-46d9-b74b-94f6d03bf080 nodeName:}" failed. No retries permitted until 2026-07-06 20:09:02.17161953 +0000 UTC m=+52.586819313 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/148d8232-818e-46d9-b74b-94f6d03bf080-metrics-tls") pod "dns-default-qwn7v" (UID: "148d8232-818e-46d9-b74b-94f6d03bf080") : secret "dns-default-metrics-tls" not found Jul 06 20:09:02.222477 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:02.222441 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/148d8232-818e-46d9-b74b-94f6d03bf080-metrics-tls\") pod \"dns-default-qwn7v\" (UID: \"148d8232-818e-46d9-b74b-94f6d03bf080\") " pod="openshift-dns/dns-default-qwn7v" Jul 06 20:09:02.222876 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:02.222595 2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 20:09:02.222876 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:02.222656 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/148d8232-818e-46d9-b74b-94f6d03bf080-metrics-tls podName:148d8232-818e-46d9-b74b-94f6d03bf080 nodeName:}" failed. No retries permitted until 2026-07-06 20:09:18.222639735 +0000 UTC m=+68.637839519 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/148d8232-818e-46d9-b74b-94f6d03bf080-metrics-tls") pod "dns-default-qwn7v" (UID: "148d8232-818e-46d9-b74b-94f6d03bf080") : secret "dns-default-metrics-tls" not found Jul 06 20:09:04.390582 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.390547 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-zszj5"] Jul 06 20:09:04.414710 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.414681 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-jxbvp"] Jul 06 20:09:04.414851 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.414799 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-zszj5" Jul 06 20:09:04.417751 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.417731 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 06 20:09:04.418116 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.418078 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 06 20:09:04.418234 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.418115 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-jc7zf\"" Jul 06 20:09:04.418982 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.418963 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 06 20:09:04.440752 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.440728 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-jxbvp"] Jul 06 20:09:04.440752 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.440759 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-zszj5"] Jul 06 20:09:04.440948 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.440846 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-jxbvp" Jul 06 20:09:04.443743 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.443721 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 06 20:09:04.444000 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.443849 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 06 20:09:04.444120 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.444049 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-5l4vb\"" Jul 06 20:09:04.444120 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.444085 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 06 20:09:04.444120 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.444096 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 06 20:09:04.448728 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.448711 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 06 20:09:04.482850 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.482823 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-847qg"] Jul 06 20:09:04.513570 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.513544 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-668d9f84f5-rrtln"] Jul 06 20:09:04.513704 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.513616 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-847qg" Jul 06 20:09:04.516416 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.516397 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 06 20:09:04.516416 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.516413 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 06 20:09:04.516593 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.516413 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-gsz5s\"" Jul 06 20:09:04.540278 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.540256 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e54c25e9-3542-48e1-9c6c-de6bd9b3f3d7-tmp\") pod \"insights-operator-5c847dcfc4-jxbvp\" (UID: \"e54c25e9-3542-48e1-9c6c-de6bd9b3f3d7\") " pod="openshift-insights/insights-operator-5c847dcfc4-jxbvp" Jul 06 20:09:04.540417 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.540283 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e54c25e9-3542-48e1-9c6c-de6bd9b3f3d7-serving-cert\") pod \"insights-operator-5c847dcfc4-jxbvp\" (UID: \"e54c25e9-3542-48e1-9c6c-de6bd9b3f3d7\") " pod="openshift-insights/insights-operator-5c847dcfc4-jxbvp" Jul 06 20:09:04.540417 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.540299 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsj9l\" (UniqueName: \"kubernetes.io/projected/e54c25e9-3542-48e1-9c6c-de6bd9b3f3d7-kube-api-access-gsj9l\") pod \"insights-operator-5c847dcfc4-jxbvp\" (UID: \"e54c25e9-3542-48e1-9c6c-de6bd9b3f3d7\") " pod="openshift-insights/insights-operator-5c847dcfc4-jxbvp" Jul 06 20:09:04.540417 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.540321 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e54c25e9-3542-48e1-9c6c-de6bd9b3f3d7-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-jxbvp\" (UID: \"e54c25e9-3542-48e1-9c6c-de6bd9b3f3d7\") " pod="openshift-insights/insights-operator-5c847dcfc4-jxbvp" Jul 06 20:09:04.540417 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.540345 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e54c25e9-3542-48e1-9c6c-de6bd9b3f3d7-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-jxbvp\" (UID: \"e54c25e9-3542-48e1-9c6c-de6bd9b3f3d7\") " pod="openshift-insights/insights-operator-5c847dcfc4-jxbvp" Jul 06 20:09:04.540624 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.540453 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28hhb\" (UniqueName: \"kubernetes.io/projected/33f76cfa-c0b5-42e8-931f-572dff9fd39e-kube-api-access-28hhb\") pod \"cluster-samples-operator-56fcf86bf-zszj5\" (UID: \"33f76cfa-c0b5-42e8-931f-572dff9fd39e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-zszj5" Jul 06 20:09:04.540624 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.540547 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/33f76cfa-c0b5-42e8-931f-572dff9fd39e-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-zszj5\" (UID: \"33f76cfa-c0b5-42e8-931f-572dff9fd39e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-zszj5" Jul 06 20:09:04.540624 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.540581 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/e54c25e9-3542-48e1-9c6c-de6bd9b3f3d7-snapshots\") pod \"insights-operator-5c847dcfc4-jxbvp\" (UID: \"e54c25e9-3542-48e1-9c6c-de6bd9b3f3d7\") " pod="openshift-insights/insights-operator-5c847dcfc4-jxbvp" Jul 06 20:09:04.540757 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.540742 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-847qg"] Jul 06 20:09:04.540791 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.540767 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-668d9f84f5-rrtln"] Jul 06 20:09:04.540851 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.540841 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-668d9f84f5-rrtln" Jul 06 20:09:04.545121 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.545094 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 06 20:09:04.545276 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.545256 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 06 20:09:04.545453 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.545437 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 06 20:09:04.545510 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.545474 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-b26bd\"" Jul 06 20:09:04.545562 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.545545 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 06 20:09:04.545615 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.545564 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 06 20:09:04.545615 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.545565 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 06 20:09:04.641152 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.641071 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/735538e5-306c-44a6-a934-30f463187f08-metrics-certs\") pod \"router-default-668d9f84f5-rrtln\" (UID: \"735538e5-306c-44a6-a934-30f463187f08\") " pod="openshift-ingress/router-default-668d9f84f5-rrtln" Jul 06 20:09:04.641152 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.641110 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/e54c25e9-3542-48e1-9c6c-de6bd9b3f3d7-snapshots\") pod \"insights-operator-5c847dcfc4-jxbvp\" (UID: \"e54c25e9-3542-48e1-9c6c-de6bd9b3f3d7\") " pod="openshift-insights/insights-operator-5c847dcfc4-jxbvp" Jul 06 20:09:04.641152 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.641140 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/735538e5-306c-44a6-a934-30f463187f08-service-ca-bundle\") pod \"router-default-668d9f84f5-rrtln\" (UID: \"735538e5-306c-44a6-a934-30f463187f08\") " pod="openshift-ingress/router-default-668d9f84f5-rrtln" Jul 06 20:09:04.641349 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.641164 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmcv7\" (UniqueName: \"kubernetes.io/projected/735538e5-306c-44a6-a934-30f463187f08-kube-api-access-mmcv7\") pod \"router-default-668d9f84f5-rrtln\" (UID: \"735538e5-306c-44a6-a934-30f463187f08\") " pod="openshift-ingress/router-default-668d9f84f5-rrtln" Jul 06 20:09:04.641349 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.641223 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlvgk\" (UniqueName: \"kubernetes.io/projected/8c2d1989-b3d7-48a4-8417-23e639cbc855-kube-api-access-xlvgk\") pod \"volume-data-source-validator-5d456fdbcd-847qg\" (UID: \"8c2d1989-b3d7-48a4-8417-23e639cbc855\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-847qg" Jul 06 20:09:04.641349 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.641263 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e54c25e9-3542-48e1-9c6c-de6bd9b3f3d7-tmp\") pod \"insights-operator-5c847dcfc4-jxbvp\" (UID: \"e54c25e9-3542-48e1-9c6c-de6bd9b3f3d7\") " pod="openshift-insights/insights-operator-5c847dcfc4-jxbvp" Jul 06 20:09:04.641349 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.641282 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e54c25e9-3542-48e1-9c6c-de6bd9b3f3d7-serving-cert\") pod \"insights-operator-5c847dcfc4-jxbvp\" (UID: \"e54c25e9-3542-48e1-9c6c-de6bd9b3f3d7\") " pod="openshift-insights/insights-operator-5c847dcfc4-jxbvp" Jul 06 20:09:04.641349 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.641301 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsj9l\" (UniqueName: \"kubernetes.io/projected/e54c25e9-3542-48e1-9c6c-de6bd9b3f3d7-kube-api-access-gsj9l\") pod \"insights-operator-5c847dcfc4-jxbvp\" (UID: \"e54c25e9-3542-48e1-9c6c-de6bd9b3f3d7\") " pod="openshift-insights/insights-operator-5c847dcfc4-jxbvp" Jul 06 20:09:04.641349 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.641320 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e54c25e9-3542-48e1-9c6c-de6bd9b3f3d7-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-jxbvp\" (UID: \"e54c25e9-3542-48e1-9c6c-de6bd9b3f3d7\") " pod="openshift-insights/insights-operator-5c847dcfc4-jxbvp" Jul 06 20:09:04.641349 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.641335 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e54c25e9-3542-48e1-9c6c-de6bd9b3f3d7-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-jxbvp\" (UID: \"e54c25e9-3542-48e1-9c6c-de6bd9b3f3d7\") " pod="openshift-insights/insights-operator-5c847dcfc4-jxbvp" Jul 06 20:09:04.641665 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.641352 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/735538e5-306c-44a6-a934-30f463187f08-default-certificate\") pod \"router-default-668d9f84f5-rrtln\" (UID: \"735538e5-306c-44a6-a934-30f463187f08\") " pod="openshift-ingress/router-default-668d9f84f5-rrtln" Jul 06 20:09:04.641665 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.641376 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/735538e5-306c-44a6-a934-30f463187f08-stats-auth\") pod \"router-default-668d9f84f5-rrtln\" (UID: \"735538e5-306c-44a6-a934-30f463187f08\") " pod="openshift-ingress/router-default-668d9f84f5-rrtln" Jul 06 20:09:04.641665 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.641427 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28hhb\" (UniqueName: \"kubernetes.io/projected/33f76cfa-c0b5-42e8-931f-572dff9fd39e-kube-api-access-28hhb\") pod \"cluster-samples-operator-56fcf86bf-zszj5\" (UID: \"33f76cfa-c0b5-42e8-931f-572dff9fd39e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-zszj5" Jul 06 20:09:04.641665 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.641517 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/33f76cfa-c0b5-42e8-931f-572dff9fd39e-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-zszj5\" (UID: \"33f76cfa-c0b5-42e8-931f-572dff9fd39e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-zszj5" Jul 06 20:09:04.641665 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:04.641612 2551 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 20:09:04.641937 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:04.641682 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33f76cfa-c0b5-42e8-931f-572dff9fd39e-samples-operator-tls podName:33f76cfa-c0b5-42e8-931f-572dff9fd39e nodeName:}" failed. No retries permitted until 2026-07-06 20:09:05.141662886 +0000 UTC m=+55.556862675 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/33f76cfa-c0b5-42e8-931f-572dff9fd39e-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-zszj5" (UID: "33f76cfa-c0b5-42e8-931f-572dff9fd39e") : secret "samples-operator-tls" not found Jul 06 20:09:04.641937 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.641873 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/e54c25e9-3542-48e1-9c6c-de6bd9b3f3d7-snapshots\") pod \"insights-operator-5c847dcfc4-jxbvp\" (UID: \"e54c25e9-3542-48e1-9c6c-de6bd9b3f3d7\") " pod="openshift-insights/insights-operator-5c847dcfc4-jxbvp" Jul 06 20:09:04.642062 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.642019 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e54c25e9-3542-48e1-9c6c-de6bd9b3f3d7-tmp\") pod \"insights-operator-5c847dcfc4-jxbvp\" (UID: \"e54c25e9-3542-48e1-9c6c-de6bd9b3f3d7\") " pod="openshift-insights/insights-operator-5c847dcfc4-jxbvp" Jul 06 20:09:04.642191 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.642172 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e54c25e9-3542-48e1-9c6c-de6bd9b3f3d7-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-jxbvp\" (UID: \"e54c25e9-3542-48e1-9c6c-de6bd9b3f3d7\") " pod="openshift-insights/insights-operator-5c847dcfc4-jxbvp" Jul 06 20:09:04.642522 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.642506 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e54c25e9-3542-48e1-9c6c-de6bd9b3f3d7-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-jxbvp\" (UID: \"e54c25e9-3542-48e1-9c6c-de6bd9b3f3d7\") " pod="openshift-insights/insights-operator-5c847dcfc4-jxbvp" Jul 06 20:09:04.645292 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.645275 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e54c25e9-3542-48e1-9c6c-de6bd9b3f3d7-serving-cert\") pod \"insights-operator-5c847dcfc4-jxbvp\" (UID: \"e54c25e9-3542-48e1-9c6c-de6bd9b3f3d7\") " pod="openshift-insights/insights-operator-5c847dcfc4-jxbvp" Jul 06 20:09:04.651517 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.651493 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-28hhb\" (UniqueName: \"kubernetes.io/projected/33f76cfa-c0b5-42e8-931f-572dff9fd39e-kube-api-access-28hhb\") pod \"cluster-samples-operator-56fcf86bf-zszj5\" (UID: \"33f76cfa-c0b5-42e8-931f-572dff9fd39e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-zszj5" Jul 06 20:09:04.653453 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.653429 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsj9l\" (UniqueName: \"kubernetes.io/projected/e54c25e9-3542-48e1-9c6c-de6bd9b3f3d7-kube-api-access-gsj9l\") pod \"insights-operator-5c847dcfc4-jxbvp\" (UID: \"e54c25e9-3542-48e1-9c6c-de6bd9b3f3d7\") " pod="openshift-insights/insights-operator-5c847dcfc4-jxbvp" Jul 06 20:09:04.742498 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.742462 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/735538e5-306c-44a6-a934-30f463187f08-default-certificate\") pod \"router-default-668d9f84f5-rrtln\" (UID: \"735538e5-306c-44a6-a934-30f463187f08\") " pod="openshift-ingress/router-default-668d9f84f5-rrtln" Jul 06 20:09:04.742498 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.742495 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/735538e5-306c-44a6-a934-30f463187f08-stats-auth\") pod \"router-default-668d9f84f5-rrtln\" (UID: \"735538e5-306c-44a6-a934-30f463187f08\") " pod="openshift-ingress/router-default-668d9f84f5-rrtln" Jul 06 20:09:04.742680 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.742564 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/735538e5-306c-44a6-a934-30f463187f08-metrics-certs\") pod \"router-default-668d9f84f5-rrtln\" (UID: \"735538e5-306c-44a6-a934-30f463187f08\") " pod="openshift-ingress/router-default-668d9f84f5-rrtln" Jul 06 20:09:04.742680 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.742590 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/735538e5-306c-44a6-a934-30f463187f08-service-ca-bundle\") pod \"router-default-668d9f84f5-rrtln\" (UID: \"735538e5-306c-44a6-a934-30f463187f08\") " pod="openshift-ingress/router-default-668d9f84f5-rrtln" Jul 06 20:09:04.742680 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.742606 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmcv7\" (UniqueName: \"kubernetes.io/projected/735538e5-306c-44a6-a934-30f463187f08-kube-api-access-mmcv7\") pod \"router-default-668d9f84f5-rrtln\" (UID: \"735538e5-306c-44a6-a934-30f463187f08\") " pod="openshift-ingress/router-default-668d9f84f5-rrtln" Jul 06 20:09:04.742680 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.742640 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlvgk\" (UniqueName: \"kubernetes.io/projected/8c2d1989-b3d7-48a4-8417-23e639cbc855-kube-api-access-xlvgk\") pod \"volume-data-source-validator-5d456fdbcd-847qg\" (UID: \"8c2d1989-b3d7-48a4-8417-23e639cbc855\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-847qg" Jul 06 20:09:04.742873 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:04.742674 2551 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 20:09:04.742873 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:04.742742 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/735538e5-306c-44a6-a934-30f463187f08-metrics-certs podName:735538e5-306c-44a6-a934-30f463187f08 nodeName:}" failed. No retries permitted until 2026-07-06 20:09:05.242721092 +0000 UTC m=+55.657920876 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/735538e5-306c-44a6-a934-30f463187f08-metrics-certs") pod "router-default-668d9f84f5-rrtln" (UID: "735538e5-306c-44a6-a934-30f463187f08") : secret "router-metrics-certs-default" not found Jul 06 20:09:04.742873 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:04.742762 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/735538e5-306c-44a6-a934-30f463187f08-service-ca-bundle podName:735538e5-306c-44a6-a934-30f463187f08 nodeName:}" failed. No retries permitted until 2026-07-06 20:09:05.242752779 +0000 UTC m=+55.657952565 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/735538e5-306c-44a6-a934-30f463187f08-service-ca-bundle") pod "router-default-668d9f84f5-rrtln" (UID: "735538e5-306c-44a6-a934-30f463187f08") : configmap references non-existent config key: service-ca.crt Jul 06 20:09:04.744820 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.744792 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/735538e5-306c-44a6-a934-30f463187f08-default-certificate\") pod \"router-default-668d9f84f5-rrtln\" (UID: \"735538e5-306c-44a6-a934-30f463187f08\") " pod="openshift-ingress/router-default-668d9f84f5-rrtln" Jul 06 20:09:04.744914 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.744809 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/735538e5-306c-44a6-a934-30f463187f08-stats-auth\") pod \"router-default-668d9f84f5-rrtln\" (UID: \"735538e5-306c-44a6-a934-30f463187f08\") " pod="openshift-ingress/router-default-668d9f84f5-rrtln" Jul 06 20:09:04.749648 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.749627 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-jxbvp" Jul 06 20:09:04.751720 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.751695 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmcv7\" (UniqueName: \"kubernetes.io/projected/735538e5-306c-44a6-a934-30f463187f08-kube-api-access-mmcv7\") pod \"router-default-668d9f84f5-rrtln\" (UID: \"735538e5-306c-44a6-a934-30f463187f08\") " pod="openshift-ingress/router-default-668d9f84f5-rrtln" Jul 06 20:09:04.751852 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.751833 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlvgk\" (UniqueName: \"kubernetes.io/projected/8c2d1989-b3d7-48a4-8417-23e639cbc855-kube-api-access-xlvgk\") pod \"volume-data-source-validator-5d456fdbcd-847qg\" (UID: \"8c2d1989-b3d7-48a4-8417-23e639cbc855\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-847qg" Jul 06 20:09:04.824264 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.824004 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-847qg" Jul 06 20:09:04.905132 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.905050 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-jxbvp"] Jul 06 20:09:04.907529 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:09:04.907500 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode54c25e9_3542_48e1_9c6c_de6bd9b3f3d7.slice/crio-1b075ef0155a7498c6b7ac52fa923fcf098364141cbd0ceed977b16d28e073a1 WatchSource:0}: Error finding container 1b075ef0155a7498c6b7ac52fa923fcf098364141cbd0ceed977b16d28e073a1: Status 404 returned error can't find the container with id 1b075ef0155a7498c6b7ac52fa923fcf098364141cbd0ceed977b16d28e073a1 Jul 06 20:09:04.959747 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:04.959713 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-847qg"] Jul 06 20:09:04.962169 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:09:04.962140 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c2d1989_b3d7_48a4_8417_23e639cbc855.slice/crio-17b0fe0548250db64605c3a02127fef73a3abdf5e479f459eda38ac20dede8a0 WatchSource:0}: Error finding container 17b0fe0548250db64605c3a02127fef73a3abdf5e479f459eda38ac20dede8a0: Status 404 returned error can't find the container with id 17b0fe0548250db64605c3a02127fef73a3abdf5e479f459eda38ac20dede8a0 Jul 06 20:09:05.147185 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:05.147151 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/33f76cfa-c0b5-42e8-931f-572dff9fd39e-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-zszj5\" (UID: \"33f76cfa-c0b5-42e8-931f-572dff9fd39e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-zszj5" Jul 06 20:09:05.147359 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:05.147268 2551 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 20:09:05.147359 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:05.147321 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33f76cfa-c0b5-42e8-931f-572dff9fd39e-samples-operator-tls podName:33f76cfa-c0b5-42e8-931f-572dff9fd39e nodeName:}" failed. No retries permitted until 2026-07-06 20:09:06.147307747 +0000 UTC m=+56.562507530 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/33f76cfa-c0b5-42e8-931f-572dff9fd39e-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-zszj5" (UID: "33f76cfa-c0b5-42e8-931f-572dff9fd39e") : secret "samples-operator-tls" not found Jul 06 20:09:05.249280 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:05.249194 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/735538e5-306c-44a6-a934-30f463187f08-metrics-certs\") pod \"router-default-668d9f84f5-rrtln\" (UID: \"735538e5-306c-44a6-a934-30f463187f08\") " pod="openshift-ingress/router-default-668d9f84f5-rrtln" Jul 06 20:09:05.249280 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:05.249243 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/735538e5-306c-44a6-a934-30f463187f08-service-ca-bundle\") pod \"router-default-668d9f84f5-rrtln\" (UID: \"735538e5-306c-44a6-a934-30f463187f08\") " pod="openshift-ingress/router-default-668d9f84f5-rrtln" Jul 06 20:09:05.249463 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:05.249344 2551 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 20:09:05.249463 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:05.249371 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/735538e5-306c-44a6-a934-30f463187f08-service-ca-bundle podName:735538e5-306c-44a6-a934-30f463187f08 nodeName:}" failed. No retries permitted until 2026-07-06 20:09:06.249355652 +0000 UTC m=+56.664555442 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/735538e5-306c-44a6-a934-30f463187f08-service-ca-bundle") pod "router-default-668d9f84f5-rrtln" (UID: "735538e5-306c-44a6-a934-30f463187f08") : configmap references non-existent config key: service-ca.crt Jul 06 20:09:05.249463 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:05.249406 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/735538e5-306c-44a6-a934-30f463187f08-metrics-certs podName:735538e5-306c-44a6-a934-30f463187f08 nodeName:}" failed. No retries permitted until 2026-07-06 20:09:06.249392866 +0000 UTC m=+56.664592648 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/735538e5-306c-44a6-a934-30f463187f08-metrics-certs") pod "router-default-668d9f84f5-rrtln" (UID: "735538e5-306c-44a6-a934-30f463187f08") : secret "router-metrics-certs-default" not found Jul 06 20:09:05.488767 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:05.488726 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-847qg" event={"ID":"8c2d1989-b3d7-48a4-8417-23e639cbc855","Type":"ContainerStarted","Data":"17b0fe0548250db64605c3a02127fef73a3abdf5e479f459eda38ac20dede8a0"} Jul 06 20:09:05.489697 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:05.489677 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-jxbvp" event={"ID":"e54c25e9-3542-48e1-9c6c-de6bd9b3f3d7","Type":"ContainerStarted","Data":"1b075ef0155a7498c6b7ac52fa923fcf098364141cbd0ceed977b16d28e073a1"} Jul 06 20:09:06.155924 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:06.155827 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/33f76cfa-c0b5-42e8-931f-572dff9fd39e-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-zszj5\" (UID: \"33f76cfa-c0b5-42e8-931f-572dff9fd39e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-zszj5" Jul 06 20:09:06.156119 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:06.156025 2551 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 20:09:06.156119 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:06.156112 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33f76cfa-c0b5-42e8-931f-572dff9fd39e-samples-operator-tls podName:33f76cfa-c0b5-42e8-931f-572dff9fd39e nodeName:}" failed. No retries permitted until 2026-07-06 20:09:08.156089039 +0000 UTC m=+58.571288826 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/33f76cfa-c0b5-42e8-931f-572dff9fd39e-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-zszj5" (UID: "33f76cfa-c0b5-42e8-931f-572dff9fd39e") : secret "samples-operator-tls" not found Jul 06 20:09:06.256885 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:06.256813 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/735538e5-306c-44a6-a934-30f463187f08-metrics-certs\") pod \"router-default-668d9f84f5-rrtln\" (UID: \"735538e5-306c-44a6-a934-30f463187f08\") " pod="openshift-ingress/router-default-668d9f84f5-rrtln" Jul 06 20:09:06.256885 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:06.256869 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/735538e5-306c-44a6-a934-30f463187f08-service-ca-bundle\") pod \"router-default-668d9f84f5-rrtln\" (UID: \"735538e5-306c-44a6-a934-30f463187f08\") " pod="openshift-ingress/router-default-668d9f84f5-rrtln" Jul 06 20:09:06.257055 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:06.256998 2551 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 20:09:06.257055 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:06.257046 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/735538e5-306c-44a6-a934-30f463187f08-service-ca-bundle podName:735538e5-306c-44a6-a934-30f463187f08 nodeName:}" failed. No retries permitted until 2026-07-06 20:09:08.257031677 +0000 UTC m=+58.672231483 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/735538e5-306c-44a6-a934-30f463187f08-service-ca-bundle") pod "router-default-668d9f84f5-rrtln" (UID: "735538e5-306c-44a6-a934-30f463187f08") : configmap references non-existent config key: service-ca.crt Jul 06 20:09:06.257173 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:06.257068 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/735538e5-306c-44a6-a934-30f463187f08-metrics-certs podName:735538e5-306c-44a6-a934-30f463187f08 nodeName:}" failed. No retries permitted until 2026-07-06 20:09:08.257058676 +0000 UTC m=+58.672258463 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/735538e5-306c-44a6-a934-30f463187f08-metrics-certs") pod "router-default-668d9f84f5-rrtln" (UID: "735538e5-306c-44a6-a934-30f463187f08") : secret "router-metrics-certs-default" not found Jul 06 20:09:06.494013 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:06.493923 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-847qg" event={"ID":"8c2d1989-b3d7-48a4-8417-23e639cbc855","Type":"ContainerStarted","Data":"a998510a08a7a304582b78295a148b7682ad90c2bc8da3a043ec924a67eac20b"} Jul 06 20:09:06.511145 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:06.511098 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-847qg" podStartSLOduration=1.272703214 podStartE2EDuration="2.511083322s" totalImagesPullingTime="1.238380108s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:09:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:09:06.510756688 +0000 UTC m=+56.925956492" watchObservedRunningTime="2026-07-06 20:09:06.511083322 +0000 UTC m=+56.926283110" Jul 06 20:09:07.498323 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:07.498231 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-jxbvp" event={"ID":"e54c25e9-3542-48e1-9c6c-de6bd9b3f3d7","Type":"ContainerStarted","Data":"d7f4fdcafee992bdec3c12d09bdb3efe1895fcf8db48d0b09c8ccb3a2d5cae1e"} Jul 06 20:09:08.170381 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:08.170350 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/33f76cfa-c0b5-42e8-931f-572dff9fd39e-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-zszj5\" (UID: \"33f76cfa-c0b5-42e8-931f-572dff9fd39e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-zszj5" Jul 06 20:09:08.170587 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:08.170519 2551 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 20:09:08.170657 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:08.170594 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33f76cfa-c0b5-42e8-931f-572dff9fd39e-samples-operator-tls podName:33f76cfa-c0b5-42e8-931f-572dff9fd39e nodeName:}" failed. No retries permitted until 2026-07-06 20:09:12.170572933 +0000 UTC m=+62.585772716 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/33f76cfa-c0b5-42e8-931f-572dff9fd39e-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-zszj5" (UID: "33f76cfa-c0b5-42e8-931f-572dff9fd39e") : secret "samples-operator-tls" not found Jul 06 20:09:08.271623 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:08.271582 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/735538e5-306c-44a6-a934-30f463187f08-metrics-certs\") pod \"router-default-668d9f84f5-rrtln\" (UID: \"735538e5-306c-44a6-a934-30f463187f08\") " pod="openshift-ingress/router-default-668d9f84f5-rrtln" Jul 06 20:09:08.271785 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:08.271644 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/735538e5-306c-44a6-a934-30f463187f08-service-ca-bundle\") pod \"router-default-668d9f84f5-rrtln\" (UID: \"735538e5-306c-44a6-a934-30f463187f08\") " pod="openshift-ingress/router-default-668d9f84f5-rrtln" Jul 06 20:09:08.271785 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:08.271729 2551 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 20:09:08.271852 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:08.271805 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/735538e5-306c-44a6-a934-30f463187f08-metrics-certs podName:735538e5-306c-44a6-a934-30f463187f08 nodeName:}" failed. No retries permitted until 2026-07-06 20:09:12.271789456 +0000 UTC m=+62.686989244 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/735538e5-306c-44a6-a934-30f463187f08-metrics-certs") pod "router-default-668d9f84f5-rrtln" (UID: "735538e5-306c-44a6-a934-30f463187f08") : secret "router-metrics-certs-default" not found Jul 06 20:09:08.271852 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:08.271820 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/735538e5-306c-44a6-a934-30f463187f08-service-ca-bundle podName:735538e5-306c-44a6-a934-30f463187f08 nodeName:}" failed. No retries permitted until 2026-07-06 20:09:12.271813848 +0000 UTC m=+62.687013630 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/735538e5-306c-44a6-a934-30f463187f08-service-ca-bundle") pod "router-default-668d9f84f5-rrtln" (UID: "735538e5-306c-44a6-a934-30f463187f08") : configmap references non-existent config key: service-ca.crt Jul 06 20:09:08.756912 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:08.756602 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-jxbvp" podStartSLOduration=2.574884209 podStartE2EDuration="4.756581507s" totalImagesPullingTime="2.181697298s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:09:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:09:07.521001511 +0000 UTC m=+57.936201344" watchObservedRunningTime="2026-07-06 20:09:08.756581507 +0000 UTC m=+59.171781312" Jul 06 20:09:08.757540 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:08.757518 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-zd4zj"] Jul 06 20:09:08.759781 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:08.759760 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zd4zj" Jul 06 20:09:08.762203 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:08.762179 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 06 20:09:08.762345 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:08.762328 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 06 20:09:08.762429 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:08.762369 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-884qp\"" Jul 06 20:09:08.773255 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:08.773233 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zd4zj"] Jul 06 20:09:08.775522 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:08.775494 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrllr\" (UniqueName: \"kubernetes.io/projected/f60316f2-a0cc-4ac1-9c8b-eaccb5977acd-kube-api-access-lrllr\") pod \"insights-runtime-extractor-zd4zj\" (UID: \"f60316f2-a0cc-4ac1-9c8b-eaccb5977acd\") " pod="openshift-insights/insights-runtime-extractor-zd4zj" Jul 06 20:09:08.775617 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:08.775545 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f60316f2-a0cc-4ac1-9c8b-eaccb5977acd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zd4zj\" (UID: \"f60316f2-a0cc-4ac1-9c8b-eaccb5977acd\") " pod="openshift-insights/insights-runtime-extractor-zd4zj" Jul 06 20:09:08.775617 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:08.775583 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f60316f2-a0cc-4ac1-9c8b-eaccb5977acd-crio-socket\") pod \"insights-runtime-extractor-zd4zj\" (UID: \"f60316f2-a0cc-4ac1-9c8b-eaccb5977acd\") " pod="openshift-insights/insights-runtime-extractor-zd4zj" Jul 06 20:09:08.775708 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:08.775664 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f60316f2-a0cc-4ac1-9c8b-eaccb5977acd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zd4zj\" (UID: \"f60316f2-a0cc-4ac1-9c8b-eaccb5977acd\") " pod="openshift-insights/insights-runtime-extractor-zd4zj" Jul 06 20:09:08.775823 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:08.775806 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f60316f2-a0cc-4ac1-9c8b-eaccb5977acd-data-volume\") pod \"insights-runtime-extractor-zd4zj\" (UID: \"f60316f2-a0cc-4ac1-9c8b-eaccb5977acd\") " pod="openshift-insights/insights-runtime-extractor-zd4zj" Jul 06 20:09:08.876101 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:08.876057 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f60316f2-a0cc-4ac1-9c8b-eaccb5977acd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zd4zj\" (UID: \"f60316f2-a0cc-4ac1-9c8b-eaccb5977acd\") " pod="openshift-insights/insights-runtime-extractor-zd4zj" Jul 06 20:09:08.876290 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:08.876161 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f60316f2-a0cc-4ac1-9c8b-eaccb5977acd-data-volume\") pod \"insights-runtime-extractor-zd4zj\" (UID: \"f60316f2-a0cc-4ac1-9c8b-eaccb5977acd\") " pod="openshift-insights/insights-runtime-extractor-zd4zj" Jul 06 20:09:08.876290 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:08.876193 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrllr\" (UniqueName: \"kubernetes.io/projected/f60316f2-a0cc-4ac1-9c8b-eaccb5977acd-kube-api-access-lrllr\") pod \"insights-runtime-extractor-zd4zj\" (UID: \"f60316f2-a0cc-4ac1-9c8b-eaccb5977acd\") " pod="openshift-insights/insights-runtime-extractor-zd4zj" Jul 06 20:09:08.876290 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:08.876205 2551 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 20:09:08.876290 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:08.876215 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f60316f2-a0cc-4ac1-9c8b-eaccb5977acd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zd4zj\" (UID: \"f60316f2-a0cc-4ac1-9c8b-eaccb5977acd\") " pod="openshift-insights/insights-runtime-extractor-zd4zj" Jul 06 20:09:08.876290 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:08.876259 2551 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 20:09:08.876533 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:08.876274 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f60316f2-a0cc-4ac1-9c8b-eaccb5977acd-insights-runtime-extractor-tls podName:f60316f2-a0cc-4ac1-9c8b-eaccb5977acd nodeName:}" failed. No retries permitted until 2026-07-06 20:09:09.376257952 +0000 UTC m=+59.791457735 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f60316f2-a0cc-4ac1-9c8b-eaccb5977acd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zd4zj" (UID: "f60316f2-a0cc-4ac1-9c8b-eaccb5977acd") : secret "insights-runtime-extractor-tls" not found Jul 06 20:09:08.876533 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:08.876356 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f60316f2-a0cc-4ac1-9c8b-eaccb5977acd-kube-rbac-proxy-cm podName:f60316f2-a0cc-4ac1-9c8b-eaccb5977acd nodeName:}" failed. No retries permitted until 2026-07-06 20:09:09.376332596 +0000 UTC m=+59.791532386 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f60316f2-a0cc-4ac1-9c8b-eaccb5977acd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zd4zj" (UID: "f60316f2-a0cc-4ac1-9c8b-eaccb5977acd") : configmap "kube-rbac-proxy" not found Jul 06 20:09:08.876533 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:08.876414 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f60316f2-a0cc-4ac1-9c8b-eaccb5977acd-crio-socket\") pod \"insights-runtime-extractor-zd4zj\" (UID: \"f60316f2-a0cc-4ac1-9c8b-eaccb5977acd\") " pod="openshift-insights/insights-runtime-extractor-zd4zj" Jul 06 20:09:08.876665 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:08.876542 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f60316f2-a0cc-4ac1-9c8b-eaccb5977acd-data-volume\") pod \"insights-runtime-extractor-zd4zj\" (UID: \"f60316f2-a0cc-4ac1-9c8b-eaccb5977acd\") " pod="openshift-insights/insights-runtime-extractor-zd4zj" Jul 06 20:09:08.876665 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:08.876519 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f60316f2-a0cc-4ac1-9c8b-eaccb5977acd-crio-socket\") pod \"insights-runtime-extractor-zd4zj\" (UID: \"f60316f2-a0cc-4ac1-9c8b-eaccb5977acd\") " pod="openshift-insights/insights-runtime-extractor-zd4zj" Jul 06 20:09:08.884869 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:08.884846 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrllr\" (UniqueName: \"kubernetes.io/projected/f60316f2-a0cc-4ac1-9c8b-eaccb5977acd-kube-api-access-lrllr\") pod \"insights-runtime-extractor-zd4zj\" (UID: \"f60316f2-a0cc-4ac1-9c8b-eaccb5977acd\") " pod="openshift-insights/insights-runtime-extractor-zd4zj" Jul 06 20:09:09.381087 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:09.381048 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f60316f2-a0cc-4ac1-9c8b-eaccb5977acd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zd4zj\" (UID: \"f60316f2-a0cc-4ac1-9c8b-eaccb5977acd\") " pod="openshift-insights/insights-runtime-extractor-zd4zj" Jul 06 20:09:09.381299 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:09.381093 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f60316f2-a0cc-4ac1-9c8b-eaccb5977acd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zd4zj\" (UID: \"f60316f2-a0cc-4ac1-9c8b-eaccb5977acd\") " pod="openshift-insights/insights-runtime-extractor-zd4zj" Jul 06 20:09:09.381299 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:09.381181 2551 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 20:09:09.381299 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:09.381260 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f60316f2-a0cc-4ac1-9c8b-eaccb5977acd-kube-rbac-proxy-cm podName:f60316f2-a0cc-4ac1-9c8b-eaccb5977acd nodeName:}" failed. No retries permitted until 2026-07-06 20:09:10.381240868 +0000 UTC m=+60.796440650 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f60316f2-a0cc-4ac1-9c8b-eaccb5977acd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zd4zj" (UID: "f60316f2-a0cc-4ac1-9c8b-eaccb5977acd") : configmap "kube-rbac-proxy" not found Jul 06 20:09:09.381299 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:09.381193 2551 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 20:09:09.381511 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:09.381320 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f60316f2-a0cc-4ac1-9c8b-eaccb5977acd-insights-runtime-extractor-tls podName:f60316f2-a0cc-4ac1-9c8b-eaccb5977acd nodeName:}" failed. No retries permitted until 2026-07-06 20:09:10.38130827 +0000 UTC m=+60.796508053 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f60316f2-a0cc-4ac1-9c8b-eaccb5977acd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zd4zj" (UID: "f60316f2-a0cc-4ac1-9c8b-eaccb5977acd") : secret "insights-runtime-extractor-tls" not found Jul 06 20:09:10.389811 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:10.389773 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f60316f2-a0cc-4ac1-9c8b-eaccb5977acd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zd4zj\" (UID: \"f60316f2-a0cc-4ac1-9c8b-eaccb5977acd\") " pod="openshift-insights/insights-runtime-extractor-zd4zj" Jul 06 20:09:10.390233 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:10.389818 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f60316f2-a0cc-4ac1-9c8b-eaccb5977acd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zd4zj\" (UID: \"f60316f2-a0cc-4ac1-9c8b-eaccb5977acd\") " pod="openshift-insights/insights-runtime-extractor-zd4zj" Jul 06 20:09:10.390233 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:10.389912 2551 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 20:09:10.390233 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:10.389970 2551 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 20:09:10.390233 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:10.389980 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f60316f2-a0cc-4ac1-9c8b-eaccb5977acd-kube-rbac-proxy-cm podName:f60316f2-a0cc-4ac1-9c8b-eaccb5977acd nodeName:}" failed. No retries permitted until 2026-07-06 20:09:12.389962693 +0000 UTC m=+62.805162475 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f60316f2-a0cc-4ac1-9c8b-eaccb5977acd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zd4zj" (UID: "f60316f2-a0cc-4ac1-9c8b-eaccb5977acd") : configmap "kube-rbac-proxy" not found Jul 06 20:09:10.390233 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:10.390014 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f60316f2-a0cc-4ac1-9c8b-eaccb5977acd-insights-runtime-extractor-tls podName:f60316f2-a0cc-4ac1-9c8b-eaccb5977acd nodeName:}" failed. No retries permitted until 2026-07-06 20:09:12.390001799 +0000 UTC m=+62.805201581 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f60316f2-a0cc-4ac1-9c8b-eaccb5977acd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zd4zj" (UID: "f60316f2-a0cc-4ac1-9c8b-eaccb5977acd") : secret "insights-runtime-extractor-tls" not found Jul 06 20:09:10.530025 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:10.529999 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-477lt_854b635c-f1e7-47d5-bf88-b5a80b572b1b/dns-node-resolver/0.log" Jul 06 20:09:11.183389 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:11.183348 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-q49qj"] Jul 06 20:09:11.210888 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:11.210853 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-q49qj"] Jul 06 20:09:11.211055 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:11.210985 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-q49qj" Jul 06 20:09:11.213783 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:11.213764 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-dmfqd\"" Jul 06 20:09:11.296202 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:11.296166 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snb6z\" (UniqueName: \"kubernetes.io/projected/5f7c44f0-1679-4a4c-b1fc-2f6e2212e0d9-kube-api-access-snb6z\") pod \"network-check-source-989f8b8dc-q49qj\" (UID: \"5f7c44f0-1679-4a4c-b1fc-2f6e2212e0d9\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-q49qj" Jul 06 20:09:11.396929 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:11.396874 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snb6z\" (UniqueName: \"kubernetes.io/projected/5f7c44f0-1679-4a4c-b1fc-2f6e2212e0d9-kube-api-access-snb6z\") pod \"network-check-source-989f8b8dc-q49qj\" (UID: \"5f7c44f0-1679-4a4c-b1fc-2f6e2212e0d9\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-q49qj" Jul 06 20:09:11.406275 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:11.406253 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 20:09:11.417469 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:11.417447 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 20:09:11.427773 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:11.427746 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-snb6z\" (UniqueName: \"kubernetes.io/projected/5f7c44f0-1679-4a4c-b1fc-2f6e2212e0d9-kube-api-access-snb6z\") pod \"network-check-source-989f8b8dc-q49qj\" (UID: \"5f7c44f0-1679-4a4c-b1fc-2f6e2212e0d9\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-q49qj" Jul 06 20:09:11.432698 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:11.432680 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-r7s8d" Jul 06 20:09:11.519289 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:11.519257 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-q49qj" Jul 06 20:09:11.660915 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:11.660869 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-q49qj"] Jul 06 20:09:11.663467 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:09:11.663440 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f7c44f0_1679_4a4c_b1fc_2f6e2212e0d9.slice/crio-683ea9c4c616bab571030486d7f37e1658bacb6395262c6ef119be1f1a4dcd57 WatchSource:0}: Error finding container 683ea9c4c616bab571030486d7f37e1658bacb6395262c6ef119be1f1a4dcd57: Status 404 returned error can't find the container with id 683ea9c4c616bab571030486d7f37e1658bacb6395262c6ef119be1f1a4dcd57 Jul 06 20:09:11.730367 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:11.730340 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mv9hq_9cd8d42e-db03-477c-8ef9-bfed943eadb3/node-ca/0.log" Jul 06 20:09:12.202662 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:12.202623 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/33f76cfa-c0b5-42e8-931f-572dff9fd39e-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-zszj5\" (UID: \"33f76cfa-c0b5-42e8-931f-572dff9fd39e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-zszj5" Jul 06 20:09:12.202843 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:12.202809 2551 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 20:09:12.202920 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:12.202910 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33f76cfa-c0b5-42e8-931f-572dff9fd39e-samples-operator-tls podName:33f76cfa-c0b5-42e8-931f-572dff9fd39e nodeName:}" failed. No retries permitted until 2026-07-06 20:09:20.202874577 +0000 UTC m=+70.618074362 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/33f76cfa-c0b5-42e8-931f-572dff9fd39e-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-zszj5" (UID: "33f76cfa-c0b5-42e8-931f-572dff9fd39e") : secret "samples-operator-tls" not found Jul 06 20:09:12.303594 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:12.303557 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/735538e5-306c-44a6-a934-30f463187f08-metrics-certs\") pod \"router-default-668d9f84f5-rrtln\" (UID: \"735538e5-306c-44a6-a934-30f463187f08\") " pod="openshift-ingress/router-default-668d9f84f5-rrtln" Jul 06 20:09:12.303594 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:12.303608 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/735538e5-306c-44a6-a934-30f463187f08-service-ca-bundle\") pod \"router-default-668d9f84f5-rrtln\" (UID: \"735538e5-306c-44a6-a934-30f463187f08\") " pod="openshift-ingress/router-default-668d9f84f5-rrtln" Jul 06 20:09:12.303830 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:12.303736 2551 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 20:09:12.303830 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:12.303774 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/735538e5-306c-44a6-a934-30f463187f08-service-ca-bundle podName:735538e5-306c-44a6-a934-30f463187f08 nodeName:}" failed. No retries permitted until 2026-07-06 20:09:20.303757307 +0000 UTC m=+70.718957111 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/735538e5-306c-44a6-a934-30f463187f08-service-ca-bundle") pod "router-default-668d9f84f5-rrtln" (UID: "735538e5-306c-44a6-a934-30f463187f08") : configmap references non-existent config key: service-ca.crt Jul 06 20:09:12.303830 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:12.303800 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/735538e5-306c-44a6-a934-30f463187f08-metrics-certs podName:735538e5-306c-44a6-a934-30f463187f08 nodeName:}" failed. No retries permitted until 2026-07-06 20:09:20.303785691 +0000 UTC m=+70.718985474 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/735538e5-306c-44a6-a934-30f463187f08-metrics-certs") pod "router-default-668d9f84f5-rrtln" (UID: "735538e5-306c-44a6-a934-30f463187f08") : secret "router-metrics-certs-default" not found Jul 06 20:09:12.370603 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:12.370563 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-p52gc"] Jul 06 20:09:12.372817 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:12.372786 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-p52gc" Jul 06 20:09:12.375384 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:12.375362 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 06 20:09:12.377452 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:12.377426 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 06 20:09:12.377579 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:12.377557 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 06 20:09:12.377642 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:12.377605 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 06 20:09:12.378484 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:12.378461 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-n4flf\"" Jul 06 20:09:12.385407 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:12.385385 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-p52gc"] Jul 06 20:09:12.404354 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:12.404325 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7znt\" (UniqueName: \"kubernetes.io/projected/2c866a8a-ebee-4296-a983-42c059034c9a-kube-api-access-l7znt\") pod \"service-ca-operator-5d46d9ffd4-p52gc\" (UID: \"2c866a8a-ebee-4296-a983-42c059034c9a\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-p52gc" Jul 06 20:09:12.404736 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:12.404381 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f60316f2-a0cc-4ac1-9c8b-eaccb5977acd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zd4zj\" (UID: \"f60316f2-a0cc-4ac1-9c8b-eaccb5977acd\") " pod="openshift-insights/insights-runtime-extractor-zd4zj" Jul 06 20:09:12.404736 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:12.404465 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2c866a8a-ebee-4296-a983-42c059034c9a-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-p52gc\" (UID: \"2c866a8a-ebee-4296-a983-42c059034c9a\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-p52gc" Jul 06 20:09:12.404736 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:12.404522 2551 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 20:09:12.404736 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:12.404583 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f60316f2-a0cc-4ac1-9c8b-eaccb5977acd-kube-rbac-proxy-cm podName:f60316f2-a0cc-4ac1-9c8b-eaccb5977acd nodeName:}" failed. No retries permitted until 2026-07-06 20:09:16.404567362 +0000 UTC m=+66.819767144 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f60316f2-a0cc-4ac1-9c8b-eaccb5977acd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zd4zj" (UID: "f60316f2-a0cc-4ac1-9c8b-eaccb5977acd") : configmap "kube-rbac-proxy" not found Jul 06 20:09:12.404736 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:12.404519 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2c866a8a-ebee-4296-a983-42c059034c9a-config\") pod \"service-ca-operator-5d46d9ffd4-p52gc\" (UID: \"2c866a8a-ebee-4296-a983-42c059034c9a\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-p52gc" Jul 06 20:09:12.404736 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:12.404634 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f60316f2-a0cc-4ac1-9c8b-eaccb5977acd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zd4zj\" (UID: \"f60316f2-a0cc-4ac1-9c8b-eaccb5977acd\") " pod="openshift-insights/insights-runtime-extractor-zd4zj" Jul 06 20:09:12.405055 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:12.404792 2551 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 20:09:12.405055 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:12.404864 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f60316f2-a0cc-4ac1-9c8b-eaccb5977acd-insights-runtime-extractor-tls podName:f60316f2-a0cc-4ac1-9c8b-eaccb5977acd nodeName:}" failed. No retries permitted until 2026-07-06 20:09:16.404847871 +0000 UTC m=+66.820047668 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f60316f2-a0cc-4ac1-9c8b-eaccb5977acd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zd4zj" (UID: "f60316f2-a0cc-4ac1-9c8b-eaccb5977acd") : secret "insights-runtime-extractor-tls" not found Jul 06 20:09:12.505881 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:12.505844 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7znt\" (UniqueName: \"kubernetes.io/projected/2c866a8a-ebee-4296-a983-42c059034c9a-kube-api-access-l7znt\") pod \"service-ca-operator-5d46d9ffd4-p52gc\" (UID: \"2c866a8a-ebee-4296-a983-42c059034c9a\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-p52gc" Jul 06 20:09:12.506071 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:12.505930 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2c866a8a-ebee-4296-a983-42c059034c9a-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-p52gc\" (UID: \"2c866a8a-ebee-4296-a983-42c059034c9a\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-p52gc" Jul 06 20:09:12.506148 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:12.506094 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2c866a8a-ebee-4296-a983-42c059034c9a-config\") pod \"service-ca-operator-5d46d9ffd4-p52gc\" (UID: \"2c866a8a-ebee-4296-a983-42c059034c9a\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-p52gc" Jul 06 20:09:12.506718 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:12.506691 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2c866a8a-ebee-4296-a983-42c059034c9a-config\") pod \"service-ca-operator-5d46d9ffd4-p52gc\" (UID: \"2c866a8a-ebee-4296-a983-42c059034c9a\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-p52gc" Jul 06 20:09:12.508421 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:12.508396 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2c866a8a-ebee-4296-a983-42c059034c9a-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-p52gc\" (UID: \"2c866a8a-ebee-4296-a983-42c059034c9a\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-p52gc" Jul 06 20:09:12.511817 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:12.511785 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-q49qj" event={"ID":"5f7c44f0-1679-4a4c-b1fc-2f6e2212e0d9","Type":"ContainerStarted","Data":"683ea9c4c616bab571030486d7f37e1658bacb6395262c6ef119be1f1a4dcd57"} Jul 06 20:09:12.516362 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:12.516336 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7znt\" (UniqueName: \"kubernetes.io/projected/2c866a8a-ebee-4296-a983-42c059034c9a-kube-api-access-l7znt\") pod \"service-ca-operator-5d46d9ffd4-p52gc\" (UID: \"2c866a8a-ebee-4296-a983-42c059034c9a\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-p52gc" Jul 06 20:09:12.526433 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:12.526411 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-z67pq_cf4f10b3-8b8d-45c2-9f64-8a37aa591b68/serve-healthcheck-canary/0.log" Jul 06 20:09:12.683704 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:12.683673 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-p52gc" Jul 06 20:09:12.822383 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:12.822352 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-p52gc"] Jul 06 20:09:12.825559 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:09:12.825529 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c866a8a_ebee_4296_a983_42c059034c9a.slice/crio-bd1c28acc947495bbf7ecb3c96c5bf113819956b9c4d17144465cdafd393ed2f WatchSource:0}: Error finding container bd1c28acc947495bbf7ecb3c96c5bf113819956b9c4d17144465cdafd393ed2f: Status 404 returned error can't find the container with id bd1c28acc947495bbf7ecb3c96c5bf113819956b9c4d17144465cdafd393ed2f Jul 06 20:09:13.516833 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:13.516678 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-p52gc" event={"ID":"2c866a8a-ebee-4296-a983-42c059034c9a","Type":"ContainerStarted","Data":"bd1c28acc947495bbf7ecb3c96c5bf113819956b9c4d17144465cdafd393ed2f"} Jul 06 20:09:15.523611 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:15.523570 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-q49qj" event={"ID":"5f7c44f0-1679-4a4c-b1fc-2f6e2212e0d9","Type":"ContainerStarted","Data":"a56328eb94054119a3a2fa54222dcf397ee487d1d7c54071fb30571015ee4e3e"} Jul 06 20:09:15.525023 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:15.524991 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-p52gc" event={"ID":"2c866a8a-ebee-4296-a983-42c059034c9a","Type":"ContainerStarted","Data":"25d5d3077be85267fb09c8d9d17d46100e0502ce2df7e16ed4f40afb18442a4c"} Jul 06 20:09:15.540853 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:15.540801 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-q49qj" podStartSLOduration=1.116049832 podStartE2EDuration="4.540786393s" totalImagesPullingTime="3.424736561s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:09:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:09:15.540518501 +0000 UTC m=+65.955718309" watchObservedRunningTime="2026-07-06 20:09:15.540786393 +0000 UTC m=+65.955986198" Jul 06 20:09:15.557571 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:15.557517 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-p52gc" podStartSLOduration=1.261286741 podStartE2EDuration="3.55749816s" totalImagesPullingTime="2.296211419s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:09:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:09:15.556140935 +0000 UTC m=+65.971340752" watchObservedRunningTime="2026-07-06 20:09:15.55749816 +0000 UTC m=+65.972697966" Jul 06 20:09:15.832684 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:15.832583 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef16cb88-f1e2-49c7-a470-47a23730758d-metrics-certs\") pod \"network-metrics-daemon-v648r\" (UID: \"ef16cb88-f1e2-49c7-a470-47a23730758d\") " pod="openshift-multus/network-metrics-daemon-v648r" Jul 06 20:09:15.832684 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:15.832628 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6909872-c920-4e38-a17a-11ad51d50379-original-pull-secret\") pod \"global-pull-secret-syncer-g6t4k\" (UID: \"d6909872-c920-4e38-a17a-11ad51d50379\") " pod="kube-system/global-pull-secret-syncer-g6t4k" Jul 06 20:09:15.835530 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:15.835509 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 20:09:15.835586 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:15.835513 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 20:09:15.843582 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:15.843556 2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 06 20:09:15.843657 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:15.843644 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef16cb88-f1e2-49c7-a470-47a23730758d-metrics-certs podName:ef16cb88-f1e2-49c7-a470-47a23730758d nodeName:}" failed. No retries permitted until 2026-07-06 20:10:19.843619894 +0000 UTC m=+130.258819679 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef16cb88-f1e2-49c7-a470-47a23730758d-metrics-certs") pod "network-metrics-daemon-v648r" (UID: "ef16cb88-f1e2-49c7-a470-47a23730758d") : secret "metrics-daemon-secret" not found Jul 06 20:09:15.845733 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:15.845700 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6909872-c920-4e38-a17a-11ad51d50379-original-pull-secret\") pod \"global-pull-secret-syncer-g6t4k\" (UID: \"d6909872-c920-4e38-a17a-11ad51d50379\") " pod="kube-system/global-pull-secret-syncer-g6t4k" Jul 06 20:09:16.034233 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:16.034190 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ds27q\" (UniqueName: \"kubernetes.io/projected/0899bbed-8629-4dd1-a6d0-be26f7870c7f-kube-api-access-ds27q\") pod \"network-check-target-6x889\" (UID: \"0899bbed-8629-4dd1-a6d0-be26f7870c7f\") " pod="openshift-network-diagnostics/network-check-target-6x889" Jul 06 20:09:16.037678 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:16.037656 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ds27q\" (UniqueName: \"kubernetes.io/projected/0899bbed-8629-4dd1-a6d0-be26f7870c7f-kube-api-access-ds27q\") pod \"network-check-target-6x889\" (UID: \"0899bbed-8629-4dd1-a6d0-be26f7870c7f\") " pod="openshift-network-diagnostics/network-check-target-6x889" Jul 06 20:09:16.075332 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:16.075299 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-rczn2\"" Jul 06 20:09:16.083220 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:16.083142 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6x889" Jul 06 20:09:16.086308 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:16.086282 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6t4k" Jul 06 20:09:16.222461 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:16.222435 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6x889"] Jul 06 20:09:16.225961 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:09:16.225931 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0899bbed_8629_4dd1_a6d0_be26f7870c7f.slice/crio-b089596b00021bc5f5458abbc3d24543cee3a120c0f6d77efc8364c84108beb9 WatchSource:0}: Error finding container b089596b00021bc5f5458abbc3d24543cee3a120c0f6d77efc8364c84108beb9: Status 404 returned error can't find the container with id b089596b00021bc5f5458abbc3d24543cee3a120c0f6d77efc8364c84108beb9 Jul 06 20:09:16.239596 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:16.239571 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-g6t4k"] Jul 06 20:09:16.243090 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:09:16.243058 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6909872_c920_4e38_a17a_11ad51d50379.slice/crio-32ba2ed5927aa373025bd3daf0dad4a6a07e147c0e82446621600f478dec6bdd WatchSource:0}: Error finding container 32ba2ed5927aa373025bd3daf0dad4a6a07e147c0e82446621600f478dec6bdd: Status 404 returned error can't find the container with id 32ba2ed5927aa373025bd3daf0dad4a6a07e147c0e82446621600f478dec6bdd Jul 06 20:09:16.437345 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:16.437254 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f60316f2-a0cc-4ac1-9c8b-eaccb5977acd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zd4zj\" (UID: \"f60316f2-a0cc-4ac1-9c8b-eaccb5977acd\") " pod="openshift-insights/insights-runtime-extractor-zd4zj" Jul 06 20:09:16.437345 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:16.437335 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f60316f2-a0cc-4ac1-9c8b-eaccb5977acd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zd4zj\" (UID: \"f60316f2-a0cc-4ac1-9c8b-eaccb5977acd\") " pod="openshift-insights/insights-runtime-extractor-zd4zj" Jul 06 20:09:16.437524 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:16.437382 2551 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 20:09:16.437524 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:16.437450 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f60316f2-a0cc-4ac1-9c8b-eaccb5977acd-kube-rbac-proxy-cm podName:f60316f2-a0cc-4ac1-9c8b-eaccb5977acd nodeName:}" failed. No retries permitted until 2026-07-06 20:09:24.43743364 +0000 UTC m=+74.852633423 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f60316f2-a0cc-4ac1-9c8b-eaccb5977acd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zd4zj" (UID: "f60316f2-a0cc-4ac1-9c8b-eaccb5977acd") : configmap "kube-rbac-proxy" not found Jul 06 20:09:16.437524 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:16.437465 2551 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 20:09:16.437524 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:16.437514 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f60316f2-a0cc-4ac1-9c8b-eaccb5977acd-insights-runtime-extractor-tls podName:f60316f2-a0cc-4ac1-9c8b-eaccb5977acd nodeName:}" failed. No retries permitted until 2026-07-06 20:09:24.437498228 +0000 UTC m=+74.852698013 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f60316f2-a0cc-4ac1-9c8b-eaccb5977acd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zd4zj" (UID: "f60316f2-a0cc-4ac1-9c8b-eaccb5977acd") : secret "insights-runtime-extractor-tls" not found Jul 06 20:09:16.529513 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:16.529473 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-g6t4k" event={"ID":"d6909872-c920-4e38-a17a-11ad51d50379","Type":"ContainerStarted","Data":"32ba2ed5927aa373025bd3daf0dad4a6a07e147c0e82446621600f478dec6bdd"} Jul 06 20:09:16.531129 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:16.531091 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6x889" event={"ID":"0899bbed-8629-4dd1-a6d0-be26f7870c7f","Type":"ContainerStarted","Data":"1079e92ab40935ff2bff3bc8acf828cc522b9a551ce3013e60e7bfd900cd9ae1"} Jul 06 20:09:16.531278 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:16.531138 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6x889" event={"ID":"0899bbed-8629-4dd1-a6d0-be26f7870c7f","Type":"ContainerStarted","Data":"b089596b00021bc5f5458abbc3d24543cee3a120c0f6d77efc8364c84108beb9"} Jul 06 20:09:16.548888 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:16.548839 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-6x889" podStartSLOduration=66.548825578 podStartE2EDuration="1m6.548825578s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:08:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:09:16.547935533 +0000 UTC m=+66.963135363" watchObservedRunningTime="2026-07-06 20:09:16.548825578 +0000 UTC m=+66.964025379" Jul 06 20:09:17.538153 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:17.538113 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-6x889" Jul 06 20:09:18.253084 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:18.253044 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/148d8232-818e-46d9-b74b-94f6d03bf080-metrics-tls\") pod \"dns-default-qwn7v\" (UID: \"148d8232-818e-46d9-b74b-94f6d03bf080\") " pod="openshift-dns/dns-default-qwn7v" Jul 06 20:09:18.253258 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:18.253194 2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 20:09:18.253258 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:18.253253 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/148d8232-818e-46d9-b74b-94f6d03bf080-metrics-tls podName:148d8232-818e-46d9-b74b-94f6d03bf080 nodeName:}" failed. No retries permitted until 2026-07-06 20:09:50.253239183 +0000 UTC m=+100.668438970 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/148d8232-818e-46d9-b74b-94f6d03bf080-metrics-tls") pod "dns-default-qwn7v" (UID: "148d8232-818e-46d9-b74b-94f6d03bf080") : secret "dns-default-metrics-tls" not found Jul 06 20:09:20.266176 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:20.266135 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/33f76cfa-c0b5-42e8-931f-572dff9fd39e-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-zszj5\" (UID: \"33f76cfa-c0b5-42e8-931f-572dff9fd39e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-zszj5" Jul 06 20:09:20.266617 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:20.266309 2551 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 20:09:20.266617 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:20.266384 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33f76cfa-c0b5-42e8-931f-572dff9fd39e-samples-operator-tls podName:33f76cfa-c0b5-42e8-931f-572dff9fd39e nodeName:}" failed. No retries permitted until 2026-07-06 20:09:36.266362976 +0000 UTC m=+86.681562763 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/33f76cfa-c0b5-42e8-931f-572dff9fd39e-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-zszj5" (UID: "33f76cfa-c0b5-42e8-931f-572dff9fd39e") : secret "samples-operator-tls" not found Jul 06 20:09:20.366872 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:20.366829 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/735538e5-306c-44a6-a934-30f463187f08-metrics-certs\") pod \"router-default-668d9f84f5-rrtln\" (UID: \"735538e5-306c-44a6-a934-30f463187f08\") " pod="openshift-ingress/router-default-668d9f84f5-rrtln" Jul 06 20:09:20.367053 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:20.366885 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/735538e5-306c-44a6-a934-30f463187f08-service-ca-bundle\") pod \"router-default-668d9f84f5-rrtln\" (UID: \"735538e5-306c-44a6-a934-30f463187f08\") " pod="openshift-ingress/router-default-668d9f84f5-rrtln" Jul 06 20:09:20.367053 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:20.366988 2551 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 20:09:20.367053 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:20.367022 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/735538e5-306c-44a6-a934-30f463187f08-service-ca-bundle podName:735538e5-306c-44a6-a934-30f463187f08 nodeName:}" failed. No retries permitted until 2026-07-06 20:09:36.3670057 +0000 UTC m=+86.782205483 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/735538e5-306c-44a6-a934-30f463187f08-service-ca-bundle") pod "router-default-668d9f84f5-rrtln" (UID: "735538e5-306c-44a6-a934-30f463187f08") : configmap references non-existent config key: service-ca.crt Jul 06 20:09:20.367053 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:20.367047 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/735538e5-306c-44a6-a934-30f463187f08-metrics-certs podName:735538e5-306c-44a6-a934-30f463187f08 nodeName:}" failed. No retries permitted until 2026-07-06 20:09:36.367040175 +0000 UTC m=+86.782239958 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/735538e5-306c-44a6-a934-30f463187f08-metrics-certs") pod "router-default-668d9f84f5-rrtln" (UID: "735538e5-306c-44a6-a934-30f463187f08") : secret "router-metrics-certs-default" not found Jul 06 20:09:20.553233 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:20.553141 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-g6t4k" event={"ID":"d6909872-c920-4e38-a17a-11ad51d50379","Type":"ContainerStarted","Data":"6a44108388724b13b96a5f3a76b6627fdbdd9894d93b2590966d5d3b03a34445"} Jul 06 20:09:20.571439 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:20.571279 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-g6t4k" podStartSLOduration=66.442504733 podStartE2EDuration="1m10.571258341s" totalImagesPullingTime="4.128753608s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:08:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:09:20.57114496 +0000 UTC m=+70.986344766" watchObservedRunningTime="2026-07-06 20:09:20.571258341 +0000 UTC m=+70.986458147" Jul 06 20:09:24.502959 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:24.502925 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f60316f2-a0cc-4ac1-9c8b-eaccb5977acd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zd4zj\" (UID: \"f60316f2-a0cc-4ac1-9c8b-eaccb5977acd\") " pod="openshift-insights/insights-runtime-extractor-zd4zj" Jul 06 20:09:24.503347 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:24.502985 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f60316f2-a0cc-4ac1-9c8b-eaccb5977acd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zd4zj\" (UID: \"f60316f2-a0cc-4ac1-9c8b-eaccb5977acd\") " pod="openshift-insights/insights-runtime-extractor-zd4zj" Jul 06 20:09:24.503347 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:24.503076 2551 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 20:09:24.503347 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:24.503125 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f60316f2-a0cc-4ac1-9c8b-eaccb5977acd-insights-runtime-extractor-tls podName:f60316f2-a0cc-4ac1-9c8b-eaccb5977acd nodeName:}" failed. No retries permitted until 2026-07-06 20:09:40.503112591 +0000 UTC m=+90.918312374 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f60316f2-a0cc-4ac1-9c8b-eaccb5977acd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zd4zj" (UID: "f60316f2-a0cc-4ac1-9c8b-eaccb5977acd") : secret "insights-runtime-extractor-tls" not found Jul 06 20:09:24.503477 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:24.503455 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f60316f2-a0cc-4ac1-9c8b-eaccb5977acd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zd4zj\" (UID: \"f60316f2-a0cc-4ac1-9c8b-eaccb5977acd\") " pod="openshift-insights/insights-runtime-extractor-zd4zj" Jul 06 20:09:36.287321 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:36.287191 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/33f76cfa-c0b5-42e8-931f-572dff9fd39e-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-zszj5\" (UID: \"33f76cfa-c0b5-42e8-931f-572dff9fd39e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-zszj5" Jul 06 20:09:36.289689 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:36.289660 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/33f76cfa-c0b5-42e8-931f-572dff9fd39e-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-zszj5\" (UID: \"33f76cfa-c0b5-42e8-931f-572dff9fd39e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-zszj5" Jul 06 20:09:36.388157 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:36.388122 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/735538e5-306c-44a6-a934-30f463187f08-metrics-certs\") pod \"router-default-668d9f84f5-rrtln\" (UID: \"735538e5-306c-44a6-a934-30f463187f08\") " pod="openshift-ingress/router-default-668d9f84f5-rrtln" Jul 06 20:09:36.388316 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:36.388167 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/735538e5-306c-44a6-a934-30f463187f08-service-ca-bundle\") pod \"router-default-668d9f84f5-rrtln\" (UID: \"735538e5-306c-44a6-a934-30f463187f08\") " pod="openshift-ingress/router-default-668d9f84f5-rrtln" Jul 06 20:09:36.388745 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:36.388724 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/735538e5-306c-44a6-a934-30f463187f08-service-ca-bundle\") pod \"router-default-668d9f84f5-rrtln\" (UID: \"735538e5-306c-44a6-a934-30f463187f08\") " pod="openshift-ingress/router-default-668d9f84f5-rrtln" Jul 06 20:09:36.390509 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:36.390490 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/735538e5-306c-44a6-a934-30f463187f08-metrics-certs\") pod \"router-default-668d9f84f5-rrtln\" (UID: \"735538e5-306c-44a6-a934-30f463187f08\") " pod="openshift-ingress/router-default-668d9f84f5-rrtln" Jul 06 20:09:36.529692 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:36.529658 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-jc7zf\"" Jul 06 20:09:36.537987 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:36.537932 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-zszj5" Jul 06 20:09:36.651282 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:36.651257 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-b26bd\"" Jul 06 20:09:36.659368 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:36.659349 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-668d9f84f5-rrtln" Jul 06 20:09:36.663426 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:36.663401 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-zszj5"] Jul 06 20:09:36.794063 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:36.793993 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-668d9f84f5-rrtln"] Jul 06 20:09:36.797122 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:09:36.797094 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod735538e5_306c_44a6_a934_30f463187f08.slice/crio-a036d45453cab9a7d3b897faf21fda48f433c9ee77e20e5ff084492f9307e9b2 WatchSource:0}: Error finding container a036d45453cab9a7d3b897faf21fda48f433c9ee77e20e5ff084492f9307e9b2: Status 404 returned error can't find the container with id a036d45453cab9a7d3b897faf21fda48f433c9ee77e20e5ff084492f9307e9b2 Jul 06 20:09:37.609806 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:37.609553 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-zszj5" event={"ID":"33f76cfa-c0b5-42e8-931f-572dff9fd39e","Type":"ContainerStarted","Data":"f66518084d3d4cdaf9d319966b355fb51426eb3899cb227fcac952d33df1e063"} Jul 06 20:09:37.614789 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:37.614752 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-668d9f84f5-rrtln" event={"ID":"735538e5-306c-44a6-a934-30f463187f08","Type":"ContainerStarted","Data":"543575f1890e55ae24ac488eed7759098d5c8578b3020c5296950d73bc244e1c"} Jul 06 20:09:37.614960 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:37.614843 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-668d9f84f5-rrtln" event={"ID":"735538e5-306c-44a6-a934-30f463187f08","Type":"ContainerStarted","Data":"a036d45453cab9a7d3b897faf21fda48f433c9ee77e20e5ff084492f9307e9b2"} Jul 06 20:09:37.637554 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:37.637502 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-668d9f84f5-rrtln" podStartSLOduration=33.637484873 podStartE2EDuration="33.637484873s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:09:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:09:37.636456811 +0000 UTC m=+88.051656642" watchObservedRunningTime="2026-07-06 20:09:37.637484873 +0000 UTC m=+88.052684680" Jul 06 20:09:37.660513 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:37.660490 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-668d9f84f5-rrtln" Jul 06 20:09:37.663292 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:37.663270 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-668d9f84f5-rrtln" Jul 06 20:09:38.617967 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:38.617942 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-668d9f84f5-rrtln" Jul 06 20:09:38.619211 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:38.619192 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-668d9f84f5-rrtln" Jul 06 20:09:39.622904 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:39.622866 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-zszj5" event={"ID":"33f76cfa-c0b5-42e8-931f-572dff9fd39e","Type":"ContainerStarted","Data":"173f2783e44c02e05ed62ca7d6609e6e09008e0ee18ff0c60beb7ce61350c12d"} Jul 06 20:09:39.623282 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:39.622942 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-zszj5" event={"ID":"33f76cfa-c0b5-42e8-931f-572dff9fd39e","Type":"ContainerStarted","Data":"3540adf1aa910c092832e5abfce4571ff59cfbb63992db07bae6bca95682e307"} Jul 06 20:09:39.643546 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:39.643502 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-zszj5" podStartSLOduration=33.746526286 podStartE2EDuration="35.643486463s" totalImagesPullingTime="1.896960177s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:09:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:09:39.642624021 +0000 UTC m=+90.057823849" watchObservedRunningTime="2026-07-06 20:09:39.643486463 +0000 UTC m=+90.058686268" Jul 06 20:09:40.521291 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:40.521257 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f60316f2-a0cc-4ac1-9c8b-eaccb5977acd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zd4zj\" (UID: \"f60316f2-a0cc-4ac1-9c8b-eaccb5977acd\") " pod="openshift-insights/insights-runtime-extractor-zd4zj" Jul 06 20:09:40.523471 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:40.523454 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f60316f2-a0cc-4ac1-9c8b-eaccb5977acd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zd4zj\" (UID: \"f60316f2-a0cc-4ac1-9c8b-eaccb5977acd\") " pod="openshift-insights/insights-runtime-extractor-zd4zj" Jul 06 20:09:40.575186 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:40.575157 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-884qp\"" Jul 06 20:09:40.582462 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:40.582445 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zd4zj" Jul 06 20:09:40.707360 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:40.707329 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zd4zj"] Jul 06 20:09:40.710374 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:09:40.710343 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf60316f2_a0cc_4ac1_9c8b_eaccb5977acd.slice/crio-68d79de2fc17ab9f5ddcb8955fc055a5fc9bd85c6ed0ed4b4cc396e05e8702c7 WatchSource:0}: Error finding container 68d79de2fc17ab9f5ddcb8955fc055a5fc9bd85c6ed0ed4b4cc396e05e8702c7: Status 404 returned error can't find the container with id 68d79de2fc17ab9f5ddcb8955fc055a5fc9bd85c6ed0ed4b4cc396e05e8702c7 Jul 06 20:09:41.633412 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:41.633317 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zd4zj" event={"ID":"f60316f2-a0cc-4ac1-9c8b-eaccb5977acd","Type":"ContainerStarted","Data":"68d79de2fc17ab9f5ddcb8955fc055a5fc9bd85c6ed0ed4b4cc396e05e8702c7"} Jul 06 20:09:44.645379 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:44.645342 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zd4zj" event={"ID":"f60316f2-a0cc-4ac1-9c8b-eaccb5977acd","Type":"ContainerStarted","Data":"b7510909d59bd10d0716f3800be9a4bcadcf5ea068dcaa34d0f781090a8b8f82"} Jul 06 20:09:45.429214 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:45.429177 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-94665fb6-4xkqt"] Jul 06 20:09:45.431472 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:45.431454 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-94665fb6-4xkqt" Jul 06 20:09:45.436547 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:45.436524 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 06 20:09:45.441456 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:45.441433 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 06 20:09:45.441804 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:45.441789 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 06 20:09:45.442465 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:45.442449 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-gn7n6\"" Jul 06 20:09:45.447953 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:45.447936 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 06 20:09:45.454130 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:45.454108 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-94665fb6-4xkqt"] Jul 06 20:09:45.559483 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:45.559447 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5d20463d-fd91-4376-90aa-b38ed6f9b1c0-image-registry-private-configuration\") pod \"image-registry-94665fb6-4xkqt\" (UID: \"5d20463d-fd91-4376-90aa-b38ed6f9b1c0\") " pod="openshift-image-registry/image-registry-94665fb6-4xkqt" Jul 06 20:09:45.559483 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:45.559485 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5d20463d-fd91-4376-90aa-b38ed6f9b1c0-registry-tls\") pod \"image-registry-94665fb6-4xkqt\" (UID: \"5d20463d-fd91-4376-90aa-b38ed6f9b1c0\") " pod="openshift-image-registry/image-registry-94665fb6-4xkqt" Jul 06 20:09:45.559745 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:45.559512 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2ztj\" (UniqueName: \"kubernetes.io/projected/5d20463d-fd91-4376-90aa-b38ed6f9b1c0-kube-api-access-n2ztj\") pod \"image-registry-94665fb6-4xkqt\" (UID: \"5d20463d-fd91-4376-90aa-b38ed6f9b1c0\") " pod="openshift-image-registry/image-registry-94665fb6-4xkqt" Jul 06 20:09:45.559745 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:45.559606 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5d20463d-fd91-4376-90aa-b38ed6f9b1c0-registry-certificates\") pod \"image-registry-94665fb6-4xkqt\" (UID: \"5d20463d-fd91-4376-90aa-b38ed6f9b1c0\") " pod="openshift-image-registry/image-registry-94665fb6-4xkqt" Jul 06 20:09:45.559745 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:45.559703 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5d20463d-fd91-4376-90aa-b38ed6f9b1c0-bound-sa-token\") pod \"image-registry-94665fb6-4xkqt\" (UID: \"5d20463d-fd91-4376-90aa-b38ed6f9b1c0\") " pod="openshift-image-registry/image-registry-94665fb6-4xkqt" Jul 06 20:09:45.559745 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:45.559731 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5d20463d-fd91-4376-90aa-b38ed6f9b1c0-installation-pull-secrets\") pod \"image-registry-94665fb6-4xkqt\" (UID: \"5d20463d-fd91-4376-90aa-b38ed6f9b1c0\") " pod="openshift-image-registry/image-registry-94665fb6-4xkqt" Jul 06 20:09:45.559953 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:45.559767 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5d20463d-fd91-4376-90aa-b38ed6f9b1c0-ca-trust-extracted\") pod \"image-registry-94665fb6-4xkqt\" (UID: \"5d20463d-fd91-4376-90aa-b38ed6f9b1c0\") " pod="openshift-image-registry/image-registry-94665fb6-4xkqt" Jul 06 20:09:45.559953 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:45.559791 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5d20463d-fd91-4376-90aa-b38ed6f9b1c0-trusted-ca\") pod \"image-registry-94665fb6-4xkqt\" (UID: \"5d20463d-fd91-4376-90aa-b38ed6f9b1c0\") " pod="openshift-image-registry/image-registry-94665fb6-4xkqt" Jul 06 20:09:45.655623 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:45.655537 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zd4zj" event={"ID":"f60316f2-a0cc-4ac1-9c8b-eaccb5977acd","Type":"ContainerStarted","Data":"2bf5c7440305d92b4178872da01e4d90366198c49c9eaddc982afa8c6bbf7489"} Jul 06 20:09:45.661034 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:45.660998 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5d20463d-fd91-4376-90aa-b38ed6f9b1c0-ca-trust-extracted\") pod \"image-registry-94665fb6-4xkqt\" (UID: \"5d20463d-fd91-4376-90aa-b38ed6f9b1c0\") " pod="openshift-image-registry/image-registry-94665fb6-4xkqt" Jul 06 20:09:45.661188 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:45.661041 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5d20463d-fd91-4376-90aa-b38ed6f9b1c0-trusted-ca\") pod \"image-registry-94665fb6-4xkqt\" (UID: \"5d20463d-fd91-4376-90aa-b38ed6f9b1c0\") " pod="openshift-image-registry/image-registry-94665fb6-4xkqt" Jul 06 20:09:45.661188 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:45.661080 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5d20463d-fd91-4376-90aa-b38ed6f9b1c0-image-registry-private-configuration\") pod \"image-registry-94665fb6-4xkqt\" (UID: \"5d20463d-fd91-4376-90aa-b38ed6f9b1c0\") " pod="openshift-image-registry/image-registry-94665fb6-4xkqt" Jul 06 20:09:45.661188 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:45.661106 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5d20463d-fd91-4376-90aa-b38ed6f9b1c0-registry-tls\") pod \"image-registry-94665fb6-4xkqt\" (UID: \"5d20463d-fd91-4376-90aa-b38ed6f9b1c0\") " pod="openshift-image-registry/image-registry-94665fb6-4xkqt" Jul 06 20:09:45.661188 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:45.661149 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2ztj\" (UniqueName: \"kubernetes.io/projected/5d20463d-fd91-4376-90aa-b38ed6f9b1c0-kube-api-access-n2ztj\") pod \"image-registry-94665fb6-4xkqt\" (UID: \"5d20463d-fd91-4376-90aa-b38ed6f9b1c0\") " pod="openshift-image-registry/image-registry-94665fb6-4xkqt" Jul 06 20:09:45.661188 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:45.661180 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5d20463d-fd91-4376-90aa-b38ed6f9b1c0-registry-certificates\") pod \"image-registry-94665fb6-4xkqt\" (UID: \"5d20463d-fd91-4376-90aa-b38ed6f9b1c0\") " pod="openshift-image-registry/image-registry-94665fb6-4xkqt" Jul 06 20:09:45.661426 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:45.661251 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5d20463d-fd91-4376-90aa-b38ed6f9b1c0-bound-sa-token\") pod \"image-registry-94665fb6-4xkqt\" (UID: \"5d20463d-fd91-4376-90aa-b38ed6f9b1c0\") " pod="openshift-image-registry/image-registry-94665fb6-4xkqt" Jul 06 20:09:45.661426 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:45.661278 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5d20463d-fd91-4376-90aa-b38ed6f9b1c0-installation-pull-secrets\") pod \"image-registry-94665fb6-4xkqt\" (UID: \"5d20463d-fd91-4376-90aa-b38ed6f9b1c0\") " pod="openshift-image-registry/image-registry-94665fb6-4xkqt" Jul 06 20:09:45.661529 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:45.661447 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5d20463d-fd91-4376-90aa-b38ed6f9b1c0-ca-trust-extracted\") pod \"image-registry-94665fb6-4xkqt\" (UID: \"5d20463d-fd91-4376-90aa-b38ed6f9b1c0\") " pod="openshift-image-registry/image-registry-94665fb6-4xkqt" Jul 06 20:09:45.662148 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:45.662092 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5d20463d-fd91-4376-90aa-b38ed6f9b1c0-trusted-ca\") pod \"image-registry-94665fb6-4xkqt\" (UID: \"5d20463d-fd91-4376-90aa-b38ed6f9b1c0\") " pod="openshift-image-registry/image-registry-94665fb6-4xkqt" Jul 06 20:09:45.662308 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:45.662288 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5d20463d-fd91-4376-90aa-b38ed6f9b1c0-registry-certificates\") pod \"image-registry-94665fb6-4xkqt\" (UID: \"5d20463d-fd91-4376-90aa-b38ed6f9b1c0\") " pod="openshift-image-registry/image-registry-94665fb6-4xkqt" Jul 06 20:09:45.663931 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:45.663888 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5d20463d-fd91-4376-90aa-b38ed6f9b1c0-image-registry-private-configuration\") pod \"image-registry-94665fb6-4xkqt\" (UID: \"5d20463d-fd91-4376-90aa-b38ed6f9b1c0\") " pod="openshift-image-registry/image-registry-94665fb6-4xkqt" Jul 06 20:09:45.664078 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:45.664056 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5d20463d-fd91-4376-90aa-b38ed6f9b1c0-installation-pull-secrets\") pod \"image-registry-94665fb6-4xkqt\" (UID: \"5d20463d-fd91-4376-90aa-b38ed6f9b1c0\") " pod="openshift-image-registry/image-registry-94665fb6-4xkqt" Jul 06 20:09:45.664486 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:45.664461 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5d20463d-fd91-4376-90aa-b38ed6f9b1c0-registry-tls\") pod \"image-registry-94665fb6-4xkqt\" (UID: \"5d20463d-fd91-4376-90aa-b38ed6f9b1c0\") " pod="openshift-image-registry/image-registry-94665fb6-4xkqt" Jul 06 20:09:45.697646 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:45.697607 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5d20463d-fd91-4376-90aa-b38ed6f9b1c0-bound-sa-token\") pod \"image-registry-94665fb6-4xkqt\" (UID: \"5d20463d-fd91-4376-90aa-b38ed6f9b1c0\") " pod="openshift-image-registry/image-registry-94665fb6-4xkqt" Jul 06 20:09:45.699347 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:45.699299 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2ztj\" (UniqueName: \"kubernetes.io/projected/5d20463d-fd91-4376-90aa-b38ed6f9b1c0-kube-api-access-n2ztj\") pod \"image-registry-94665fb6-4xkqt\" (UID: \"5d20463d-fd91-4376-90aa-b38ed6f9b1c0\") " pod="openshift-image-registry/image-registry-94665fb6-4xkqt" Jul 06 20:09:45.739997 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:45.739959 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-94665fb6-4xkqt" Jul 06 20:09:45.897689 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:45.897658 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-94665fb6-4xkqt"] Jul 06 20:09:45.900869 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:09:45.900831 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d20463d_fd91_4376_90aa_b38ed6f9b1c0.slice/crio-8dc52909f550fcc49265c751474f55923e214428a9b72de89ad2d06f5a61acdd WatchSource:0}: Error finding container 8dc52909f550fcc49265c751474f55923e214428a9b72de89ad2d06f5a61acdd: Status 404 returned error can't find the container with id 8dc52909f550fcc49265c751474f55923e214428a9b72de89ad2d06f5a61acdd Jul 06 20:09:46.661801 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:46.661759 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-94665fb6-4xkqt" event={"ID":"5d20463d-fd91-4376-90aa-b38ed6f9b1c0","Type":"ContainerStarted","Data":"4825fed96563865eaf2ac6fecf58f7125d80947d9a82a93b1498b010f197fcd8"} Jul 06 20:09:46.662206 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:46.661822 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-94665fb6-4xkqt" event={"ID":"5d20463d-fd91-4376-90aa-b38ed6f9b1c0","Type":"ContainerStarted","Data":"8dc52909f550fcc49265c751474f55923e214428a9b72de89ad2d06f5a61acdd"} Jul 06 20:09:46.662206 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:46.661870 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-94665fb6-4xkqt" Jul 06 20:09:46.710850 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:46.710787 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-94665fb6-4xkqt" podStartSLOduration=1.7107663720000001 podStartE2EDuration="1.710766372s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:09:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:09:46.708985002 +0000 UTC m=+97.124184830" watchObservedRunningTime="2026-07-06 20:09:46.710766372 +0000 UTC m=+97.125966197" Jul 06 20:09:48.546674 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:48.546640 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-6x889" Jul 06 20:09:48.671466 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:48.671431 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zd4zj" event={"ID":"f60316f2-a0cc-4ac1-9c8b-eaccb5977acd","Type":"ContainerStarted","Data":"ac95083419e340f4fb1b0eefd0091e2cd295bec2c70deeb0e12cf441b8df814d"} Jul 06 20:09:48.692617 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:48.692562 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-zd4zj" podStartSLOduration=33.362515638 podStartE2EDuration="40.692545038s" totalImagesPullingTime="7.3300294s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:09:08 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:09:48.691610859 +0000 UTC m=+99.106810703" watchObservedRunningTime="2026-07-06 20:09:48.692545038 +0000 UTC m=+99.107744843" Jul 06 20:09:50.302078 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:50.302032 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/148d8232-818e-46d9-b74b-94f6d03bf080-metrics-tls\") pod \"dns-default-qwn7v\" (UID: \"148d8232-818e-46d9-b74b-94f6d03bf080\") " pod="openshift-dns/dns-default-qwn7v" Jul 06 20:09:50.304444 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:50.304417 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/148d8232-818e-46d9-b74b-94f6d03bf080-metrics-tls\") pod \"dns-default-qwn7v\" (UID: \"148d8232-818e-46d9-b74b-94f6d03bf080\") " pod="openshift-dns/dns-default-qwn7v" Jul 06 20:09:50.607797 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:50.607492 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-t7dkq\"" Jul 06 20:09:50.615673 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:50.615645 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qwn7v" Jul 06 20:09:50.752593 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:50.752565 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qwn7v"] Jul 06 20:09:50.754941 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:09:50.754907 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod148d8232_818e_46d9_b74b_94f6d03bf080.slice/crio-c6dcc85fa6abc3d12693f917493c07a915472d5f1a434da7b1e3186dfacc9322 WatchSource:0}: Error finding container c6dcc85fa6abc3d12693f917493c07a915472d5f1a434da7b1e3186dfacc9322: Status 404 returned error can't find the container with id c6dcc85fa6abc3d12693f917493c07a915472d5f1a434da7b1e3186dfacc9322 Jul 06 20:09:51.685371 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:51.685337 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qwn7v" event={"ID":"148d8232-818e-46d9-b74b-94f6d03bf080","Type":"ContainerStarted","Data":"c6dcc85fa6abc3d12693f917493c07a915472d5f1a434da7b1e3186dfacc9322"} Jul 06 20:09:52.690847 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:52.690751 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qwn7v" event={"ID":"148d8232-818e-46d9-b74b-94f6d03bf080","Type":"ContainerStarted","Data":"8fc7ba4097568447c8cc5dec08274cf6cb988c64d50496274546dcdc00a4f91a"} Jul 06 20:09:52.690847 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:52.690806 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qwn7v" event={"ID":"148d8232-818e-46d9-b74b-94f6d03bf080","Type":"ContainerStarted","Data":"ad0f470c283acc94969ab69cd27fbca60aea9654e836500ee87198829a19089d"} Jul 06 20:09:52.691269 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:52.690873 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-qwn7v" Jul 06 20:09:52.710262 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:52.710210 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-qwn7v" podStartSLOduration=65.17339432 podStartE2EDuration="1m6.710166805s" totalImagesPullingTime="1.536772485s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:08:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:09:52.709523251 +0000 UTC m=+103.124723079" watchObservedRunningTime="2026-07-06 20:09:52.710166805 +0000 UTC m=+103.125366610" Jul 06 20:09:53.331865 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:53.331828 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-78wq2"] Jul 06 20:09:53.333879 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:53.333861 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-78wq2" Jul 06 20:09:53.337341 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:53.337316 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 06 20:09:53.338317 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:53.338294 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 06 20:09:53.338518 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:53.338492 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 06 20:09:53.338648 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:53.338628 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 06 20:09:53.338719 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:53.338663 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 06 20:09:53.338719 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:53.338671 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-kn6gw\"" Jul 06 20:09:53.345879 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:53.345859 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-78wq2"] Jul 06 20:09:53.384425 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:53.384393 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-v2rzd"] Jul 06 20:09:53.386497 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:53.386477 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-v2rzd" Jul 06 20:09:53.389061 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:53.389028 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 06 20:09:53.389414 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:53.389307 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 06 20:09:53.389414 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:53.389342 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 06 20:09:53.389971 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:53.389942 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-dsscq\"" Jul 06 20:09:53.427984 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:53.427946 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c82a93e6-feb4-4977-889d-175921fd1395-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-78wq2\" (UID: \"c82a93e6-feb4-4977-889d-175921fd1395\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-78wq2" Jul 06 20:09:53.428190 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:53.428000 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9k4r2\" (UniqueName: \"kubernetes.io/projected/c82a93e6-feb4-4977-889d-175921fd1395-kube-api-access-9k4r2\") pod \"openshift-state-metrics-54cf96dfcf-78wq2\" (UID: \"c82a93e6-feb4-4977-889d-175921fd1395\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-78wq2" Jul 06 20:09:53.428190 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:53.428028 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c82a93e6-feb4-4977-889d-175921fd1395-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-78wq2\" (UID: \"c82a93e6-feb4-4977-889d-175921fd1395\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-78wq2" Jul 06 20:09:53.428190 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:53.428093 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c82a93e6-feb4-4977-889d-175921fd1395-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-78wq2\" (UID: \"c82a93e6-feb4-4977-889d-175921fd1395\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-78wq2" Jul 06 20:09:53.529246 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:53.529202 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c82a93e6-feb4-4977-889d-175921fd1395-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-78wq2\" (UID: \"c82a93e6-feb4-4977-889d-175921fd1395\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-78wq2" Jul 06 20:09:53.529246 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:53.529250 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7226da11-2638-40c4-92fd-d5e2801dc9ac-node-exporter-tls\") pod \"node-exporter-v2rzd\" (UID: \"7226da11-2638-40c4-92fd-d5e2801dc9ac\") " pod="openshift-monitoring/node-exporter-v2rzd" Jul 06 20:09:53.529519 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:53.529273 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7226da11-2638-40c4-92fd-d5e2801dc9ac-node-exporter-accelerators-collector-config\") pod \"node-exporter-v2rzd\" (UID: \"7226da11-2638-40c4-92fd-d5e2801dc9ac\") " pod="openshift-monitoring/node-exporter-v2rzd" Jul 06 20:09:53.529519 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:53.529301 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7226da11-2638-40c4-92fd-d5e2801dc9ac-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-v2rzd\" (UID: \"7226da11-2638-40c4-92fd-d5e2801dc9ac\") " pod="openshift-monitoring/node-exporter-v2rzd" Jul 06 20:09:53.529519 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:53.529362 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxl2v\" (UniqueName: \"kubernetes.io/projected/7226da11-2638-40c4-92fd-d5e2801dc9ac-kube-api-access-nxl2v\") pod \"node-exporter-v2rzd\" (UID: \"7226da11-2638-40c4-92fd-d5e2801dc9ac\") " pod="openshift-monitoring/node-exporter-v2rzd" Jul 06 20:09:53.529519 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:53.529397 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7226da11-2638-40c4-92fd-d5e2801dc9ac-sys\") pod \"node-exporter-v2rzd\" (UID: \"7226da11-2638-40c4-92fd-d5e2801dc9ac\") " pod="openshift-monitoring/node-exporter-v2rzd" Jul 06 20:09:53.529519 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:53.529421 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7226da11-2638-40c4-92fd-d5e2801dc9ac-node-exporter-wtmp\") pod \"node-exporter-v2rzd\" (UID: \"7226da11-2638-40c4-92fd-d5e2801dc9ac\") " pod="openshift-monitoring/node-exporter-v2rzd" Jul 06 20:09:53.529519 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:53.529442 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9k4r2\" (UniqueName: \"kubernetes.io/projected/c82a93e6-feb4-4977-889d-175921fd1395-kube-api-access-9k4r2\") pod \"openshift-state-metrics-54cf96dfcf-78wq2\" (UID: \"c82a93e6-feb4-4977-889d-175921fd1395\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-78wq2" Jul 06 20:09:53.529519 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:53.529456 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7226da11-2638-40c4-92fd-d5e2801dc9ac-node-exporter-textfile\") pod \"node-exporter-v2rzd\" (UID: \"7226da11-2638-40c4-92fd-d5e2801dc9ac\") " pod="openshift-monitoring/node-exporter-v2rzd" Jul 06 20:09:53.529827 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:53.529526 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c82a93e6-feb4-4977-889d-175921fd1395-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-78wq2\" (UID: \"c82a93e6-feb4-4977-889d-175921fd1395\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-78wq2" Jul 06 20:09:53.529827 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:53.529553 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7226da11-2638-40c4-92fd-d5e2801dc9ac-metrics-client-ca\") pod \"node-exporter-v2rzd\" (UID: \"7226da11-2638-40c4-92fd-d5e2801dc9ac\") " pod="openshift-monitoring/node-exporter-v2rzd" Jul 06 20:09:53.529827 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:53.529583 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c82a93e6-feb4-4977-889d-175921fd1395-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-78wq2\" (UID: \"c82a93e6-feb4-4977-889d-175921fd1395\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-78wq2" Jul 06 20:09:53.529827 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:53.529604 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7226da11-2638-40c4-92fd-d5e2801dc9ac-root\") pod \"node-exporter-v2rzd\" (UID: \"7226da11-2638-40c4-92fd-d5e2801dc9ac\") " pod="openshift-monitoring/node-exporter-v2rzd" Jul 06 20:09:53.529827 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:53.529737 2551 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 06 20:09:53.529827 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:09:53.529800 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c82a93e6-feb4-4977-889d-175921fd1395-openshift-state-metrics-tls podName:c82a93e6-feb4-4977-889d-175921fd1395 nodeName:}" failed. No retries permitted until 2026-07-06 20:09:54.029782134 +0000 UTC m=+104.444981917 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/c82a93e6-feb4-4977-889d-175921fd1395-openshift-state-metrics-tls") pod "openshift-state-metrics-54cf96dfcf-78wq2" (UID: "c82a93e6-feb4-4977-889d-175921fd1395") : secret "openshift-state-metrics-tls" not found Jul 06 20:09:53.530250 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:53.530224 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c82a93e6-feb4-4977-889d-175921fd1395-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-78wq2\" (UID: \"c82a93e6-feb4-4977-889d-175921fd1395\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-78wq2" Jul 06 20:09:53.531710 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:53.531685 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c82a93e6-feb4-4977-889d-175921fd1395-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-78wq2\" (UID: \"c82a93e6-feb4-4977-889d-175921fd1395\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-78wq2" Jul 06 20:09:53.539215 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:53.539183 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9k4r2\" (UniqueName: \"kubernetes.io/projected/c82a93e6-feb4-4977-889d-175921fd1395-kube-api-access-9k4r2\") pod \"openshift-state-metrics-54cf96dfcf-78wq2\" (UID: \"c82a93e6-feb4-4977-889d-175921fd1395\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-78wq2" Jul 06 20:09:53.630609 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:53.630492 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7226da11-2638-40c4-92fd-d5e2801dc9ac-node-exporter-wtmp\") pod \"node-exporter-v2rzd\" (UID: \"7226da11-2638-40c4-92fd-d5e2801dc9ac\") " pod="openshift-monitoring/node-exporter-v2rzd" Jul 06 20:09:53.630609 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:53.630553 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7226da11-2638-40c4-92fd-d5e2801dc9ac-node-exporter-textfile\") pod \"node-exporter-v2rzd\" (UID: \"7226da11-2638-40c4-92fd-d5e2801dc9ac\") " pod="openshift-monitoring/node-exporter-v2rzd" Jul 06 20:09:53.630609 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:53.630590 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7226da11-2638-40c4-92fd-d5e2801dc9ac-metrics-client-ca\") pod \"node-exporter-v2rzd\" (UID: \"7226da11-2638-40c4-92fd-d5e2801dc9ac\") " pod="openshift-monitoring/node-exporter-v2rzd" Jul 06 20:09:53.630916 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:53.630629 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7226da11-2638-40c4-92fd-d5e2801dc9ac-root\") pod \"node-exporter-v2rzd\" (UID: \"7226da11-2638-40c4-92fd-d5e2801dc9ac\") " pod="openshift-monitoring/node-exporter-v2rzd" Jul 06 20:09:53.630916 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:53.630685 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7226da11-2638-40c4-92fd-d5e2801dc9ac-node-exporter-tls\") pod \"node-exporter-v2rzd\" (UID: \"7226da11-2638-40c4-92fd-d5e2801dc9ac\") " pod="openshift-monitoring/node-exporter-v2rzd" Jul 06 20:09:53.630916 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:53.630704 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7226da11-2638-40c4-92fd-d5e2801dc9ac-node-exporter-wtmp\") pod \"node-exporter-v2rzd\" (UID: \"7226da11-2638-40c4-92fd-d5e2801dc9ac\") " pod="openshift-monitoring/node-exporter-v2rzd" Jul 06 20:09:53.630916 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:53.630748 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7226da11-2638-40c4-92fd-d5e2801dc9ac-root\") pod \"node-exporter-v2rzd\" (UID: \"7226da11-2638-40c4-92fd-d5e2801dc9ac\") " pod="openshift-monitoring/node-exporter-v2rzd" Jul 06 20:09:53.630916 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:53.630770 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7226da11-2638-40c4-92fd-d5e2801dc9ac-node-exporter-accelerators-collector-config\") pod \"node-exporter-v2rzd\" (UID: \"7226da11-2638-40c4-92fd-d5e2801dc9ac\") " pod="openshift-monitoring/node-exporter-v2rzd" Jul 06 20:09:53.630916 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:53.630807 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7226da11-2638-40c4-92fd-d5e2801dc9ac-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-v2rzd\" (UID: \"7226da11-2638-40c4-92fd-d5e2801dc9ac\") " pod="openshift-monitoring/node-exporter-v2rzd" Jul 06 20:09:53.631204 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:53.630932 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxl2v\" (UniqueName: \"kubernetes.io/projected/7226da11-2638-40c4-92fd-d5e2801dc9ac-kube-api-access-nxl2v\") pod \"node-exporter-v2rzd\" (UID: \"7226da11-2638-40c4-92fd-d5e2801dc9ac\") " pod="openshift-monitoring/node-exporter-v2rzd" Jul 06 20:09:53.631204 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:53.630961 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7226da11-2638-40c4-92fd-d5e2801dc9ac-sys\") pod \"node-exporter-v2rzd\" (UID: \"7226da11-2638-40c4-92fd-d5e2801dc9ac\") " pod="openshift-monitoring/node-exporter-v2rzd" Jul 06 20:09:53.631204 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:53.631026 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7226da11-2638-40c4-92fd-d5e2801dc9ac-sys\") pod \"node-exporter-v2rzd\" (UID: \"7226da11-2638-40c4-92fd-d5e2801dc9ac\") " pod="openshift-monitoring/node-exporter-v2rzd" Jul 06 20:09:53.631204 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:53.631138 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7226da11-2638-40c4-92fd-d5e2801dc9ac-node-exporter-textfile\") pod \"node-exporter-v2rzd\" (UID: \"7226da11-2638-40c4-92fd-d5e2801dc9ac\") " pod="openshift-monitoring/node-exporter-v2rzd" Jul 06 20:09:53.631363 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:53.631318 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7226da11-2638-40c4-92fd-d5e2801dc9ac-metrics-client-ca\") pod \"node-exporter-v2rzd\" (UID: \"7226da11-2638-40c4-92fd-d5e2801dc9ac\") " pod="openshift-monitoring/node-exporter-v2rzd" Jul 06 20:09:53.631363 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:53.631327 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7226da11-2638-40c4-92fd-d5e2801dc9ac-node-exporter-accelerators-collector-config\") pod \"node-exporter-v2rzd\" (UID: \"7226da11-2638-40c4-92fd-d5e2801dc9ac\") " pod="openshift-monitoring/node-exporter-v2rzd" Jul 06 20:09:53.633114 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:53.633098 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7226da11-2638-40c4-92fd-d5e2801dc9ac-node-exporter-tls\") pod \"node-exporter-v2rzd\" (UID: \"7226da11-2638-40c4-92fd-d5e2801dc9ac\") " pod="openshift-monitoring/node-exporter-v2rzd" Jul 06 20:09:53.633216 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:53.633198 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7226da11-2638-40c4-92fd-d5e2801dc9ac-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-v2rzd\" (UID: \"7226da11-2638-40c4-92fd-d5e2801dc9ac\") " pod="openshift-monitoring/node-exporter-v2rzd" Jul 06 20:09:53.640407 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:53.640382 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxl2v\" (UniqueName: \"kubernetes.io/projected/7226da11-2638-40c4-92fd-d5e2801dc9ac-kube-api-access-nxl2v\") pod \"node-exporter-v2rzd\" (UID: \"7226da11-2638-40c4-92fd-d5e2801dc9ac\") " pod="openshift-monitoring/node-exporter-v2rzd" Jul 06 20:09:53.697107 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:53.697077 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-v2rzd" Jul 06 20:09:53.707125 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:09:53.707100 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7226da11_2638_40c4_92fd_d5e2801dc9ac.slice/crio-dfecf7e79fa1cf13b3aa3b341d6859d94bc8c57aadaf319f8e63986e4f38fccd WatchSource:0}: Error finding container dfecf7e79fa1cf13b3aa3b341d6859d94bc8c57aadaf319f8e63986e4f38fccd: Status 404 returned error can't find the container with id dfecf7e79fa1cf13b3aa3b341d6859d94bc8c57aadaf319f8e63986e4f38fccd Jul 06 20:09:54.034618 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.034581 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c82a93e6-feb4-4977-889d-175921fd1395-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-78wq2\" (UID: \"c82a93e6-feb4-4977-889d-175921fd1395\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-78wq2" Jul 06 20:09:54.036979 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.036958 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c82a93e6-feb4-4977-889d-175921fd1395-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-78wq2\" (UID: \"c82a93e6-feb4-4977-889d-175921fd1395\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-78wq2" Jul 06 20:09:54.243443 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.243400 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-78wq2" Jul 06 20:09:54.431041 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.431003 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-78wq2"] Jul 06 20:09:54.434283 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:09:54.434257 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc82a93e6_feb4_4977_889d_175921fd1395.slice/crio-5b69fa1ac052647a9b994fdc4edd04daa831785d26768bd1c2938ba09924c456 WatchSource:0}: Error finding container 5b69fa1ac052647a9b994fdc4edd04daa831785d26768bd1c2938ba09924c456: Status 404 returned error can't find the container with id 5b69fa1ac052647a9b994fdc4edd04daa831785d26768bd1c2938ba09924c456 Jul 06 20:09:54.522708 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.522679 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 20:09:54.557326 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.557293 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 20:09:54.557458 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.557434 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:09:54.561418 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.561197 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 06 20:09:54.561418 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.561215 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 06 20:09:54.561418 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.561266 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 06 20:09:54.561418 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.561290 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 06 20:09:54.561418 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.561316 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 06 20:09:54.561418 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.561374 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-ncgw6\"" Jul 06 20:09:54.561779 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.561674 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 06 20:09:54.561779 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.561692 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 06 20:09:54.562266 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.562239 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 06 20:09:54.562382 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.562270 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 06 20:09:54.641020 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.640988 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bf663ce5-e990-4655-ab2b-6d0a5b74b823-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:09:54.641165 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.641060 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bf663ce5-e990-4655-ab2b-6d0a5b74b823-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:09:54.641165 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.641093 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bf663ce5-e990-4655-ab2b-6d0a5b74b823-config-out\") pod \"alertmanager-main-0\" (UID: \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:09:54.641165 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.641123 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bf663ce5-e990-4655-ab2b-6d0a5b74b823-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:09:54.641336 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.641183 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bf663ce5-e990-4655-ab2b-6d0a5b74b823-tls-assets\") pod \"alertmanager-main-0\" (UID: \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:09:54.641336 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.641212 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bf663ce5-e990-4655-ab2b-6d0a5b74b823-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:09:54.641336 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.641261 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bf663ce5-e990-4655-ab2b-6d0a5b74b823-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:09:54.641336 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.641291 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bf663ce5-e990-4655-ab2b-6d0a5b74b823-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:09:54.641336 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.641317 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bf663ce5-e990-4655-ab2b-6d0a5b74b823-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:09:54.641504 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.641369 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bf663ce5-e990-4655-ab2b-6d0a5b74b823-config-volume\") pod \"alertmanager-main-0\" (UID: \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:09:54.641504 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.641396 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf663ce5-e990-4655-ab2b-6d0a5b74b823-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:09:54.641504 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.641428 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62qkx\" (UniqueName: \"kubernetes.io/projected/bf663ce5-e990-4655-ab2b-6d0a5b74b823-kube-api-access-62qkx\") pod \"alertmanager-main-0\" (UID: \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:09:54.641504 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.641453 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bf663ce5-e990-4655-ab2b-6d0a5b74b823-web-config\") pod \"alertmanager-main-0\" (UID: \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:09:54.698737 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.698699 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-78wq2" event={"ID":"c82a93e6-feb4-4977-889d-175921fd1395","Type":"ContainerStarted","Data":"5e4511eb365ce213d5c8e6a5cb5fe4a255d78c7861c4ccd57f7af5ee10504cf9"} Jul 06 20:09:54.699219 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.698763 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-78wq2" event={"ID":"c82a93e6-feb4-4977-889d-175921fd1395","Type":"ContainerStarted","Data":"32317b1f300fd330ba194e3e4b8915cac3b26d6164c1c3eede36f8993f7607f8"} Jul 06 20:09:54.699219 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.698792 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-78wq2" event={"ID":"c82a93e6-feb4-4977-889d-175921fd1395","Type":"ContainerStarted","Data":"5b69fa1ac052647a9b994fdc4edd04daa831785d26768bd1c2938ba09924c456"} Jul 06 20:09:54.699982 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.699958 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-v2rzd" event={"ID":"7226da11-2638-40c4-92fd-d5e2801dc9ac","Type":"ContainerStarted","Data":"dfecf7e79fa1cf13b3aa3b341d6859d94bc8c57aadaf319f8e63986e4f38fccd"} Jul 06 20:09:54.742398 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.742359 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bf663ce5-e990-4655-ab2b-6d0a5b74b823-web-config\") pod \"alertmanager-main-0\" (UID: \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:09:54.742398 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.742402 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bf663ce5-e990-4655-ab2b-6d0a5b74b823-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:09:54.742617 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.742587 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bf663ce5-e990-4655-ab2b-6d0a5b74b823-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:09:54.742654 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.742640 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bf663ce5-e990-4655-ab2b-6d0a5b74b823-config-out\") pod \"alertmanager-main-0\" (UID: \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:09:54.742817 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.742797 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bf663ce5-e990-4655-ab2b-6d0a5b74b823-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:09:54.742881 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.742865 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bf663ce5-e990-4655-ab2b-6d0a5b74b823-tls-assets\") pod \"alertmanager-main-0\" (UID: \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:09:54.742968 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.742917 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bf663ce5-e990-4655-ab2b-6d0a5b74b823-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:09:54.743025 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.742971 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bf663ce5-e990-4655-ab2b-6d0a5b74b823-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:09:54.743025 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.743010 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bf663ce5-e990-4655-ab2b-6d0a5b74b823-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:09:54.743127 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.743045 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bf663ce5-e990-4655-ab2b-6d0a5b74b823-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:09:54.743127 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.743084 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bf663ce5-e990-4655-ab2b-6d0a5b74b823-config-volume\") pod \"alertmanager-main-0\" (UID: \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:09:54.743127 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.743109 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf663ce5-e990-4655-ab2b-6d0a5b74b823-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:09:54.743265 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.743147 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62qkx\" (UniqueName: \"kubernetes.io/projected/bf663ce5-e990-4655-ab2b-6d0a5b74b823-kube-api-access-62qkx\") pod \"alertmanager-main-0\" (UID: \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:09:54.743265 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.743196 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bf663ce5-e990-4655-ab2b-6d0a5b74b823-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:09:54.743491 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.743465 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bf663ce5-e990-4655-ab2b-6d0a5b74b823-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:09:54.744388 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.744358 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf663ce5-e990-4655-ab2b-6d0a5b74b823-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:09:54.745941 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.745911 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bf663ce5-e990-4655-ab2b-6d0a5b74b823-config-out\") pod \"alertmanager-main-0\" (UID: \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:09:54.746027 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.745918 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bf663ce5-e990-4655-ab2b-6d0a5b74b823-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:09:54.746087 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.746065 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bf663ce5-e990-4655-ab2b-6d0a5b74b823-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:09:54.746365 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.746335 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bf663ce5-e990-4655-ab2b-6d0a5b74b823-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:09:54.746445 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.746424 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bf663ce5-e990-4655-ab2b-6d0a5b74b823-tls-assets\") pod \"alertmanager-main-0\" (UID: \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:09:54.746683 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.746659 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bf663ce5-e990-4655-ab2b-6d0a5b74b823-web-config\") pod \"alertmanager-main-0\" (UID: \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:09:54.746997 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.746977 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bf663ce5-e990-4655-ab2b-6d0a5b74b823-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:09:54.747089 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.746980 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bf663ce5-e990-4655-ab2b-6d0a5b74b823-config-volume\") pod \"alertmanager-main-0\" (UID: \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:09:54.747239 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.747220 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bf663ce5-e990-4655-ab2b-6d0a5b74b823-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:09:54.751828 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.751803 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-62qkx\" (UniqueName: \"kubernetes.io/projected/bf663ce5-e990-4655-ab2b-6d0a5b74b823-kube-api-access-62qkx\") pod \"alertmanager-main-0\" (UID: \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:09:54.868340 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:54.868253 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:09:55.162605 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:55.162582 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 20:09:55.164799 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:09:55.164772 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf663ce5_e990_4655_ab2b_6d0a5b74b823.slice/crio-0588f6b468b602a58aa5e3b1f4d990a5046905bce47b4aa41e3afc14b386be30 WatchSource:0}: Error finding container 0588f6b468b602a58aa5e3b1f4d990a5046905bce47b4aa41e3afc14b386be30: Status 404 returned error can't find the container with id 0588f6b468b602a58aa5e3b1f4d990a5046905bce47b4aa41e3afc14b386be30 Jul 06 20:09:55.704513 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:55.704422 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bf663ce5-e990-4655-ab2b-6d0a5b74b823","Type":"ContainerStarted","Data":"0588f6b468b602a58aa5e3b1f4d990a5046905bce47b4aa41e3afc14b386be30"} Jul 06 20:09:55.705985 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:55.705960 2551 generic.go:356] "Generic (PLEG): container finished" podID="7226da11-2638-40c4-92fd-d5e2801dc9ac" containerID="0d085f653e615fe545c86cc43fd5ea7aed58181d3b14eabe37702c76b4f0ed69" exitCode=0 Jul 06 20:09:55.706084 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:55.706013 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-v2rzd" event={"ID":"7226da11-2638-40c4-92fd-d5e2801dc9ac","Type":"ContainerDied","Data":"0d085f653e615fe545c86cc43fd5ea7aed58181d3b14eabe37702c76b4f0ed69"} Jul 06 20:09:56.711027 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:56.710988 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-78wq2" event={"ID":"c82a93e6-feb4-4977-889d-175921fd1395","Type":"ContainerStarted","Data":"0483d86d82265575a5cf52b12bb541538e24d68dc9b785ba1748897cfda8ff01"} Jul 06 20:09:56.712574 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:56.712549 2551 generic.go:356] "Generic (PLEG): container finished" podID="bf663ce5-e990-4655-ab2b-6d0a5b74b823" containerID="f16a4ef976e6b17e9890def02048885d5538a6cf7ca2ab217f11769633de90fa" exitCode=0 Jul 06 20:09:56.712720 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:56.712638 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bf663ce5-e990-4655-ab2b-6d0a5b74b823","Type":"ContainerDied","Data":"f16a4ef976e6b17e9890def02048885d5538a6cf7ca2ab217f11769633de90fa"} Jul 06 20:09:56.714802 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:56.714778 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-v2rzd" event={"ID":"7226da11-2638-40c4-92fd-d5e2801dc9ac","Type":"ContainerStarted","Data":"9fffdcc074ba4421970f219568699085299e37109e9a16e166983287c0b2ac01"} Jul 06 20:09:56.714917 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:56.714820 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-v2rzd" event={"ID":"7226da11-2638-40c4-92fd-d5e2801dc9ac","Type":"ContainerStarted","Data":"7dcc3cbb26e170dab2dae0f5ca91d17d2634401633099b0e71e93d57361ecd8c"} Jul 06 20:09:56.732155 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:56.732058 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-78wq2" podStartSLOduration=2.3755674190000002 podStartE2EDuration="3.732043625s" totalImagesPullingTime="1.356476206s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:09:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:09:56.731030595 +0000 UTC m=+107.146230418" watchObservedRunningTime="2026-07-06 20:09:56.732043625 +0000 UTC m=+107.147243431" Jul 06 20:09:56.748826 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:56.748779 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-v2rzd" podStartSLOduration=1.98289297 podStartE2EDuration="3.748761665s" totalImagesPullingTime="1.656882921s" totalInitContainerRuntime="108.985774ms" isStatefulPod=true podCreationTimestamp="2026-07-06 20:09:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:09:56.748437038 +0000 UTC m=+107.163636834" watchObservedRunningTime="2026-07-06 20:09:56.748761665 +0000 UTC m=+107.163961461" Jul 06 20:09:57.748049 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:57.748013 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5456bc64bb-dlqgd"] Jul 06 20:09:57.751249 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:57.751232 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5456bc64bb-dlqgd" Jul 06 20:09:57.754875 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:57.754840 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-fhce54nuvnhv8\"" Jul 06 20:09:57.755048 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:57.754841 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 06 20:09:57.755048 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:57.754841 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 06 20:09:57.755048 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:57.754851 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 06 20:09:57.755262 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:57.755241 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-f82n5\"" Jul 06 20:09:57.755515 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:57.755501 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 06 20:09:57.763369 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:57.763345 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5456bc64bb-dlqgd"] Jul 06 20:09:57.874407 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:57.874364 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/56e7150a-0f69-48ff-8e0d-0029a85a1341-metrics-server-audit-profiles\") pod \"metrics-server-5456bc64bb-dlqgd\" (UID: \"56e7150a-0f69-48ff-8e0d-0029a85a1341\") " pod="openshift-monitoring/metrics-server-5456bc64bb-dlqgd" Jul 06 20:09:57.874407 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:57.874415 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/56e7150a-0f69-48ff-8e0d-0029a85a1341-client-ca-bundle\") pod \"metrics-server-5456bc64bb-dlqgd\" (UID: \"56e7150a-0f69-48ff-8e0d-0029a85a1341\") " pod="openshift-monitoring/metrics-server-5456bc64bb-dlqgd" Jul 06 20:09:57.874682 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:57.874637 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/56e7150a-0f69-48ff-8e0d-0029a85a1341-secret-metrics-server-client-certs\") pod \"metrics-server-5456bc64bb-dlqgd\" (UID: \"56e7150a-0f69-48ff-8e0d-0029a85a1341\") " pod="openshift-monitoring/metrics-server-5456bc64bb-dlqgd" Jul 06 20:09:57.874764 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:57.874709 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/56e7150a-0f69-48ff-8e0d-0029a85a1341-secret-metrics-server-tls\") pod \"metrics-server-5456bc64bb-dlqgd\" (UID: \"56e7150a-0f69-48ff-8e0d-0029a85a1341\") " pod="openshift-monitoring/metrics-server-5456bc64bb-dlqgd" Jul 06 20:09:57.874764 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:57.874736 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56e7150a-0f69-48ff-8e0d-0029a85a1341-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5456bc64bb-dlqgd\" (UID: \"56e7150a-0f69-48ff-8e0d-0029a85a1341\") " pod="openshift-monitoring/metrics-server-5456bc64bb-dlqgd" Jul 06 20:09:57.874864 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:57.874770 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/56e7150a-0f69-48ff-8e0d-0029a85a1341-audit-log\") pod \"metrics-server-5456bc64bb-dlqgd\" (UID: \"56e7150a-0f69-48ff-8e0d-0029a85a1341\") " pod="openshift-monitoring/metrics-server-5456bc64bb-dlqgd" Jul 06 20:09:57.874927 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:57.874872 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6f2z\" (UniqueName: \"kubernetes.io/projected/56e7150a-0f69-48ff-8e0d-0029a85a1341-kube-api-access-s6f2z\") pod \"metrics-server-5456bc64bb-dlqgd\" (UID: \"56e7150a-0f69-48ff-8e0d-0029a85a1341\") " pod="openshift-monitoring/metrics-server-5456bc64bb-dlqgd" Jul 06 20:09:57.976264 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:57.976220 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56e7150a-0f69-48ff-8e0d-0029a85a1341-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5456bc64bb-dlqgd\" (UID: \"56e7150a-0f69-48ff-8e0d-0029a85a1341\") " pod="openshift-monitoring/metrics-server-5456bc64bb-dlqgd" Jul 06 20:09:57.976264 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:57.976268 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/56e7150a-0f69-48ff-8e0d-0029a85a1341-audit-log\") pod \"metrics-server-5456bc64bb-dlqgd\" (UID: \"56e7150a-0f69-48ff-8e0d-0029a85a1341\") " pod="openshift-monitoring/metrics-server-5456bc64bb-dlqgd" Jul 06 20:09:57.976511 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:57.976299 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6f2z\" (UniqueName: \"kubernetes.io/projected/56e7150a-0f69-48ff-8e0d-0029a85a1341-kube-api-access-s6f2z\") pod \"metrics-server-5456bc64bb-dlqgd\" (UID: \"56e7150a-0f69-48ff-8e0d-0029a85a1341\") " pod="openshift-monitoring/metrics-server-5456bc64bb-dlqgd" Jul 06 20:09:57.976511 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:57.976369 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/56e7150a-0f69-48ff-8e0d-0029a85a1341-metrics-server-audit-profiles\") pod \"metrics-server-5456bc64bb-dlqgd\" (UID: \"56e7150a-0f69-48ff-8e0d-0029a85a1341\") " pod="openshift-monitoring/metrics-server-5456bc64bb-dlqgd" Jul 06 20:09:57.976511 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:57.976407 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/56e7150a-0f69-48ff-8e0d-0029a85a1341-client-ca-bundle\") pod \"metrics-server-5456bc64bb-dlqgd\" (UID: \"56e7150a-0f69-48ff-8e0d-0029a85a1341\") " pod="openshift-monitoring/metrics-server-5456bc64bb-dlqgd" Jul 06 20:09:57.976511 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:57.976457 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/56e7150a-0f69-48ff-8e0d-0029a85a1341-secret-metrics-server-client-certs\") pod \"metrics-server-5456bc64bb-dlqgd\" (UID: \"56e7150a-0f69-48ff-8e0d-0029a85a1341\") " pod="openshift-monitoring/metrics-server-5456bc64bb-dlqgd" Jul 06 20:09:57.976511 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:57.976502 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/56e7150a-0f69-48ff-8e0d-0029a85a1341-secret-metrics-server-tls\") pod \"metrics-server-5456bc64bb-dlqgd\" (UID: \"56e7150a-0f69-48ff-8e0d-0029a85a1341\") " pod="openshift-monitoring/metrics-server-5456bc64bb-dlqgd" Jul 06 20:09:57.976865 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:57.976835 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/56e7150a-0f69-48ff-8e0d-0029a85a1341-audit-log\") pod \"metrics-server-5456bc64bb-dlqgd\" (UID: \"56e7150a-0f69-48ff-8e0d-0029a85a1341\") " pod="openshift-monitoring/metrics-server-5456bc64bb-dlqgd" Jul 06 20:09:57.977112 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:57.977091 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56e7150a-0f69-48ff-8e0d-0029a85a1341-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5456bc64bb-dlqgd\" (UID: \"56e7150a-0f69-48ff-8e0d-0029a85a1341\") " pod="openshift-monitoring/metrics-server-5456bc64bb-dlqgd" Jul 06 20:09:57.977379 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:57.977355 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/56e7150a-0f69-48ff-8e0d-0029a85a1341-metrics-server-audit-profiles\") pod \"metrics-server-5456bc64bb-dlqgd\" (UID: \"56e7150a-0f69-48ff-8e0d-0029a85a1341\") " pod="openshift-monitoring/metrics-server-5456bc64bb-dlqgd" Jul 06 20:09:57.979142 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:57.979117 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/56e7150a-0f69-48ff-8e0d-0029a85a1341-secret-metrics-server-tls\") pod \"metrics-server-5456bc64bb-dlqgd\" (UID: \"56e7150a-0f69-48ff-8e0d-0029a85a1341\") " pod="openshift-monitoring/metrics-server-5456bc64bb-dlqgd" Jul 06 20:09:57.979222 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:57.979159 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/56e7150a-0f69-48ff-8e0d-0029a85a1341-secret-metrics-server-client-certs\") pod \"metrics-server-5456bc64bb-dlqgd\" (UID: \"56e7150a-0f69-48ff-8e0d-0029a85a1341\") " pod="openshift-monitoring/metrics-server-5456bc64bb-dlqgd" Jul 06 20:09:57.979222 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:57.979193 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/56e7150a-0f69-48ff-8e0d-0029a85a1341-client-ca-bundle\") pod \"metrics-server-5456bc64bb-dlqgd\" (UID: \"56e7150a-0f69-48ff-8e0d-0029a85a1341\") " pod="openshift-monitoring/metrics-server-5456bc64bb-dlqgd" Jul 06 20:09:57.985014 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:57.984990 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6f2z\" (UniqueName: \"kubernetes.io/projected/56e7150a-0f69-48ff-8e0d-0029a85a1341-kube-api-access-s6f2z\") pod \"metrics-server-5456bc64bb-dlqgd\" (UID: \"56e7150a-0f69-48ff-8e0d-0029a85a1341\") " pod="openshift-monitoring/metrics-server-5456bc64bb-dlqgd" Jul 06 20:09:58.062299 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:58.062261 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5456bc64bb-dlqgd" Jul 06 20:09:58.202391 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:58.199057 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5456bc64bb-dlqgd"] Jul 06 20:09:58.722944 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:58.722886 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5456bc64bb-dlqgd" event={"ID":"56e7150a-0f69-48ff-8e0d-0029a85a1341","Type":"ContainerStarted","Data":"294981d68dbd3cddc52ec7993c412fc99bd92cc53a46dc509da61489b5b9dafc"} Jul 06 20:09:58.724671 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:58.724644 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bf663ce5-e990-4655-ab2b-6d0a5b74b823","Type":"ContainerStarted","Data":"4e02ee3488139856db6c896d2292cb453e6899667a992ac3f8ce400a4af21e4a"} Jul 06 20:09:59.731349 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:59.731315 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bf663ce5-e990-4655-ab2b-6d0a5b74b823","Type":"ContainerStarted","Data":"5da85b720296b56b44b79ccb7c2d32e965319a6f022bc14ae1d7dfbdd5def71e"} Jul 06 20:09:59.731780 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:59.731381 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bf663ce5-e990-4655-ab2b-6d0a5b74b823","Type":"ContainerStarted","Data":"47bf2103793b3208a750e9cddebc9dec4d7ae2905c4443e365d54421cd4fc44c"} Jul 06 20:09:59.731780 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:59.731406 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bf663ce5-e990-4655-ab2b-6d0a5b74b823","Type":"ContainerStarted","Data":"62dba82b522b6c17ed1c9fb975964f0222bd4665657d842e3f6e549401a0680c"} Jul 06 20:09:59.731780 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:09:59.731430 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bf663ce5-e990-4655-ab2b-6d0a5b74b823","Type":"ContainerStarted","Data":"419c750c0fe3ac72e93e004d8499597adb8960ac1c21684bb8690ea39471a6b8"} Jul 06 20:10:00.739264 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:00.739218 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5456bc64bb-dlqgd" event={"ID":"56e7150a-0f69-48ff-8e0d-0029a85a1341","Type":"ContainerStarted","Data":"6db5f8da0a9fbf3c27a2325f35e23dbe488a8aa9c87f17b7279135d5092f24f9"} Jul 06 20:10:00.742457 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:00.742432 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bf663ce5-e990-4655-ab2b-6d0a5b74b823","Type":"ContainerStarted","Data":"3a7492a348a5d4db60bc96685eceb757e2f2915f1524d74a8e537f435f65dc9b"} Jul 06 20:10:00.763300 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:00.763249 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5456bc64bb-dlqgd" podStartSLOduration=2.068355375 podStartE2EDuration="3.763233476s" totalImagesPullingTime="1.694878101s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:09:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:10:00.761106899 +0000 UTC m=+111.176306719" watchObservedRunningTime="2026-07-06 20:10:00.763233476 +0000 UTC m=+111.178433281" Jul 06 20:10:00.790946 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:00.790864 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.629301871 podStartE2EDuration="6.790848027s" totalImagesPullingTime="4.141090491s" totalInitContainerRuntime="20.455665ms" isStatefulPod=false podCreationTimestamp="2026-07-06 20:09:54 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:10:00.78858502 +0000 UTC m=+111.203784849" watchObservedRunningTime="2026-07-06 20:10:00.790848027 +0000 UTC m=+111.206047831" Jul 06 20:10:02.696924 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:02.696857 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-qwn7v" Jul 06 20:10:07.670993 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:07.670962 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-94665fb6-4xkqt" Jul 06 20:10:08.325418 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:08.325380 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-684b859d-dwnm2"] Jul 06 20:10:08.329932 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:08.329914 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-684b859d-dwnm2" Jul 06 20:10:08.332502 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:08.332481 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-jnph4\"" Jul 06 20:10:08.332625 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:08.332570 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 06 20:10:08.333562 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:08.333544 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 06 20:10:08.333653 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:08.333601 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 06 20:10:08.333774 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:08.333757 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 06 20:10:08.333848 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:08.333808 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 06 20:10:08.333946 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:08.333929 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 06 20:10:08.333991 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:08.333966 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 06 20:10:08.338911 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:08.338872 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 06 20:10:08.339070 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:08.339050 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-684b859d-dwnm2"] Jul 06 20:10:08.363392 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:08.363356 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5a824c13-6774-47e3-aacf-9396ae146422-oauth-serving-cert\") pod \"console-684b859d-dwnm2\" (UID: \"5a824c13-6774-47e3-aacf-9396ae146422\") " pod="openshift-console/console-684b859d-dwnm2" Jul 06 20:10:08.363392 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:08.363393 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5a824c13-6774-47e3-aacf-9396ae146422-service-ca\") pod \"console-684b859d-dwnm2\" (UID: \"5a824c13-6774-47e3-aacf-9396ae146422\") " pod="openshift-console/console-684b859d-dwnm2" Jul 06 20:10:08.363609 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:08.363415 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a824c13-6774-47e3-aacf-9396ae146422-trusted-ca-bundle\") pod \"console-684b859d-dwnm2\" (UID: \"5a824c13-6774-47e3-aacf-9396ae146422\") " pod="openshift-console/console-684b859d-dwnm2" Jul 06 20:10:08.363609 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:08.363531 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5a824c13-6774-47e3-aacf-9396ae146422-console-serving-cert\") pod \"console-684b859d-dwnm2\" (UID: \"5a824c13-6774-47e3-aacf-9396ae146422\") " pod="openshift-console/console-684b859d-dwnm2" Jul 06 20:10:08.363609 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:08.363571 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5a824c13-6774-47e3-aacf-9396ae146422-console-config\") pod \"console-684b859d-dwnm2\" (UID: \"5a824c13-6774-47e3-aacf-9396ae146422\") " pod="openshift-console/console-684b859d-dwnm2" Jul 06 20:10:08.363758 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:08.363675 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lz4qz\" (UniqueName: \"kubernetes.io/projected/5a824c13-6774-47e3-aacf-9396ae146422-kube-api-access-lz4qz\") pod \"console-684b859d-dwnm2\" (UID: \"5a824c13-6774-47e3-aacf-9396ae146422\") " pod="openshift-console/console-684b859d-dwnm2" Jul 06 20:10:08.363758 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:08.363724 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5a824c13-6774-47e3-aacf-9396ae146422-console-oauth-config\") pod \"console-684b859d-dwnm2\" (UID: \"5a824c13-6774-47e3-aacf-9396ae146422\") " pod="openshift-console/console-684b859d-dwnm2" Jul 06 20:10:08.464636 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:08.464599 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5a824c13-6774-47e3-aacf-9396ae146422-console-serving-cert\") pod \"console-684b859d-dwnm2\" (UID: \"5a824c13-6774-47e3-aacf-9396ae146422\") " pod="openshift-console/console-684b859d-dwnm2" Jul 06 20:10:08.464836 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:08.464655 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5a824c13-6774-47e3-aacf-9396ae146422-console-config\") pod \"console-684b859d-dwnm2\" (UID: \"5a824c13-6774-47e3-aacf-9396ae146422\") " pod="openshift-console/console-684b859d-dwnm2" Jul 06 20:10:08.464836 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:08.464701 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lz4qz\" (UniqueName: \"kubernetes.io/projected/5a824c13-6774-47e3-aacf-9396ae146422-kube-api-access-lz4qz\") pod \"console-684b859d-dwnm2\" (UID: \"5a824c13-6774-47e3-aacf-9396ae146422\") " pod="openshift-console/console-684b859d-dwnm2" Jul 06 20:10:08.464836 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:08.464735 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5a824c13-6774-47e3-aacf-9396ae146422-console-oauth-config\") pod \"console-684b859d-dwnm2\" (UID: \"5a824c13-6774-47e3-aacf-9396ae146422\") " pod="openshift-console/console-684b859d-dwnm2" Jul 06 20:10:08.464836 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:08.464771 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5a824c13-6774-47e3-aacf-9396ae146422-oauth-serving-cert\") pod \"console-684b859d-dwnm2\" (UID: \"5a824c13-6774-47e3-aacf-9396ae146422\") " pod="openshift-console/console-684b859d-dwnm2" Jul 06 20:10:08.465076 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:08.464921 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5a824c13-6774-47e3-aacf-9396ae146422-service-ca\") pod \"console-684b859d-dwnm2\" (UID: \"5a824c13-6774-47e3-aacf-9396ae146422\") " pod="openshift-console/console-684b859d-dwnm2" Jul 06 20:10:08.465076 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:08.464963 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a824c13-6774-47e3-aacf-9396ae146422-trusted-ca-bundle\") pod \"console-684b859d-dwnm2\" (UID: \"5a824c13-6774-47e3-aacf-9396ae146422\") " pod="openshift-console/console-684b859d-dwnm2" Jul 06 20:10:08.465600 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:08.465574 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5a824c13-6774-47e3-aacf-9396ae146422-console-config\") pod \"console-684b859d-dwnm2\" (UID: \"5a824c13-6774-47e3-aacf-9396ae146422\") " pod="openshift-console/console-684b859d-dwnm2" Jul 06 20:10:08.465715 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:08.465634 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5a824c13-6774-47e3-aacf-9396ae146422-oauth-serving-cert\") pod \"console-684b859d-dwnm2\" (UID: \"5a824c13-6774-47e3-aacf-9396ae146422\") " pod="openshift-console/console-684b859d-dwnm2" Jul 06 20:10:08.465715 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:08.465676 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5a824c13-6774-47e3-aacf-9396ae146422-service-ca\") pod \"console-684b859d-dwnm2\" (UID: \"5a824c13-6774-47e3-aacf-9396ae146422\") " pod="openshift-console/console-684b859d-dwnm2" Jul 06 20:10:08.465825 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:08.465809 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a824c13-6774-47e3-aacf-9396ae146422-trusted-ca-bundle\") pod \"console-684b859d-dwnm2\" (UID: \"5a824c13-6774-47e3-aacf-9396ae146422\") " pod="openshift-console/console-684b859d-dwnm2" Jul 06 20:10:08.467252 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:08.467230 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5a824c13-6774-47e3-aacf-9396ae146422-console-oauth-config\") pod \"console-684b859d-dwnm2\" (UID: \"5a824c13-6774-47e3-aacf-9396ae146422\") " pod="openshift-console/console-684b859d-dwnm2" Jul 06 20:10:08.467380 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:08.467361 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5a824c13-6774-47e3-aacf-9396ae146422-console-serving-cert\") pod \"console-684b859d-dwnm2\" (UID: \"5a824c13-6774-47e3-aacf-9396ae146422\") " pod="openshift-console/console-684b859d-dwnm2" Jul 06 20:10:08.473513 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:08.473491 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lz4qz\" (UniqueName: \"kubernetes.io/projected/5a824c13-6774-47e3-aacf-9396ae146422-kube-api-access-lz4qz\") pod \"console-684b859d-dwnm2\" (UID: \"5a824c13-6774-47e3-aacf-9396ae146422\") " pod="openshift-console/console-684b859d-dwnm2" Jul 06 20:10:08.640156 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:08.640070 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-684b859d-dwnm2" Jul 06 20:10:08.778596 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:08.778566 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-684b859d-dwnm2"] Jul 06 20:10:08.781804 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:10:08.781770 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a824c13_6774_47e3_aacf_9396ae146422.slice/crio-c0222a415302faa7a61e5eb31e78d819cd58e50bcc4532d66b1cfe3f1c227f55 WatchSource:0}: Error finding container c0222a415302faa7a61e5eb31e78d819cd58e50bcc4532d66b1cfe3f1c227f55: Status 404 returned error can't find the container with id c0222a415302faa7a61e5eb31e78d819cd58e50bcc4532d66b1cfe3f1c227f55 Jul 06 20:10:09.776316 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:09.776282 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-684b859d-dwnm2" event={"ID":"5a824c13-6774-47e3-aacf-9396ae146422","Type":"ContainerStarted","Data":"c0222a415302faa7a61e5eb31e78d819cd58e50bcc4532d66b1cfe3f1c227f55"} Jul 06 20:10:12.789560 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:12.789525 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-684b859d-dwnm2" event={"ID":"5a824c13-6774-47e3-aacf-9396ae146422","Type":"ContainerStarted","Data":"fe2a8f13533cc628055532ac63cd4d3bf7315b4ac777a4b3addd862d98cc0af0"} Jul 06 20:10:12.809621 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:12.809572 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-684b859d-dwnm2" podStartSLOduration=1.8013819199999999 podStartE2EDuration="4.809556798s" totalImagesPullingTime="3.008174878s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:10:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:10:12.808026026 +0000 UTC m=+123.223225850" watchObservedRunningTime="2026-07-06 20:10:12.809556798 +0000 UTC m=+123.224756602" Jul 06 20:10:13.798837 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:13.796161 2551 generic.go:356] "Generic (PLEG): container finished" podID="e54c25e9-3542-48e1-9c6c-de6bd9b3f3d7" containerID="d7f4fdcafee992bdec3c12d09bdb3efe1895fcf8db48d0b09c8ccb3a2d5cae1e" exitCode=0 Jul 06 20:10:13.798837 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:13.797153 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-jxbvp" event={"ID":"e54c25e9-3542-48e1-9c6c-de6bd9b3f3d7","Type":"ContainerDied","Data":"d7f4fdcafee992bdec3c12d09bdb3efe1895fcf8db48d0b09c8ccb3a2d5cae1e"} Jul 06 20:10:13.798837 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:13.797617 2551 scope.go:117] "RemoveContainer" containerID="d7f4fdcafee992bdec3c12d09bdb3efe1895fcf8db48d0b09c8ccb3a2d5cae1e" Jul 06 20:10:14.801578 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:14.801543 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-jxbvp" event={"ID":"e54c25e9-3542-48e1-9c6c-de6bd9b3f3d7","Type":"ContainerStarted","Data":"d3daf14260c031367e251ba19dc6b5f8c3ecdf681e2fbb2524eda0b979fa5251"} Jul 06 20:10:15.398379 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:15.398352 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bf663ce5-e990-4655-ab2b-6d0a5b74b823/init-config-reloader/0.log" Jul 06 20:10:15.649094 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:15.649004 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bf663ce5-e990-4655-ab2b-6d0a5b74b823/alertmanager/0.log" Jul 06 20:10:15.798356 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:15.798323 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bf663ce5-e990-4655-ab2b-6d0a5b74b823/config-reloader/0.log" Jul 06 20:10:15.998707 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:15.998614 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bf663ce5-e990-4655-ab2b-6d0a5b74b823/kube-rbac-proxy-web/0.log" Jul 06 20:10:16.197969 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:16.197941 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bf663ce5-e990-4655-ab2b-6d0a5b74b823/kube-rbac-proxy/0.log" Jul 06 20:10:16.398360 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:16.398324 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bf663ce5-e990-4655-ab2b-6d0a5b74b823/kube-rbac-proxy-metric/0.log" Jul 06 20:10:16.597776 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:16.597745 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bf663ce5-e990-4655-ab2b-6d0a5b74b823/prom-label-proxy/0.log" Jul 06 20:10:17.599737 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:17.599710 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5456bc64bb-dlqgd_56e7150a-0f69-48ff-8e0d-0029a85a1341/metrics-server/0.log" Jul 06 20:10:18.062344 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:18.062304 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5456bc64bb-dlqgd" Jul 06 20:10:18.062549 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:18.062376 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5456bc64bb-dlqgd" Jul 06 20:10:18.640984 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:18.640949 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-684b859d-dwnm2" Jul 06 20:10:18.641433 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:18.641029 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-684b859d-dwnm2" Jul 06 20:10:18.646128 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:18.646104 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-684b859d-dwnm2" Jul 06 20:10:18.820960 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:18.820934 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-684b859d-dwnm2" Jul 06 20:10:19.201839 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:19.201810 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-v2rzd_7226da11-2638-40c4-92fd-d5e2801dc9ac/init-textfile/0.log" Jul 06 20:10:19.403643 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:19.403613 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-v2rzd_7226da11-2638-40c4-92fd-d5e2801dc9ac/node-exporter/0.log" Jul 06 20:10:19.600791 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:19.600763 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-v2rzd_7226da11-2638-40c4-92fd-d5e2801dc9ac/kube-rbac-proxy/0.log" Jul 06 20:10:19.798841 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:19.798811 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-78wq2_c82a93e6-feb4-4977-889d-175921fd1395/kube-rbac-proxy-main/0.log" Jul 06 20:10:19.869472 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:19.869364 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef16cb88-f1e2-49c7-a470-47a23730758d-metrics-certs\") pod \"network-metrics-daemon-v648r\" (UID: \"ef16cb88-f1e2-49c7-a470-47a23730758d\") " pod="openshift-multus/network-metrics-daemon-v648r" Jul 06 20:10:19.871714 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:19.871683 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef16cb88-f1e2-49c7-a470-47a23730758d-metrics-certs\") pod \"network-metrics-daemon-v648r\" (UID: \"ef16cb88-f1e2-49c7-a470-47a23730758d\") " pod="openshift-multus/network-metrics-daemon-v648r" Jul 06 20:10:19.984555 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:19.984523 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7lmbq\"" Jul 06 20:10:19.990973 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:19.990946 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v648r" Jul 06 20:10:20.003742 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:20.003715 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-78wq2_c82a93e6-feb4-4977-889d-175921fd1395/kube-rbac-proxy-self/0.log" Jul 06 20:10:20.145835 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:20.145810 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-v648r"] Jul 06 20:10:20.148104 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:10:20.148069 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef16cb88_f1e2_49c7_a470_47a23730758d.slice/crio-f9da05bad5ef56598c65432cb3e620746b6eee1cce02e6ee81f7adca38ca711e WatchSource:0}: Error finding container f9da05bad5ef56598c65432cb3e620746b6eee1cce02e6ee81f7adca38ca711e: Status 404 returned error can't find the container with id f9da05bad5ef56598c65432cb3e620746b6eee1cce02e6ee81f7adca38ca711e Jul 06 20:10:20.199917 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:20.199861 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-78wq2_c82a93e6-feb4-4977-889d-175921fd1395/openshift-state-metrics/0.log" Jul 06 20:10:20.826064 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:20.826027 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-v648r" event={"ID":"ef16cb88-f1e2-49c7-a470-47a23730758d","Type":"ContainerStarted","Data":"f9da05bad5ef56598c65432cb3e620746b6eee1cce02e6ee81f7adca38ca711e"} Jul 06 20:10:21.831474 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:21.831431 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-v648r" event={"ID":"ef16cb88-f1e2-49c7-a470-47a23730758d","Type":"ContainerStarted","Data":"1d4cd64679e861a272cafaae996ab99275148492926390b82eda7675b4ba5536"} Jul 06 20:10:21.832003 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:21.831525 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-v648r" event={"ID":"ef16cb88-f1e2-49c7-a470-47a23730758d","Type":"ContainerStarted","Data":"d7cdb44cd6db37cae68cce6c778b42f97867c8be6b618217a1ca8c29dc6d84b9"} Jul 06 20:10:21.851746 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:21.851691 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-v648r" podStartSLOduration=130.923823919 podStartE2EDuration="2m11.851675776s" totalImagesPullingTime="927.851857ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:08:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:10:21.850509956 +0000 UTC m=+132.265709762" watchObservedRunningTime="2026-07-06 20:10:21.851675776 +0000 UTC m=+132.266875620" Jul 06 20:10:24.405255 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:24.405222 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-684b859d-dwnm2_5a824c13-6774-47e3-aacf-9396ae146422/console/0.log" Jul 06 20:10:24.798341 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:24.798305 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-668d9f84f5-rrtln_735538e5-306c-44a6-a934-30f463187f08/router/0.log" Jul 06 20:10:25.399539 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:25.399511 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-z67pq_cf4f10b3-8b8d-45c2-9f64-8a37aa591b68/serve-healthcheck-canary/0.log" Jul 06 20:10:31.876861 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:31.876823 2551 generic.go:356] "Generic (PLEG): container finished" podID="2c866a8a-ebee-4296-a983-42c059034c9a" containerID="25d5d3077be85267fb09c8d9d17d46100e0502ce2df7e16ed4f40afb18442a4c" exitCode=0 Jul 06 20:10:31.877355 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:31.876918 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-p52gc" event={"ID":"2c866a8a-ebee-4296-a983-42c059034c9a","Type":"ContainerDied","Data":"25d5d3077be85267fb09c8d9d17d46100e0502ce2df7e16ed4f40afb18442a4c"} Jul 06 20:10:31.877355 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:31.877303 2551 scope.go:117] "RemoveContainer" containerID="25d5d3077be85267fb09c8d9d17d46100e0502ce2df7e16ed4f40afb18442a4c" Jul 06 20:10:32.882967 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:32.882933 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-p52gc" event={"ID":"2c866a8a-ebee-4296-a983-42c059034c9a","Type":"ContainerStarted","Data":"52ddb4dcc2f15bd45fe017f5c2bb7804a54cbe6397b2b4bfb4241f52c5f9fbbd"} Jul 06 20:10:38.067755 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:38.067707 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5456bc64bb-dlqgd" Jul 06 20:10:38.071694 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:10:38.071670 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5456bc64bb-dlqgd" Jul 06 20:11:13.735586 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:13.735552 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 20:11:13.736140 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:13.735824 2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="bf663ce5-e990-4655-ab2b-6d0a5b74b823" containerName="alertmanager" containerID="cri-o://4e02ee3488139856db6c896d2292cb453e6899667a992ac3f8ce400a4af21e4a" gracePeriod=120 Jul 06 20:11:13.736140 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:13.735916 2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="bf663ce5-e990-4655-ab2b-6d0a5b74b823" containerName="kube-rbac-proxy-metric" containerID="cri-o://5da85b720296b56b44b79ccb7c2d32e965319a6f022bc14ae1d7dfbdd5def71e" gracePeriod=120 Jul 06 20:11:13.736140 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:13.735960 2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="bf663ce5-e990-4655-ab2b-6d0a5b74b823" containerName="kube-rbac-proxy-web" containerID="cri-o://62dba82b522b6c17ed1c9fb975964f0222bd4665657d842e3f6e549401a0680c" gracePeriod=120 Jul 06 20:11:13.736140 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:13.735973 2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="bf663ce5-e990-4655-ab2b-6d0a5b74b823" containerName="prom-label-proxy" containerID="cri-o://3a7492a348a5d4db60bc96685eceb757e2f2915f1524d74a8e537f435f65dc9b" gracePeriod=120 Jul 06 20:11:13.736140 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:13.736026 2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="bf663ce5-e990-4655-ab2b-6d0a5b74b823" containerName="kube-rbac-proxy" containerID="cri-o://47bf2103793b3208a750e9cddebc9dec4d7ae2905c4443e365d54421cd4fc44c" gracePeriod=120 Jul 06 20:11:13.736140 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:13.736095 2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="bf663ce5-e990-4655-ab2b-6d0a5b74b823" containerName="config-reloader" containerID="cri-o://419c750c0fe3ac72e93e004d8499597adb8960ac1c21684bb8690ea39471a6b8" gracePeriod=120 Jul 06 20:11:14.044536 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:14.044497 2551 generic.go:356] "Generic (PLEG): container finished" podID="bf663ce5-e990-4655-ab2b-6d0a5b74b823" containerID="3a7492a348a5d4db60bc96685eceb757e2f2915f1524d74a8e537f435f65dc9b" exitCode=0 Jul 06 20:11:14.044536 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:14.044521 2551 generic.go:356] "Generic (PLEG): container finished" podID="bf663ce5-e990-4655-ab2b-6d0a5b74b823" containerID="47bf2103793b3208a750e9cddebc9dec4d7ae2905c4443e365d54421cd4fc44c" exitCode=0 Jul 06 20:11:14.044536 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:14.044530 2551 generic.go:356] "Generic (PLEG): container finished" podID="bf663ce5-e990-4655-ab2b-6d0a5b74b823" containerID="419c750c0fe3ac72e93e004d8499597adb8960ac1c21684bb8690ea39471a6b8" exitCode=0 Jul 06 20:11:14.044536 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:14.044538 2551 generic.go:356] "Generic (PLEG): container finished" podID="bf663ce5-e990-4655-ab2b-6d0a5b74b823" containerID="4e02ee3488139856db6c896d2292cb453e6899667a992ac3f8ce400a4af21e4a" exitCode=0 Jul 06 20:11:14.044806 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:14.044568 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bf663ce5-e990-4655-ab2b-6d0a5b74b823","Type":"ContainerDied","Data":"3a7492a348a5d4db60bc96685eceb757e2f2915f1524d74a8e537f435f65dc9b"} Jul 06 20:11:14.044806 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:14.044631 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bf663ce5-e990-4655-ab2b-6d0a5b74b823","Type":"ContainerDied","Data":"47bf2103793b3208a750e9cddebc9dec4d7ae2905c4443e365d54421cd4fc44c"} Jul 06 20:11:14.044806 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:14.044657 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bf663ce5-e990-4655-ab2b-6d0a5b74b823","Type":"ContainerDied","Data":"419c750c0fe3ac72e93e004d8499597adb8960ac1c21684bb8690ea39471a6b8"} Jul 06 20:11:14.044806 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:14.044683 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bf663ce5-e990-4655-ab2b-6d0a5b74b823","Type":"ContainerDied","Data":"4e02ee3488139856db6c896d2292cb453e6899667a992ac3f8ce400a4af21e4a"} Jul 06 20:11:14.985582 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:14.985557 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:11:15.052790 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.052706 2551 generic.go:356] "Generic (PLEG): container finished" podID="bf663ce5-e990-4655-ab2b-6d0a5b74b823" containerID="5da85b720296b56b44b79ccb7c2d32e965319a6f022bc14ae1d7dfbdd5def71e" exitCode=0 Jul 06 20:11:15.052790 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.052734 2551 generic.go:356] "Generic (PLEG): container finished" podID="bf663ce5-e990-4655-ab2b-6d0a5b74b823" containerID="62dba82b522b6c17ed1c9fb975964f0222bd4665657d842e3f6e549401a0680c" exitCode=0 Jul 06 20:11:15.053023 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.052783 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bf663ce5-e990-4655-ab2b-6d0a5b74b823","Type":"ContainerDied","Data":"5da85b720296b56b44b79ccb7c2d32e965319a6f022bc14ae1d7dfbdd5def71e"} Jul 06 20:11:15.053023 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.052825 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:11:15.053023 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.052857 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bf663ce5-e990-4655-ab2b-6d0a5b74b823","Type":"ContainerDied","Data":"62dba82b522b6c17ed1c9fb975964f0222bd4665657d842e3f6e549401a0680c"} Jul 06 20:11:15.053023 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.052886 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bf663ce5-e990-4655-ab2b-6d0a5b74b823","Type":"ContainerDied","Data":"0588f6b468b602a58aa5e3b1f4d990a5046905bce47b4aa41e3afc14b386be30"} Jul 06 20:11:15.053023 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.052943 2551 scope.go:117] "RemoveContainer" containerID="3a7492a348a5d4db60bc96685eceb757e2f2915f1524d74a8e537f435f65dc9b" Jul 06 20:11:15.060812 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.060792 2551 scope.go:117] "RemoveContainer" containerID="5da85b720296b56b44b79ccb7c2d32e965319a6f022bc14ae1d7dfbdd5def71e" Jul 06 20:11:15.069945 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.069926 2551 scope.go:117] "RemoveContainer" containerID="47bf2103793b3208a750e9cddebc9dec4d7ae2905c4443e365d54421cd4fc44c" Jul 06 20:11:15.076636 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.076617 2551 scope.go:117] "RemoveContainer" containerID="62dba82b522b6c17ed1c9fb975964f0222bd4665657d842e3f6e549401a0680c" Jul 06 20:11:15.083292 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.083268 2551 scope.go:117] "RemoveContainer" containerID="419c750c0fe3ac72e93e004d8499597adb8960ac1c21684bb8690ea39471a6b8" Jul 06 20:11:15.090057 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.090039 2551 scope.go:117] "RemoveContainer" containerID="4e02ee3488139856db6c896d2292cb453e6899667a992ac3f8ce400a4af21e4a" Jul 06 20:11:15.097058 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.097042 2551 scope.go:117] "RemoveContainer" containerID="f16a4ef976e6b17e9890def02048885d5538a6cf7ca2ab217f11769633de90fa" Jul 06 20:11:15.103748 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.103733 2551 scope.go:117] "RemoveContainer" containerID="3a7492a348a5d4db60bc96685eceb757e2f2915f1524d74a8e537f435f65dc9b" Jul 06 20:11:15.104029 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:11:15.104008 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a7492a348a5d4db60bc96685eceb757e2f2915f1524d74a8e537f435f65dc9b\": container with ID starting with 3a7492a348a5d4db60bc96685eceb757e2f2915f1524d74a8e537f435f65dc9b not found: ID does not exist" containerID="3a7492a348a5d4db60bc96685eceb757e2f2915f1524d74a8e537f435f65dc9b" Jul 06 20:11:15.104080 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.104039 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a7492a348a5d4db60bc96685eceb757e2f2915f1524d74a8e537f435f65dc9b"} err="failed to get container status \"3a7492a348a5d4db60bc96685eceb757e2f2915f1524d74a8e537f435f65dc9b\": rpc error: code = NotFound desc = could not find container \"3a7492a348a5d4db60bc96685eceb757e2f2915f1524d74a8e537f435f65dc9b\": container with ID starting with 3a7492a348a5d4db60bc96685eceb757e2f2915f1524d74a8e537f435f65dc9b not found: ID does not exist" Jul 06 20:11:15.104080 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.104074 2551 scope.go:117] "RemoveContainer" containerID="5da85b720296b56b44b79ccb7c2d32e965319a6f022bc14ae1d7dfbdd5def71e" Jul 06 20:11:15.104317 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:11:15.104300 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5da85b720296b56b44b79ccb7c2d32e965319a6f022bc14ae1d7dfbdd5def71e\": container with ID starting with 5da85b720296b56b44b79ccb7c2d32e965319a6f022bc14ae1d7dfbdd5def71e not found: ID does not exist" containerID="5da85b720296b56b44b79ccb7c2d32e965319a6f022bc14ae1d7dfbdd5def71e" Jul 06 20:11:15.104380 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.104326 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5da85b720296b56b44b79ccb7c2d32e965319a6f022bc14ae1d7dfbdd5def71e"} err="failed to get container status \"5da85b720296b56b44b79ccb7c2d32e965319a6f022bc14ae1d7dfbdd5def71e\": rpc error: code = NotFound desc = could not find container \"5da85b720296b56b44b79ccb7c2d32e965319a6f022bc14ae1d7dfbdd5def71e\": container with ID starting with 5da85b720296b56b44b79ccb7c2d32e965319a6f022bc14ae1d7dfbdd5def71e not found: ID does not exist" Jul 06 20:11:15.104380 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.104347 2551 scope.go:117] "RemoveContainer" containerID="47bf2103793b3208a750e9cddebc9dec4d7ae2905c4443e365d54421cd4fc44c" Jul 06 20:11:15.104592 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:11:15.104577 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47bf2103793b3208a750e9cddebc9dec4d7ae2905c4443e365d54421cd4fc44c\": container with ID starting with 47bf2103793b3208a750e9cddebc9dec4d7ae2905c4443e365d54421cd4fc44c not found: ID does not exist" containerID="47bf2103793b3208a750e9cddebc9dec4d7ae2905c4443e365d54421cd4fc44c" Jul 06 20:11:15.104648 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.104595 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47bf2103793b3208a750e9cddebc9dec4d7ae2905c4443e365d54421cd4fc44c"} err="failed to get container status \"47bf2103793b3208a750e9cddebc9dec4d7ae2905c4443e365d54421cd4fc44c\": rpc error: code = NotFound desc = could not find container \"47bf2103793b3208a750e9cddebc9dec4d7ae2905c4443e365d54421cd4fc44c\": container with ID starting with 47bf2103793b3208a750e9cddebc9dec4d7ae2905c4443e365d54421cd4fc44c not found: ID does not exist" Jul 06 20:11:15.104648 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.104607 2551 scope.go:117] "RemoveContainer" containerID="62dba82b522b6c17ed1c9fb975964f0222bd4665657d842e3f6e549401a0680c" Jul 06 20:11:15.104818 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:11:15.104802 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62dba82b522b6c17ed1c9fb975964f0222bd4665657d842e3f6e549401a0680c\": container with ID starting with 62dba82b522b6c17ed1c9fb975964f0222bd4665657d842e3f6e549401a0680c not found: ID does not exist" containerID="62dba82b522b6c17ed1c9fb975964f0222bd4665657d842e3f6e549401a0680c" Jul 06 20:11:15.104860 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.104823 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62dba82b522b6c17ed1c9fb975964f0222bd4665657d842e3f6e549401a0680c"} err="failed to get container status \"62dba82b522b6c17ed1c9fb975964f0222bd4665657d842e3f6e549401a0680c\": rpc error: code = NotFound desc = could not find container \"62dba82b522b6c17ed1c9fb975964f0222bd4665657d842e3f6e549401a0680c\": container with ID starting with 62dba82b522b6c17ed1c9fb975964f0222bd4665657d842e3f6e549401a0680c not found: ID does not exist" Jul 06 20:11:15.104860 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.104837 2551 scope.go:117] "RemoveContainer" containerID="419c750c0fe3ac72e93e004d8499597adb8960ac1c21684bb8690ea39471a6b8" Jul 06 20:11:15.105147 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:11:15.105132 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"419c750c0fe3ac72e93e004d8499597adb8960ac1c21684bb8690ea39471a6b8\": container with ID starting with 419c750c0fe3ac72e93e004d8499597adb8960ac1c21684bb8690ea39471a6b8 not found: ID does not exist" containerID="419c750c0fe3ac72e93e004d8499597adb8960ac1c21684bb8690ea39471a6b8" Jul 06 20:11:15.105186 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.105152 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"419c750c0fe3ac72e93e004d8499597adb8960ac1c21684bb8690ea39471a6b8"} err="failed to get container status \"419c750c0fe3ac72e93e004d8499597adb8960ac1c21684bb8690ea39471a6b8\": rpc error: code = NotFound desc = could not find container \"419c750c0fe3ac72e93e004d8499597adb8960ac1c21684bb8690ea39471a6b8\": container with ID starting with 419c750c0fe3ac72e93e004d8499597adb8960ac1c21684bb8690ea39471a6b8 not found: ID does not exist" Jul 06 20:11:15.105186 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.105165 2551 scope.go:117] "RemoveContainer" containerID="4e02ee3488139856db6c896d2292cb453e6899667a992ac3f8ce400a4af21e4a" Jul 06 20:11:15.105379 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:11:15.105364 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e02ee3488139856db6c896d2292cb453e6899667a992ac3f8ce400a4af21e4a\": container with ID starting with 4e02ee3488139856db6c896d2292cb453e6899667a992ac3f8ce400a4af21e4a not found: ID does not exist" containerID="4e02ee3488139856db6c896d2292cb453e6899667a992ac3f8ce400a4af21e4a" Jul 06 20:11:15.105419 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.105382 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e02ee3488139856db6c896d2292cb453e6899667a992ac3f8ce400a4af21e4a"} err="failed to get container status \"4e02ee3488139856db6c896d2292cb453e6899667a992ac3f8ce400a4af21e4a\": rpc error: code = NotFound desc = could not find container \"4e02ee3488139856db6c896d2292cb453e6899667a992ac3f8ce400a4af21e4a\": container with ID starting with 4e02ee3488139856db6c896d2292cb453e6899667a992ac3f8ce400a4af21e4a not found: ID does not exist" Jul 06 20:11:15.105419 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.105397 2551 scope.go:117] "RemoveContainer" containerID="f16a4ef976e6b17e9890def02048885d5538a6cf7ca2ab217f11769633de90fa" Jul 06 20:11:15.105623 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:11:15.105608 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f16a4ef976e6b17e9890def02048885d5538a6cf7ca2ab217f11769633de90fa\": container with ID starting with f16a4ef976e6b17e9890def02048885d5538a6cf7ca2ab217f11769633de90fa not found: ID does not exist" containerID="f16a4ef976e6b17e9890def02048885d5538a6cf7ca2ab217f11769633de90fa" Jul 06 20:11:15.105669 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.105625 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f16a4ef976e6b17e9890def02048885d5538a6cf7ca2ab217f11769633de90fa"} err="failed to get container status \"f16a4ef976e6b17e9890def02048885d5538a6cf7ca2ab217f11769633de90fa\": rpc error: code = NotFound desc = could not find container \"f16a4ef976e6b17e9890def02048885d5538a6cf7ca2ab217f11769633de90fa\": container with ID starting with f16a4ef976e6b17e9890def02048885d5538a6cf7ca2ab217f11769633de90fa not found: ID does not exist" Jul 06 20:11:15.105669 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.105637 2551 scope.go:117] "RemoveContainer" containerID="3a7492a348a5d4db60bc96685eceb757e2f2915f1524d74a8e537f435f65dc9b" Jul 06 20:11:15.105822 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.105806 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a7492a348a5d4db60bc96685eceb757e2f2915f1524d74a8e537f435f65dc9b"} err="failed to get container status \"3a7492a348a5d4db60bc96685eceb757e2f2915f1524d74a8e537f435f65dc9b\": rpc error: code = NotFound desc = could not find container \"3a7492a348a5d4db60bc96685eceb757e2f2915f1524d74a8e537f435f65dc9b\": container with ID starting with 3a7492a348a5d4db60bc96685eceb757e2f2915f1524d74a8e537f435f65dc9b not found: ID does not exist" Jul 06 20:11:15.105861 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.105824 2551 scope.go:117] "RemoveContainer" containerID="5da85b720296b56b44b79ccb7c2d32e965319a6f022bc14ae1d7dfbdd5def71e" Jul 06 20:11:15.111057 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.111024 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5da85b720296b56b44b79ccb7c2d32e965319a6f022bc14ae1d7dfbdd5def71e"} err="failed to get container status \"5da85b720296b56b44b79ccb7c2d32e965319a6f022bc14ae1d7dfbdd5def71e\": rpc error: code = NotFound desc = could not find container \"5da85b720296b56b44b79ccb7c2d32e965319a6f022bc14ae1d7dfbdd5def71e\": container with ID starting with 5da85b720296b56b44b79ccb7c2d32e965319a6f022bc14ae1d7dfbdd5def71e not found: ID does not exist" Jul 06 20:11:15.111057 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.111056 2551 scope.go:117] "RemoveContainer" containerID="47bf2103793b3208a750e9cddebc9dec4d7ae2905c4443e365d54421cd4fc44c" Jul 06 20:11:15.111344 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.111324 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47bf2103793b3208a750e9cddebc9dec4d7ae2905c4443e365d54421cd4fc44c"} err="failed to get container status \"47bf2103793b3208a750e9cddebc9dec4d7ae2905c4443e365d54421cd4fc44c\": rpc error: code = NotFound desc = could not find container \"47bf2103793b3208a750e9cddebc9dec4d7ae2905c4443e365d54421cd4fc44c\": container with ID starting with 47bf2103793b3208a750e9cddebc9dec4d7ae2905c4443e365d54421cd4fc44c not found: ID does not exist" Jul 06 20:11:15.111406 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.111345 2551 scope.go:117] "RemoveContainer" containerID="62dba82b522b6c17ed1c9fb975964f0222bd4665657d842e3f6e549401a0680c" Jul 06 20:11:15.111589 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.111566 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62dba82b522b6c17ed1c9fb975964f0222bd4665657d842e3f6e549401a0680c"} err="failed to get container status \"62dba82b522b6c17ed1c9fb975964f0222bd4665657d842e3f6e549401a0680c\": rpc error: code = NotFound desc = could not find container \"62dba82b522b6c17ed1c9fb975964f0222bd4665657d842e3f6e549401a0680c\": container with ID starting with 62dba82b522b6c17ed1c9fb975964f0222bd4665657d842e3f6e549401a0680c not found: ID does not exist" Jul 06 20:11:15.111644 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.111590 2551 scope.go:117] "RemoveContainer" containerID="419c750c0fe3ac72e93e004d8499597adb8960ac1c21684bb8690ea39471a6b8" Jul 06 20:11:15.111826 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.111808 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"419c750c0fe3ac72e93e004d8499597adb8960ac1c21684bb8690ea39471a6b8"} err="failed to get container status \"419c750c0fe3ac72e93e004d8499597adb8960ac1c21684bb8690ea39471a6b8\": rpc error: code = NotFound desc = could not find container \"419c750c0fe3ac72e93e004d8499597adb8960ac1c21684bb8690ea39471a6b8\": container with ID starting with 419c750c0fe3ac72e93e004d8499597adb8960ac1c21684bb8690ea39471a6b8 not found: ID does not exist" Jul 06 20:11:15.111878 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.111826 2551 scope.go:117] "RemoveContainer" containerID="4e02ee3488139856db6c896d2292cb453e6899667a992ac3f8ce400a4af21e4a" Jul 06 20:11:15.112053 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.112034 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e02ee3488139856db6c896d2292cb453e6899667a992ac3f8ce400a4af21e4a"} err="failed to get container status \"4e02ee3488139856db6c896d2292cb453e6899667a992ac3f8ce400a4af21e4a\": rpc error: code = NotFound desc = could not find container \"4e02ee3488139856db6c896d2292cb453e6899667a992ac3f8ce400a4af21e4a\": container with ID starting with 4e02ee3488139856db6c896d2292cb453e6899667a992ac3f8ce400a4af21e4a not found: ID does not exist" Jul 06 20:11:15.112124 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.112054 2551 scope.go:117] "RemoveContainer" containerID="f16a4ef976e6b17e9890def02048885d5538a6cf7ca2ab217f11769633de90fa" Jul 06 20:11:15.112273 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.112255 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f16a4ef976e6b17e9890def02048885d5538a6cf7ca2ab217f11769633de90fa"} err="failed to get container status \"f16a4ef976e6b17e9890def02048885d5538a6cf7ca2ab217f11769633de90fa\": rpc error: code = NotFound desc = could not find container \"f16a4ef976e6b17e9890def02048885d5538a6cf7ca2ab217f11769633de90fa\": container with ID starting with f16a4ef976e6b17e9890def02048885d5538a6cf7ca2ab217f11769633de90fa not found: ID does not exist" Jul 06 20:11:15.143292 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.143257 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bf663ce5-e990-4655-ab2b-6d0a5b74b823-config-volume\") pod \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\" (UID: \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\") " Jul 06 20:11:15.143445 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.143306 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bf663ce5-e990-4655-ab2b-6d0a5b74b823-tls-assets\") pod \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\" (UID: \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\") " Jul 06 20:11:15.143445 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.143334 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf663ce5-e990-4655-ab2b-6d0a5b74b823-alertmanager-trusted-ca-bundle\") pod \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\" (UID: \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\") " Jul 06 20:11:15.143445 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.143359 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bf663ce5-e990-4655-ab2b-6d0a5b74b823-secret-alertmanager-kube-rbac-proxy\") pod \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\" (UID: \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\") " Jul 06 20:11:15.143445 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.143381 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bf663ce5-e990-4655-ab2b-6d0a5b74b823-secret-alertmanager-main-tls\") pod \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\" (UID: \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\") " Jul 06 20:11:15.143445 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.143407 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-62qkx\" (UniqueName: \"kubernetes.io/projected/bf663ce5-e990-4655-ab2b-6d0a5b74b823-kube-api-access-62qkx\") pod \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\" (UID: \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\") " Jul 06 20:11:15.143445 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.143427 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bf663ce5-e990-4655-ab2b-6d0a5b74b823-secret-alertmanager-kube-rbac-proxy-metric\") pod \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\" (UID: \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\") " Jul 06 20:11:15.143732 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.143466 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bf663ce5-e990-4655-ab2b-6d0a5b74b823-alertmanager-main-db\") pod \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\" (UID: \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\") " Jul 06 20:11:15.143732 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.143512 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bf663ce5-e990-4655-ab2b-6d0a5b74b823-cluster-tls-config\") pod \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\" (UID: \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\") " Jul 06 20:11:15.143732 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.143544 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bf663ce5-e990-4655-ab2b-6d0a5b74b823-config-out\") pod \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\" (UID: \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\") " Jul 06 20:11:15.143732 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.143567 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bf663ce5-e990-4655-ab2b-6d0a5b74b823-web-config\") pod \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\" (UID: \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\") " Jul 06 20:11:15.143732 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.143604 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bf663ce5-e990-4655-ab2b-6d0a5b74b823-secret-alertmanager-kube-rbac-proxy-web\") pod \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\" (UID: \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\") " Jul 06 20:11:15.143732 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.143649 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bf663ce5-e990-4655-ab2b-6d0a5b74b823-metrics-client-ca\") pod \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\" (UID: \"bf663ce5-e990-4655-ab2b-6d0a5b74b823\") " Jul 06 20:11:15.144077 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.143825 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf663ce5-e990-4655-ab2b-6d0a5b74b823-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "bf663ce5-e990-4655-ab2b-6d0a5b74b823" (UID: "bf663ce5-e990-4655-ab2b-6d0a5b74b823"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:11:15.144077 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.143993 2551 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf663ce5-e990-4655-ab2b-6d0a5b74b823-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:11:15.144283 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.144247 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf663ce5-e990-4655-ab2b-6d0a5b74b823-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "bf663ce5-e990-4655-ab2b-6d0a5b74b823" (UID: "bf663ce5-e990-4655-ab2b-6d0a5b74b823"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:11:15.146335 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.146301 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf663ce5-e990-4655-ab2b-6d0a5b74b823-kube-api-access-62qkx" (OuterVolumeSpecName: "kube-api-access-62qkx") pod "bf663ce5-e990-4655-ab2b-6d0a5b74b823" (UID: "bf663ce5-e990-4655-ab2b-6d0a5b74b823"). InnerVolumeSpecName "kube-api-access-62qkx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:11:15.146448 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.146342 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf663ce5-e990-4655-ab2b-6d0a5b74b823-config-out" (OuterVolumeSpecName: "config-out") pod "bf663ce5-e990-4655-ab2b-6d0a5b74b823" (UID: "bf663ce5-e990-4655-ab2b-6d0a5b74b823"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:11:15.146448 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.146348 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf663ce5-e990-4655-ab2b-6d0a5b74b823-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "bf663ce5-e990-4655-ab2b-6d0a5b74b823" (UID: "bf663ce5-e990-4655-ab2b-6d0a5b74b823"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 20:11:15.146448 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.146381 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf663ce5-e990-4655-ab2b-6d0a5b74b823-config-volume" (OuterVolumeSpecName: "config-volume") pod "bf663ce5-e990-4655-ab2b-6d0a5b74b823" (UID: "bf663ce5-e990-4655-ab2b-6d0a5b74b823"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 20:11:15.146715 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.146691 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf663ce5-e990-4655-ab2b-6d0a5b74b823-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "bf663ce5-e990-4655-ab2b-6d0a5b74b823" (UID: "bf663ce5-e990-4655-ab2b-6d0a5b74b823"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:11:15.146956 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.146914 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf663ce5-e990-4655-ab2b-6d0a5b74b823-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "bf663ce5-e990-4655-ab2b-6d0a5b74b823" (UID: "bf663ce5-e990-4655-ab2b-6d0a5b74b823"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:11:15.147168 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.147139 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf663ce5-e990-4655-ab2b-6d0a5b74b823-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "bf663ce5-e990-4655-ab2b-6d0a5b74b823" (UID: "bf663ce5-e990-4655-ab2b-6d0a5b74b823"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 20:11:15.147238 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.147222 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf663ce5-e990-4655-ab2b-6d0a5b74b823-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "bf663ce5-e990-4655-ab2b-6d0a5b74b823" (UID: "bf663ce5-e990-4655-ab2b-6d0a5b74b823"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 20:11:15.148514 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.148492 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf663ce5-e990-4655-ab2b-6d0a5b74b823-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "bf663ce5-e990-4655-ab2b-6d0a5b74b823" (UID: "bf663ce5-e990-4655-ab2b-6d0a5b74b823"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 20:11:15.151328 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.151292 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf663ce5-e990-4655-ab2b-6d0a5b74b823-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "bf663ce5-e990-4655-ab2b-6d0a5b74b823" (UID: "bf663ce5-e990-4655-ab2b-6d0a5b74b823"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 20:11:15.156280 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.156253 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf663ce5-e990-4655-ab2b-6d0a5b74b823-web-config" (OuterVolumeSpecName: "web-config") pod "bf663ce5-e990-4655-ab2b-6d0a5b74b823" (UID: "bf663ce5-e990-4655-ab2b-6d0a5b74b823"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 20:11:15.245248 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.245210 2551 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bf663ce5-e990-4655-ab2b-6d0a5b74b823-alertmanager-main-db\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:11:15.245248 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.245244 2551 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bf663ce5-e990-4655-ab2b-6d0a5b74b823-cluster-tls-config\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:11:15.245248 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.245258 2551 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bf663ce5-e990-4655-ab2b-6d0a5b74b823-config-out\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:11:15.245496 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.245269 2551 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bf663ce5-e990-4655-ab2b-6d0a5b74b823-web-config\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:11:15.245496 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.245283 2551 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bf663ce5-e990-4655-ab2b-6d0a5b74b823-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:11:15.245496 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.245297 2551 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bf663ce5-e990-4655-ab2b-6d0a5b74b823-metrics-client-ca\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:11:15.245496 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.245309 2551 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bf663ce5-e990-4655-ab2b-6d0a5b74b823-config-volume\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:11:15.245496 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.245323 2551 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bf663ce5-e990-4655-ab2b-6d0a5b74b823-tls-assets\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:11:15.245496 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.245334 2551 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bf663ce5-e990-4655-ab2b-6d0a5b74b823-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:11:15.245496 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.245346 2551 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bf663ce5-e990-4655-ab2b-6d0a5b74b823-secret-alertmanager-main-tls\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:11:15.245496 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.245358 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-62qkx\" (UniqueName: \"kubernetes.io/projected/bf663ce5-e990-4655-ab2b-6d0a5b74b823-kube-api-access-62qkx\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:11:15.245496 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.245375 2551 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bf663ce5-e990-4655-ab2b-6d0a5b74b823-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:11:15.387076 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.387037 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 20:11:15.394128 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.394098 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 20:11:15.433289 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.433258 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 20:11:15.433550 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.433536 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf663ce5-e990-4655-ab2b-6d0a5b74b823" containerName="kube-rbac-proxy-web" Jul 06 20:11:15.433588 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.433551 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf663ce5-e990-4655-ab2b-6d0a5b74b823" containerName="kube-rbac-proxy-web" Jul 06 20:11:15.433588 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.433561 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf663ce5-e990-4655-ab2b-6d0a5b74b823" containerName="prom-label-proxy" Jul 06 20:11:15.433588 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.433567 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf663ce5-e990-4655-ab2b-6d0a5b74b823" containerName="prom-label-proxy" Jul 06 20:11:15.433588 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.433573 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf663ce5-e990-4655-ab2b-6d0a5b74b823" containerName="config-reloader" Jul 06 20:11:15.433588 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.433578 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf663ce5-e990-4655-ab2b-6d0a5b74b823" containerName="config-reloader" Jul 06 20:11:15.433588 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.433585 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf663ce5-e990-4655-ab2b-6d0a5b74b823" containerName="kube-rbac-proxy" Jul 06 20:11:15.433588 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.433590 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf663ce5-e990-4655-ab2b-6d0a5b74b823" containerName="kube-rbac-proxy" Jul 06 20:11:15.433784 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.433605 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf663ce5-e990-4655-ab2b-6d0a5b74b823" containerName="init-config-reloader" Jul 06 20:11:15.433784 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.433610 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf663ce5-e990-4655-ab2b-6d0a5b74b823" containerName="init-config-reloader" Jul 06 20:11:15.433784 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.433617 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf663ce5-e990-4655-ab2b-6d0a5b74b823" containerName="kube-rbac-proxy-metric" Jul 06 20:11:15.433784 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.433621 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf663ce5-e990-4655-ab2b-6d0a5b74b823" containerName="kube-rbac-proxy-metric" Jul 06 20:11:15.433784 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.433628 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf663ce5-e990-4655-ab2b-6d0a5b74b823" containerName="alertmanager" Jul 06 20:11:15.433784 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.433633 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf663ce5-e990-4655-ab2b-6d0a5b74b823" containerName="alertmanager" Jul 06 20:11:15.433784 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.433679 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf663ce5-e990-4655-ab2b-6d0a5b74b823" containerName="kube-rbac-proxy-web" Jul 06 20:11:15.433784 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.433685 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf663ce5-e990-4655-ab2b-6d0a5b74b823" containerName="kube-rbac-proxy" Jul 06 20:11:15.433784 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.433691 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf663ce5-e990-4655-ab2b-6d0a5b74b823" containerName="prom-label-proxy" Jul 06 20:11:15.433784 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.433697 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf663ce5-e990-4655-ab2b-6d0a5b74b823" containerName="config-reloader" Jul 06 20:11:15.433784 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.433704 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf663ce5-e990-4655-ab2b-6d0a5b74b823" containerName="alertmanager" Jul 06 20:11:15.433784 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.433710 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf663ce5-e990-4655-ab2b-6d0a5b74b823" containerName="kube-rbac-proxy-metric" Jul 06 20:11:15.438970 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.438950 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:11:15.441669 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.441651 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 06 20:11:15.441920 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.441885 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 06 20:11:15.441985 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.441968 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 06 20:11:15.442241 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.442210 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 06 20:11:15.442241 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.442238 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 06 20:11:15.442416 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.442352 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 06 20:11:15.442838 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.442820 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 06 20:11:15.442965 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.442953 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 06 20:11:15.443163 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.443150 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-ncgw6\"" Jul 06 20:11:15.447846 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.447824 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 06 20:11:15.454061 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.454040 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 20:11:15.547503 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.547461 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6af805a7-e59a-4c68-8b0c-39625fbba718-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6af805a7-e59a-4c68-8b0c-39625fbba718\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:11:15.547728 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.547522 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6af805a7-e59a-4c68-8b0c-39625fbba718-config-out\") pod \"alertmanager-main-0\" (UID: \"6af805a7-e59a-4c68-8b0c-39625fbba718\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:11:15.547728 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.547550 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6af805a7-e59a-4c68-8b0c-39625fbba718-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6af805a7-e59a-4c68-8b0c-39625fbba718\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:11:15.547728 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.547582 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6af805a7-e59a-4c68-8b0c-39625fbba718-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6af805a7-e59a-4c68-8b0c-39625fbba718\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:11:15.547728 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.547607 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ght42\" (UniqueName: \"kubernetes.io/projected/6af805a7-e59a-4c68-8b0c-39625fbba718-kube-api-access-ght42\") pod \"alertmanager-main-0\" (UID: \"6af805a7-e59a-4c68-8b0c-39625fbba718\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:11:15.547728 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.547650 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6af805a7-e59a-4c68-8b0c-39625fbba718-web-config\") pod \"alertmanager-main-0\" (UID: \"6af805a7-e59a-4c68-8b0c-39625fbba718\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:11:15.547728 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.547697 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6af805a7-e59a-4c68-8b0c-39625fbba718-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6af805a7-e59a-4c68-8b0c-39625fbba718\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:11:15.547728 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.547724 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6af805a7-e59a-4c68-8b0c-39625fbba718-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6af805a7-e59a-4c68-8b0c-39625fbba718\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:11:15.548070 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.547757 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6af805a7-e59a-4c68-8b0c-39625fbba718-config-volume\") pod \"alertmanager-main-0\" (UID: \"6af805a7-e59a-4c68-8b0c-39625fbba718\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:11:15.548070 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.547781 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6af805a7-e59a-4c68-8b0c-39625fbba718-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6af805a7-e59a-4c68-8b0c-39625fbba718\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:11:15.548070 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.547813 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6af805a7-e59a-4c68-8b0c-39625fbba718-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6af805a7-e59a-4c68-8b0c-39625fbba718\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:11:15.548070 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.547851 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6af805a7-e59a-4c68-8b0c-39625fbba718-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6af805a7-e59a-4c68-8b0c-39625fbba718\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:11:15.548070 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.547886 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6af805a7-e59a-4c68-8b0c-39625fbba718-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6af805a7-e59a-4c68-8b0c-39625fbba718\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:11:15.648885 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.648785 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6af805a7-e59a-4c68-8b0c-39625fbba718-config-out\") pod \"alertmanager-main-0\" (UID: \"6af805a7-e59a-4c68-8b0c-39625fbba718\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:11:15.648885 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.648833 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6af805a7-e59a-4c68-8b0c-39625fbba718-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6af805a7-e59a-4c68-8b0c-39625fbba718\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:11:15.648885 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.648862 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6af805a7-e59a-4c68-8b0c-39625fbba718-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6af805a7-e59a-4c68-8b0c-39625fbba718\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:11:15.649183 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.648889 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ght42\" (UniqueName: \"kubernetes.io/projected/6af805a7-e59a-4c68-8b0c-39625fbba718-kube-api-access-ght42\") pod \"alertmanager-main-0\" (UID: \"6af805a7-e59a-4c68-8b0c-39625fbba718\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:11:15.649183 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.648950 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6af805a7-e59a-4c68-8b0c-39625fbba718-web-config\") pod \"alertmanager-main-0\" (UID: \"6af805a7-e59a-4c68-8b0c-39625fbba718\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:11:15.649183 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.648992 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6af805a7-e59a-4c68-8b0c-39625fbba718-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6af805a7-e59a-4c68-8b0c-39625fbba718\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:11:15.649183 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.649025 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6af805a7-e59a-4c68-8b0c-39625fbba718-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6af805a7-e59a-4c68-8b0c-39625fbba718\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:11:15.649183 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.649060 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6af805a7-e59a-4c68-8b0c-39625fbba718-config-volume\") pod \"alertmanager-main-0\" (UID: \"6af805a7-e59a-4c68-8b0c-39625fbba718\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:11:15.649183 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.649081 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6af805a7-e59a-4c68-8b0c-39625fbba718-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6af805a7-e59a-4c68-8b0c-39625fbba718\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:11:15.649183 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.649120 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6af805a7-e59a-4c68-8b0c-39625fbba718-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6af805a7-e59a-4c68-8b0c-39625fbba718\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:11:15.649183 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.649154 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6af805a7-e59a-4c68-8b0c-39625fbba718-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6af805a7-e59a-4c68-8b0c-39625fbba718\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:11:15.649560 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.649193 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6af805a7-e59a-4c68-8b0c-39625fbba718-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6af805a7-e59a-4c68-8b0c-39625fbba718\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:11:15.649560 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.649235 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6af805a7-e59a-4c68-8b0c-39625fbba718-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6af805a7-e59a-4c68-8b0c-39625fbba718\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:11:15.649675 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.649574 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6af805a7-e59a-4c68-8b0c-39625fbba718-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6af805a7-e59a-4c68-8b0c-39625fbba718\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:11:15.649825 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.649801 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6af805a7-e59a-4c68-8b0c-39625fbba718-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6af805a7-e59a-4c68-8b0c-39625fbba718\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:11:15.649962 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.649858 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6af805a7-e59a-4c68-8b0c-39625fbba718-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6af805a7-e59a-4c68-8b0c-39625fbba718\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:11:15.651790 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.651765 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6af805a7-e59a-4c68-8b0c-39625fbba718-config-out\") pod \"alertmanager-main-0\" (UID: \"6af805a7-e59a-4c68-8b0c-39625fbba718\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:11:15.652228 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.652042 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6af805a7-e59a-4c68-8b0c-39625fbba718-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6af805a7-e59a-4c68-8b0c-39625fbba718\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:11:15.652228 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.652119 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6af805a7-e59a-4c68-8b0c-39625fbba718-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6af805a7-e59a-4c68-8b0c-39625fbba718\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:11:15.652228 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.652177 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6af805a7-e59a-4c68-8b0c-39625fbba718-config-volume\") pod \"alertmanager-main-0\" (UID: \"6af805a7-e59a-4c68-8b0c-39625fbba718\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:11:15.652465 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.652443 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6af805a7-e59a-4c68-8b0c-39625fbba718-web-config\") pod \"alertmanager-main-0\" (UID: \"6af805a7-e59a-4c68-8b0c-39625fbba718\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:11:15.652518 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.652477 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6af805a7-e59a-4c68-8b0c-39625fbba718-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6af805a7-e59a-4c68-8b0c-39625fbba718\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:11:15.652733 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.652713 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6af805a7-e59a-4c68-8b0c-39625fbba718-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6af805a7-e59a-4c68-8b0c-39625fbba718\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:11:15.652768 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.652728 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6af805a7-e59a-4c68-8b0c-39625fbba718-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6af805a7-e59a-4c68-8b0c-39625fbba718\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:11:15.653694 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.653678 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6af805a7-e59a-4c68-8b0c-39625fbba718-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6af805a7-e59a-4c68-8b0c-39625fbba718\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:11:15.657977 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.657957 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ght42\" (UniqueName: \"kubernetes.io/projected/6af805a7-e59a-4c68-8b0c-39625fbba718-kube-api-access-ght42\") pod \"alertmanager-main-0\" (UID: \"6af805a7-e59a-4c68-8b0c-39625fbba718\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:11:15.748885 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.748855 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:11:15.899586 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:15.899506 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 20:11:15.902668 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:11:15.902628 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6af805a7_e59a_4c68_8b0c_39625fbba718.slice/crio-ca27d443a0873b399e027fe2aeea2b5d0ecfc4496bf3151bd5dd7e73888182b9 WatchSource:0}: Error finding container ca27d443a0873b399e027fe2aeea2b5d0ecfc4496bf3151bd5dd7e73888182b9: Status 404 returned error can't find the container with id ca27d443a0873b399e027fe2aeea2b5d0ecfc4496bf3151bd5dd7e73888182b9 Jul 06 20:11:16.058779 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:16.058742 2551 generic.go:356] "Generic (PLEG): container finished" podID="6af805a7-e59a-4c68-8b0c-39625fbba718" containerID="dcc9def2a7aa768f38bfa78795e250aff08aa08a44caf837319cd1b4969fed6a" exitCode=0 Jul 06 20:11:16.059301 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:16.058784 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6af805a7-e59a-4c68-8b0c-39625fbba718","Type":"ContainerDied","Data":"dcc9def2a7aa768f38bfa78795e250aff08aa08a44caf837319cd1b4969fed6a"} Jul 06 20:11:16.059301 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:16.058850 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6af805a7-e59a-4c68-8b0c-39625fbba718","Type":"ContainerStarted","Data":"ca27d443a0873b399e027fe2aeea2b5d0ecfc4496bf3151bd5dd7e73888182b9"} Jul 06 20:11:16.163762 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:16.163728 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf663ce5-e990-4655-ab2b-6d0a5b74b823" path="/var/lib/kubelet/pods/bf663ce5-e990-4655-ab2b-6d0a5b74b823/volumes" Jul 06 20:11:17.066289 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:17.066194 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6af805a7-e59a-4c68-8b0c-39625fbba718","Type":"ContainerStarted","Data":"a38a162e2179c9f688c2f84a2870ed777a94e26602e0ce59ffca17f8e7e7d179"} Jul 06 20:11:17.066289 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:17.066272 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6af805a7-e59a-4c68-8b0c-39625fbba718","Type":"ContainerStarted","Data":"e20be74ec7dbd2879eafe236b3de9db606ea34f4c458f699088faf14a4a6f4ff"} Jul 06 20:11:17.066799 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:17.066311 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6af805a7-e59a-4c68-8b0c-39625fbba718","Type":"ContainerStarted","Data":"76bbeb34e95c6f66e72cd8aa2aa0bbc10f309c76a697196a8c548e1d8168af6b"} Jul 06 20:11:17.066799 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:17.066347 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6af805a7-e59a-4c68-8b0c-39625fbba718","Type":"ContainerStarted","Data":"528b482ca3d1e273254fe407e9bd998ef5db9243491299753e4781fe006a781d"} Jul 06 20:11:17.066799 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:17.066384 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6af805a7-e59a-4c68-8b0c-39625fbba718","Type":"ContainerStarted","Data":"745e5ac14b344fe0187451a8750cc9db5f46f4e10082e2cd5c32d37bf71d374d"} Jul 06 20:11:17.066799 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:17.066425 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6af805a7-e59a-4c68-8b0c-39625fbba718","Type":"ContainerStarted","Data":"1f5819aa014971047055b8e3618ce75de1f7515a84c17dec55889b5a34653c25"} Jul 06 20:11:17.100696 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:17.100566 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.081516344 podStartE2EDuration="2.100536018s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.019674ms" isStatefulPod=false podCreationTimestamp="2026-07-06 20:11:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:11:17.095809154 +0000 UTC m=+187.511008994" watchObservedRunningTime="2026-07-06 20:11:17.100536018 +0000 UTC m=+187.515735818" Jul 06 20:11:17.799307 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:17.799268 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7b95cf59d8-42hsw"] Jul 06 20:11:17.802699 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:17.802682 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7b95cf59d8-42hsw" Jul 06 20:11:17.805126 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:17.805100 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 06 20:11:17.805302 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:17.805284 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 06 20:11:17.805419 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:17.805287 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 06 20:11:17.805419 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:17.805321 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-kwgcq\"" Jul 06 20:11:17.805588 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:17.805572 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 06 20:11:17.806066 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:17.806050 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 06 20:11:17.812374 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:17.812352 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 06 20:11:17.815401 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:17.815380 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7b95cf59d8-42hsw"] Jul 06 20:11:17.870124 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:17.870090 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a1ae0621-1054-4d52-83d9-e3cb51bc509a-serving-certs-ca-bundle\") pod \"telemeter-client-7b95cf59d8-42hsw\" (UID: \"a1ae0621-1054-4d52-83d9-e3cb51bc509a\") " pod="openshift-monitoring/telemeter-client-7b95cf59d8-42hsw" Jul 06 20:11:17.870124 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:17.870138 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a1ae0621-1054-4d52-83d9-e3cb51bc509a-metrics-client-ca\") pod \"telemeter-client-7b95cf59d8-42hsw\" (UID: \"a1ae0621-1054-4d52-83d9-e3cb51bc509a\") " pod="openshift-monitoring/telemeter-client-7b95cf59d8-42hsw" Jul 06 20:11:17.870365 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:17.870160 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a1ae0621-1054-4d52-83d9-e3cb51bc509a-secret-telemeter-client\") pod \"telemeter-client-7b95cf59d8-42hsw\" (UID: \"a1ae0621-1054-4d52-83d9-e3cb51bc509a\") " pod="openshift-monitoring/telemeter-client-7b95cf59d8-42hsw" Jul 06 20:11:17.870365 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:17.870183 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gngqz\" (UniqueName: \"kubernetes.io/projected/a1ae0621-1054-4d52-83d9-e3cb51bc509a-kube-api-access-gngqz\") pod \"telemeter-client-7b95cf59d8-42hsw\" (UID: \"a1ae0621-1054-4d52-83d9-e3cb51bc509a\") " pod="openshift-monitoring/telemeter-client-7b95cf59d8-42hsw" Jul 06 20:11:17.870365 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:17.870207 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a1ae0621-1054-4d52-83d9-e3cb51bc509a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7b95cf59d8-42hsw\" (UID: \"a1ae0621-1054-4d52-83d9-e3cb51bc509a\") " pod="openshift-monitoring/telemeter-client-7b95cf59d8-42hsw" Jul 06 20:11:17.870365 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:17.870228 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a1ae0621-1054-4d52-83d9-e3cb51bc509a-telemeter-client-tls\") pod \"telemeter-client-7b95cf59d8-42hsw\" (UID: \"a1ae0621-1054-4d52-83d9-e3cb51bc509a\") " pod="openshift-monitoring/telemeter-client-7b95cf59d8-42hsw" Jul 06 20:11:17.870365 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:17.870244 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a1ae0621-1054-4d52-83d9-e3cb51bc509a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7b95cf59d8-42hsw\" (UID: \"a1ae0621-1054-4d52-83d9-e3cb51bc509a\") " pod="openshift-monitoring/telemeter-client-7b95cf59d8-42hsw" Jul 06 20:11:17.870365 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:17.870290 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a1ae0621-1054-4d52-83d9-e3cb51bc509a-federate-client-tls\") pod \"telemeter-client-7b95cf59d8-42hsw\" (UID: \"a1ae0621-1054-4d52-83d9-e3cb51bc509a\") " pod="openshift-monitoring/telemeter-client-7b95cf59d8-42hsw" Jul 06 20:11:17.971566 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:17.971514 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a1ae0621-1054-4d52-83d9-e3cb51bc509a-serving-certs-ca-bundle\") pod \"telemeter-client-7b95cf59d8-42hsw\" (UID: \"a1ae0621-1054-4d52-83d9-e3cb51bc509a\") " pod="openshift-monitoring/telemeter-client-7b95cf59d8-42hsw" Jul 06 20:11:17.971700 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:17.971585 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a1ae0621-1054-4d52-83d9-e3cb51bc509a-metrics-client-ca\") pod \"telemeter-client-7b95cf59d8-42hsw\" (UID: \"a1ae0621-1054-4d52-83d9-e3cb51bc509a\") " pod="openshift-monitoring/telemeter-client-7b95cf59d8-42hsw" Jul 06 20:11:17.971700 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:17.971605 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a1ae0621-1054-4d52-83d9-e3cb51bc509a-secret-telemeter-client\") pod \"telemeter-client-7b95cf59d8-42hsw\" (UID: \"a1ae0621-1054-4d52-83d9-e3cb51bc509a\") " pod="openshift-monitoring/telemeter-client-7b95cf59d8-42hsw" Jul 06 20:11:17.971700 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:17.971620 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gngqz\" (UniqueName: \"kubernetes.io/projected/a1ae0621-1054-4d52-83d9-e3cb51bc509a-kube-api-access-gngqz\") pod \"telemeter-client-7b95cf59d8-42hsw\" (UID: \"a1ae0621-1054-4d52-83d9-e3cb51bc509a\") " pod="openshift-monitoring/telemeter-client-7b95cf59d8-42hsw" Jul 06 20:11:17.971700 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:17.971643 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a1ae0621-1054-4d52-83d9-e3cb51bc509a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7b95cf59d8-42hsw\" (UID: \"a1ae0621-1054-4d52-83d9-e3cb51bc509a\") " pod="openshift-monitoring/telemeter-client-7b95cf59d8-42hsw" Jul 06 20:11:17.971700 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:17.971661 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a1ae0621-1054-4d52-83d9-e3cb51bc509a-telemeter-client-tls\") pod \"telemeter-client-7b95cf59d8-42hsw\" (UID: \"a1ae0621-1054-4d52-83d9-e3cb51bc509a\") " pod="openshift-monitoring/telemeter-client-7b95cf59d8-42hsw" Jul 06 20:11:17.971700 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:17.971677 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a1ae0621-1054-4d52-83d9-e3cb51bc509a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7b95cf59d8-42hsw\" (UID: \"a1ae0621-1054-4d52-83d9-e3cb51bc509a\") " pod="openshift-monitoring/telemeter-client-7b95cf59d8-42hsw" Jul 06 20:11:17.972115 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:17.972080 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a1ae0621-1054-4d52-83d9-e3cb51bc509a-federate-client-tls\") pod \"telemeter-client-7b95cf59d8-42hsw\" (UID: \"a1ae0621-1054-4d52-83d9-e3cb51bc509a\") " pod="openshift-monitoring/telemeter-client-7b95cf59d8-42hsw" Jul 06 20:11:17.972421 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:17.972394 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a1ae0621-1054-4d52-83d9-e3cb51bc509a-metrics-client-ca\") pod \"telemeter-client-7b95cf59d8-42hsw\" (UID: \"a1ae0621-1054-4d52-83d9-e3cb51bc509a\") " pod="openshift-monitoring/telemeter-client-7b95cf59d8-42hsw" Jul 06 20:11:17.972511 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:17.972394 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a1ae0621-1054-4d52-83d9-e3cb51bc509a-serving-certs-ca-bundle\") pod \"telemeter-client-7b95cf59d8-42hsw\" (UID: \"a1ae0621-1054-4d52-83d9-e3cb51bc509a\") " pod="openshift-monitoring/telemeter-client-7b95cf59d8-42hsw" Jul 06 20:11:17.972570 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:17.972548 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a1ae0621-1054-4d52-83d9-e3cb51bc509a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7b95cf59d8-42hsw\" (UID: \"a1ae0621-1054-4d52-83d9-e3cb51bc509a\") " pod="openshift-monitoring/telemeter-client-7b95cf59d8-42hsw" Jul 06 20:11:17.974320 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:17.974298 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a1ae0621-1054-4d52-83d9-e3cb51bc509a-secret-telemeter-client\") pod \"telemeter-client-7b95cf59d8-42hsw\" (UID: \"a1ae0621-1054-4d52-83d9-e3cb51bc509a\") " pod="openshift-monitoring/telemeter-client-7b95cf59d8-42hsw" Jul 06 20:11:17.974445 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:17.974345 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a1ae0621-1054-4d52-83d9-e3cb51bc509a-telemeter-client-tls\") pod \"telemeter-client-7b95cf59d8-42hsw\" (UID: \"a1ae0621-1054-4d52-83d9-e3cb51bc509a\") " pod="openshift-monitoring/telemeter-client-7b95cf59d8-42hsw" Jul 06 20:11:17.974445 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:17.974400 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a1ae0621-1054-4d52-83d9-e3cb51bc509a-federate-client-tls\") pod \"telemeter-client-7b95cf59d8-42hsw\" (UID: \"a1ae0621-1054-4d52-83d9-e3cb51bc509a\") " pod="openshift-monitoring/telemeter-client-7b95cf59d8-42hsw" Jul 06 20:11:17.974613 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:17.974591 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a1ae0621-1054-4d52-83d9-e3cb51bc509a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7b95cf59d8-42hsw\" (UID: \"a1ae0621-1054-4d52-83d9-e3cb51bc509a\") " pod="openshift-monitoring/telemeter-client-7b95cf59d8-42hsw" Jul 06 20:11:17.979520 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:17.979496 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gngqz\" (UniqueName: \"kubernetes.io/projected/a1ae0621-1054-4d52-83d9-e3cb51bc509a-kube-api-access-gngqz\") pod \"telemeter-client-7b95cf59d8-42hsw\" (UID: \"a1ae0621-1054-4d52-83d9-e3cb51bc509a\") " pod="openshift-monitoring/telemeter-client-7b95cf59d8-42hsw" Jul 06 20:11:18.112796 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:18.112705 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7b95cf59d8-42hsw" Jul 06 20:11:18.250177 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:18.250145 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7b95cf59d8-42hsw"] Jul 06 20:11:18.253272 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:11:18.253240 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1ae0621_1054_4d52_83d9_e3cb51bc509a.slice/crio-dfbe18060869201519a4fa0490e344b8cba4843e98246c52fe3665fc0fb614aa WatchSource:0}: Error finding container dfbe18060869201519a4fa0490e344b8cba4843e98246c52fe3665fc0fb614aa: Status 404 returned error can't find the container with id dfbe18060869201519a4fa0490e344b8cba4843e98246c52fe3665fc0fb614aa Jul 06 20:11:19.075858 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:19.075817 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7b95cf59d8-42hsw" event={"ID":"a1ae0621-1054-4d52-83d9-e3cb51bc509a","Type":"ContainerStarted","Data":"dfbe18060869201519a4fa0490e344b8cba4843e98246c52fe3665fc0fb614aa"} Jul 06 20:11:21.087336 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:21.087249 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7b95cf59d8-42hsw" event={"ID":"a1ae0621-1054-4d52-83d9-e3cb51bc509a","Type":"ContainerStarted","Data":"8a5ac015862a6571891b2ca09d8e2345fd8df3f4d88b5cd01b59ac2b6cae15d5"} Jul 06 20:11:21.087336 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:21.087334 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7b95cf59d8-42hsw" event={"ID":"a1ae0621-1054-4d52-83d9-e3cb51bc509a","Type":"ContainerStarted","Data":"ba7efb471f63ee8f3239430ca385a447fb870e9d1142b4841e46bb4d70846f28"} Jul 06 20:11:21.087746 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:21.087356 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7b95cf59d8-42hsw" event={"ID":"a1ae0621-1054-4d52-83d9-e3cb51bc509a","Type":"ContainerStarted","Data":"c5c5294d4df709fbd99b4adb5d1af2181444d7b08f4ecca3f96ca47e9eab4b9f"} Jul 06 20:11:21.112097 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:21.112040 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7b95cf59d8-42hsw" podStartSLOduration=1.614165205 podStartE2EDuration="4.112023021s" totalImagesPullingTime="2.497857816s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:11:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:11:21.110996025 +0000 UTC m=+191.526195832" watchObservedRunningTime="2026-07-06 20:11:21.112023021 +0000 UTC m=+191.527222879" Jul 06 20:11:33.232597 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:33.232556 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-684b859d-dwnm2"] Jul 06 20:11:58.255614 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:58.255569 2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-684b859d-dwnm2" podUID="5a824c13-6774-47e3-aacf-9396ae146422" containerName="console" containerID="cri-o://fe2a8f13533cc628055532ac63cd4d3bf7315b4ac777a4b3addd862d98cc0af0" gracePeriod=15 Jul 06 20:11:58.510193 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:58.510129 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-684b859d-dwnm2" Jul 06 20:11:58.614438 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:58.614403 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5a824c13-6774-47e3-aacf-9396ae146422-oauth-serving-cert\") pod \"5a824c13-6774-47e3-aacf-9396ae146422\" (UID: \"5a824c13-6774-47e3-aacf-9396ae146422\") " Jul 06 20:11:58.614658 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:58.614461 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5a824c13-6774-47e3-aacf-9396ae146422-service-ca\") pod \"5a824c13-6774-47e3-aacf-9396ae146422\" (UID: \"5a824c13-6774-47e3-aacf-9396ae146422\") " Jul 06 20:11:58.614658 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:58.614511 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5a824c13-6774-47e3-aacf-9396ae146422-console-config\") pod \"5a824c13-6774-47e3-aacf-9396ae146422\" (UID: \"5a824c13-6774-47e3-aacf-9396ae146422\") " Jul 06 20:11:58.614658 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:58.614538 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5a824c13-6774-47e3-aacf-9396ae146422-console-serving-cert\") pod \"5a824c13-6774-47e3-aacf-9396ae146422\" (UID: \"5a824c13-6774-47e3-aacf-9396ae146422\") " Jul 06 20:11:58.614658 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:58.614586 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a824c13-6774-47e3-aacf-9396ae146422-trusted-ca-bundle\") pod \"5a824c13-6774-47e3-aacf-9396ae146422\" (UID: \"5a824c13-6774-47e3-aacf-9396ae146422\") " Jul 06 20:11:58.614658 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:58.614608 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5a824c13-6774-47e3-aacf-9396ae146422-console-oauth-config\") pod \"5a824c13-6774-47e3-aacf-9396ae146422\" (UID: \"5a824c13-6774-47e3-aacf-9396ae146422\") " Jul 06 20:11:58.614658 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:58.614635 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lz4qz\" (UniqueName: \"kubernetes.io/projected/5a824c13-6774-47e3-aacf-9396ae146422-kube-api-access-lz4qz\") pod \"5a824c13-6774-47e3-aacf-9396ae146422\" (UID: \"5a824c13-6774-47e3-aacf-9396ae146422\") " Jul 06 20:11:58.614961 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:58.614916 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a824c13-6774-47e3-aacf-9396ae146422-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5a824c13-6774-47e3-aacf-9396ae146422" (UID: "5a824c13-6774-47e3-aacf-9396ae146422"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:11:58.615002 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:58.614968 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a824c13-6774-47e3-aacf-9396ae146422-console-config" (OuterVolumeSpecName: "console-config") pod "5a824c13-6774-47e3-aacf-9396ae146422" (UID: "5a824c13-6774-47e3-aacf-9396ae146422"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:11:58.615002 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:58.614982 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a824c13-6774-47e3-aacf-9396ae146422-service-ca" (OuterVolumeSpecName: "service-ca") pod "5a824c13-6774-47e3-aacf-9396ae146422" (UID: "5a824c13-6774-47e3-aacf-9396ae146422"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:11:58.615348 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:58.615321 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a824c13-6774-47e3-aacf-9396ae146422-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "5a824c13-6774-47e3-aacf-9396ae146422" (UID: "5a824c13-6774-47e3-aacf-9396ae146422"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:11:58.616982 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:58.616936 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a824c13-6774-47e3-aacf-9396ae146422-kube-api-access-lz4qz" (OuterVolumeSpecName: "kube-api-access-lz4qz") pod "5a824c13-6774-47e3-aacf-9396ae146422" (UID: "5a824c13-6774-47e3-aacf-9396ae146422"). InnerVolumeSpecName "kube-api-access-lz4qz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:11:58.616982 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:58.616952 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a824c13-6774-47e3-aacf-9396ae146422-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5a824c13-6774-47e3-aacf-9396ae146422" (UID: "5a824c13-6774-47e3-aacf-9396ae146422"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 20:11:58.617117 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:58.617031 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a824c13-6774-47e3-aacf-9396ae146422-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5a824c13-6774-47e3-aacf-9396ae146422" (UID: "5a824c13-6774-47e3-aacf-9396ae146422"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 20:11:58.715874 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:58.715837 2551 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5a824c13-6774-47e3-aacf-9396ae146422-oauth-serving-cert\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:11:58.715874 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:58.715867 2551 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5a824c13-6774-47e3-aacf-9396ae146422-service-ca\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:11:58.715874 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:58.715877 2551 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5a824c13-6774-47e3-aacf-9396ae146422-console-config\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:11:58.716140 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:58.715885 2551 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5a824c13-6774-47e3-aacf-9396ae146422-console-serving-cert\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:11:58.716140 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:58.715924 2551 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a824c13-6774-47e3-aacf-9396ae146422-trusted-ca-bundle\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:11:58.716140 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:58.715933 2551 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5a824c13-6774-47e3-aacf-9396ae146422-console-oauth-config\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:11:58.716140 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:58.715941 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lz4qz\" (UniqueName: \"kubernetes.io/projected/5a824c13-6774-47e3-aacf-9396ae146422-kube-api-access-lz4qz\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:11:59.236861 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:59.236829 2551 generic.go:356] "Generic (PLEG): container finished" podID="5a824c13-6774-47e3-aacf-9396ae146422" containerID="fe2a8f13533cc628055532ac63cd4d3bf7315b4ac777a4b3addd862d98cc0af0" exitCode=0 Jul 06 20:11:59.237062 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:59.236874 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-684b859d-dwnm2" Jul 06 20:11:59.237062 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:59.236918 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-684b859d-dwnm2" event={"ID":"5a824c13-6774-47e3-aacf-9396ae146422","Type":"ContainerDied","Data":"fe2a8f13533cc628055532ac63cd4d3bf7315b4ac777a4b3addd862d98cc0af0"} Jul 06 20:11:59.237062 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:59.236968 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-684b859d-dwnm2" event={"ID":"5a824c13-6774-47e3-aacf-9396ae146422","Type":"ContainerDied","Data":"c0222a415302faa7a61e5eb31e78d819cd58e50bcc4532d66b1cfe3f1c227f55"} Jul 06 20:11:59.237062 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:59.236989 2551 scope.go:117] "RemoveContainer" containerID="fe2a8f13533cc628055532ac63cd4d3bf7315b4ac777a4b3addd862d98cc0af0" Jul 06 20:11:59.245732 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:59.245711 2551 scope.go:117] "RemoveContainer" containerID="fe2a8f13533cc628055532ac63cd4d3bf7315b4ac777a4b3addd862d98cc0af0" Jul 06 20:11:59.246023 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:11:59.246005 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe2a8f13533cc628055532ac63cd4d3bf7315b4ac777a4b3addd862d98cc0af0\": container with ID starting with fe2a8f13533cc628055532ac63cd4d3bf7315b4ac777a4b3addd862d98cc0af0 not found: ID does not exist" containerID="fe2a8f13533cc628055532ac63cd4d3bf7315b4ac777a4b3addd862d98cc0af0" Jul 06 20:11:59.246108 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:59.246030 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe2a8f13533cc628055532ac63cd4d3bf7315b4ac777a4b3addd862d98cc0af0"} err="failed to get container status \"fe2a8f13533cc628055532ac63cd4d3bf7315b4ac777a4b3addd862d98cc0af0\": rpc error: code = NotFound desc = could not find container \"fe2a8f13533cc628055532ac63cd4d3bf7315b4ac777a4b3addd862d98cc0af0\": container with ID starting with fe2a8f13533cc628055532ac63cd4d3bf7315b4ac777a4b3addd862d98cc0af0 not found: ID does not exist" Jul 06 20:11:59.259387 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:59.259358 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-684b859d-dwnm2"] Jul 06 20:11:59.265369 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:11:59.265335 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-684b859d-dwnm2"] Jul 06 20:12:00.162920 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:12:00.162868 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a824c13-6774-47e3-aacf-9396ae146422" path="/var/lib/kubelet/pods/5a824c13-6774-47e3-aacf-9396ae146422/volumes" Jul 06 20:13:10.041996 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:13:10.041967 2551 kubelet.go:1648] "Image garbage collection succeeded" Jul 06 20:13:17.060964 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:13:17.060928 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-554687b44c-jtnjk"] Jul 06 20:13:17.063151 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:13:17.061241 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a824c13-6774-47e3-aacf-9396ae146422" containerName="console" Jul 06 20:13:17.063151 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:13:17.061251 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a824c13-6774-47e3-aacf-9396ae146422" containerName="console" Jul 06 20:13:17.063151 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:13:17.061302 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a824c13-6774-47e3-aacf-9396ae146422" containerName="console" Jul 06 20:13:17.063928 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:13:17.063912 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-554687b44c-jtnjk" Jul 06 20:13:17.078540 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:13:17.078508 2551 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"open-cluster-management-image-pull-credentials\" is forbidden: User \"system:node:ip-10-0-137-178.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"open-cluster-management-agent-addon\": no relationship found between node 'ip-10-0-137-178.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" type="*v1.Secret" Jul 06 20:13:17.078851 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:13:17.078828 2551 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"work-manager-hub-kubeconfig\" is forbidden: User \"system:node:ip-10-0-137-178.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"open-cluster-management-agent-addon\": no relationship found between node 'ip-10-0-137-178.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" type="*v1.Secret" Jul 06 20:13:17.079132 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:13:17.079112 2551 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-137-178.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"open-cluster-management-agent-addon\": no relationship found between node 'ip-10-0-137-178.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" Jul 06 20:13:17.079490 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:13:17.079469 2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"klusterlet-addon-workmgr-554687b44c-jtnjk\" is forbidden: User \"system:node:ip-10-0-137-178.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"open-cluster-management-agent-addon\": no relationship found between node 'ip-10-0-137-178.ec2.internal' and this object" podUID="0589be5e-b797-44bf-90b4-6f778f3f42f1" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-554687b44c-jtnjk" Jul 06 20:13:17.081034 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:13:17.081016 2551 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-137-178.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"open-cluster-management-agent-addon\": no relationship found between node 'ip-10-0-137-178.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" Jul 06 20:13:17.122313 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:13:17.122274 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-554687b44c-jtnjk"] Jul 06 20:13:17.176180 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:13:17.176142 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0589be5e-b797-44bf-90b4-6f778f3f42f1-tmp\") pod \"klusterlet-addon-workmgr-554687b44c-jtnjk\" (UID: \"0589be5e-b797-44bf-90b4-6f778f3f42f1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-554687b44c-jtnjk" Jul 06 20:13:17.176363 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:13:17.176199 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgds4\" (UniqueName: \"kubernetes.io/projected/0589be5e-b797-44bf-90b4-6f778f3f42f1-kube-api-access-pgds4\") pod \"klusterlet-addon-workmgr-554687b44c-jtnjk\" (UID: \"0589be5e-b797-44bf-90b4-6f778f3f42f1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-554687b44c-jtnjk" Jul 06 20:13:17.176363 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:13:17.176232 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/0589be5e-b797-44bf-90b4-6f778f3f42f1-klusterlet-config\") pod \"klusterlet-addon-workmgr-554687b44c-jtnjk\" (UID: \"0589be5e-b797-44bf-90b4-6f778f3f42f1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-554687b44c-jtnjk" Jul 06 20:13:17.276647 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:13:17.276606 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/0589be5e-b797-44bf-90b4-6f778f3f42f1-klusterlet-config\") pod \"klusterlet-addon-workmgr-554687b44c-jtnjk\" (UID: \"0589be5e-b797-44bf-90b4-6f778f3f42f1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-554687b44c-jtnjk" Jul 06 20:13:17.276802 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:13:17.276696 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0589be5e-b797-44bf-90b4-6f778f3f42f1-tmp\") pod \"klusterlet-addon-workmgr-554687b44c-jtnjk\" (UID: \"0589be5e-b797-44bf-90b4-6f778f3f42f1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-554687b44c-jtnjk" Jul 06 20:13:17.276802 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:13:17.276746 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgds4\" (UniqueName: \"kubernetes.io/projected/0589be5e-b797-44bf-90b4-6f778f3f42f1-kube-api-access-pgds4\") pod \"klusterlet-addon-workmgr-554687b44c-jtnjk\" (UID: \"0589be5e-b797-44bf-90b4-6f778f3f42f1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-554687b44c-jtnjk" Jul 06 20:13:17.277126 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:13:17.277106 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0589be5e-b797-44bf-90b4-6f778f3f42f1-tmp\") pod \"klusterlet-addon-workmgr-554687b44c-jtnjk\" (UID: \"0589be5e-b797-44bf-90b4-6f778f3f42f1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-554687b44c-jtnjk" Jul 06 20:13:17.941047 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:13:17.941009 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 06 20:13:17.949555 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:13:17.949525 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/0589be5e-b797-44bf-90b4-6f778f3f42f1-klusterlet-config\") pod \"klusterlet-addon-workmgr-554687b44c-jtnjk\" (UID: \"0589be5e-b797-44bf-90b4-6f778f3f42f1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-554687b44c-jtnjk" Jul 06 20:13:17.974190 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:13:17.974153 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 06 20:13:18.638181 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:13:18.638148 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 06 20:13:18.655221 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:13:18.655193 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 06 20:13:18.667223 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:13:18.667195 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgds4\" (UniqueName: \"kubernetes.io/projected/0589be5e-b797-44bf-90b4-6f778f3f42f1-kube-api-access-pgds4\") pod \"klusterlet-addon-workmgr-554687b44c-jtnjk\" (UID: \"0589be5e-b797-44bf-90b4-6f778f3f42f1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-554687b44c-jtnjk" Jul 06 20:13:18.872942 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:13:18.872877 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-554687b44c-jtnjk" Jul 06 20:13:19.012542 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:13:19.012430 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-554687b44c-jtnjk"] Jul 06 20:13:19.015343 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:13:19.015319 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0589be5e_b797_44bf_90b4_6f778f3f42f1.slice/crio-ce92d8e2868bc63def2697f52b8c885854738f245a94b4210cc8258a6fdff7b8 WatchSource:0}: Error finding container ce92d8e2868bc63def2697f52b8c885854738f245a94b4210cc8258a6fdff7b8: Status 404 returned error can't find the container with id ce92d8e2868bc63def2697f52b8c885854738f245a94b4210cc8258a6fdff7b8 Jul 06 20:13:19.017869 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:13:19.017854 2551 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 20:13:19.541310 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:13:19.541271 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-554687b44c-jtnjk" event={"ID":"0589be5e-b797-44bf-90b4-6f778f3f42f1","Type":"ContainerStarted","Data":"ce92d8e2868bc63def2697f52b8c885854738f245a94b4210cc8258a6fdff7b8"} Jul 06 20:13:23.560177 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:13:23.560142 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-554687b44c-jtnjk" event={"ID":"0589be5e-b797-44bf-90b4-6f778f3f42f1","Type":"ContainerStarted","Data":"8e847f87ec67f7b4d798c665b66c32e290ebbd7ee6429ae45c35369ce4f74a83"} Jul 06 20:13:23.560558 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:13:23.560348 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-554687b44c-jtnjk" Jul 06 20:13:23.562211 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:13:23.562185 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-554687b44c-jtnjk" Jul 06 20:13:23.578391 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:13:23.578331 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-554687b44c-jtnjk" podStartSLOduration=2.93109573 podStartE2EDuration="6.578308103s" totalImagesPullingTime="3.647212373s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:13:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:13:23.577517504 +0000 UTC m=+313.992717329" watchObservedRunningTime="2026-07-06 20:13:23.578308103 +0000 UTC m=+313.993507908" Jul 06 20:14:31.274405 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:31.274370 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vhgl7"] Jul 06 20:14:31.276933 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:31.276915 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vhgl7" Jul 06 20:14:31.279421 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:31.279399 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 06 20:14:31.279554 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:31.279454 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 06 20:14:31.279619 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:31.279554 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-wt4k5\"" Jul 06 20:14:31.285831 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:31.285807 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vhgl7"] Jul 06 20:14:31.404908 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:31.404855 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fe1adf61-fa2b-447b-82ec-7856c02d40d0-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vhgl7\" (UID: \"fe1adf61-fa2b-447b-82ec-7856c02d40d0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vhgl7" Jul 06 20:14:31.405089 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:31.404921 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nmb9\" (UniqueName: \"kubernetes.io/projected/fe1adf61-fa2b-447b-82ec-7856c02d40d0-kube-api-access-7nmb9\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vhgl7\" (UID: \"fe1adf61-fa2b-447b-82ec-7856c02d40d0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vhgl7" Jul 06 20:14:31.405089 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:31.404980 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fe1adf61-fa2b-447b-82ec-7856c02d40d0-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vhgl7\" (UID: \"fe1adf61-fa2b-447b-82ec-7856c02d40d0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vhgl7" Jul 06 20:14:31.506365 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:31.506314 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fe1adf61-fa2b-447b-82ec-7856c02d40d0-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vhgl7\" (UID: \"fe1adf61-fa2b-447b-82ec-7856c02d40d0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vhgl7" Jul 06 20:14:31.506495 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:31.506387 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nmb9\" (UniqueName: \"kubernetes.io/projected/fe1adf61-fa2b-447b-82ec-7856c02d40d0-kube-api-access-7nmb9\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vhgl7\" (UID: \"fe1adf61-fa2b-447b-82ec-7856c02d40d0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vhgl7" Jul 06 20:14:31.506495 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:31.506441 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fe1adf61-fa2b-447b-82ec-7856c02d40d0-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vhgl7\" (UID: \"fe1adf61-fa2b-447b-82ec-7856c02d40d0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vhgl7" Jul 06 20:14:31.506815 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:31.506794 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fe1adf61-fa2b-447b-82ec-7856c02d40d0-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vhgl7\" (UID: \"fe1adf61-fa2b-447b-82ec-7856c02d40d0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vhgl7" Jul 06 20:14:31.506856 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:31.506803 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fe1adf61-fa2b-447b-82ec-7856c02d40d0-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vhgl7\" (UID: \"fe1adf61-fa2b-447b-82ec-7856c02d40d0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vhgl7" Jul 06 20:14:31.515257 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:31.515231 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nmb9\" (UniqueName: \"kubernetes.io/projected/fe1adf61-fa2b-447b-82ec-7856c02d40d0-kube-api-access-7nmb9\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vhgl7\" (UID: \"fe1adf61-fa2b-447b-82ec-7856c02d40d0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vhgl7" Jul 06 20:14:31.587346 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:31.587255 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vhgl7" Jul 06 20:14:31.721669 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:31.721636 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vhgl7"] Jul 06 20:14:31.725426 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:14:31.725397 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe1adf61_fa2b_447b_82ec_7856c02d40d0.slice/crio-5c5e2883d18452c3d3f8e8a09caed8b428efa98450854697b82c5defd176b4d0 WatchSource:0}: Error finding container 5c5e2883d18452c3d3f8e8a09caed8b428efa98450854697b82c5defd176b4d0: Status 404 returned error can't find the container with id 5c5e2883d18452c3d3f8e8a09caed8b428efa98450854697b82c5defd176b4d0 Jul 06 20:14:31.826194 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:31.826160 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vhgl7" event={"ID":"fe1adf61-fa2b-447b-82ec-7856c02d40d0","Type":"ContainerStarted","Data":"5c5e2883d18452c3d3f8e8a09caed8b428efa98450854697b82c5defd176b4d0"} Jul 06 20:14:40.864762 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:40.864720 2551 generic.go:356] "Generic (PLEG): container finished" podID="fe1adf61-fa2b-447b-82ec-7856c02d40d0" containerID="4fc2338849100f489471a7e4aa517dbba38af29af21d350ad7024bfd6564cefa" exitCode=0 Jul 06 20:14:40.865164 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:40.864802 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vhgl7" event={"ID":"fe1adf61-fa2b-447b-82ec-7856c02d40d0","Type":"ContainerDied","Data":"4fc2338849100f489471a7e4aa517dbba38af29af21d350ad7024bfd6564cefa"} Jul 06 20:14:43.877127 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:43.877089 2551 generic.go:356] "Generic (PLEG): container finished" podID="fe1adf61-fa2b-447b-82ec-7856c02d40d0" containerID="05aa1eaa881acc8c53674dc478dcba7bc917ca9077e2fb775283d9fb2c048ea2" exitCode=0 Jul 06 20:14:43.877531 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:43.877148 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vhgl7" event={"ID":"fe1adf61-fa2b-447b-82ec-7856c02d40d0","Type":"ContainerDied","Data":"05aa1eaa881acc8c53674dc478dcba7bc917ca9077e2fb775283d9fb2c048ea2"} Jul 06 20:14:52.920917 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:52.920870 2551 generic.go:356] "Generic (PLEG): container finished" podID="fe1adf61-fa2b-447b-82ec-7856c02d40d0" containerID="1462bb57d2a8462228409e5add770456ea513882c53c70568b508ad56d5aa527" exitCode=0 Jul 06 20:14:52.921316 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:52.920950 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vhgl7" event={"ID":"fe1adf61-fa2b-447b-82ec-7856c02d40d0","Type":"ContainerDied","Data":"1462bb57d2a8462228409e5add770456ea513882c53c70568b508ad56d5aa527"} Jul 06 20:14:54.106055 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:54.106020 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vhgl7" Jul 06 20:14:54.214705 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:54.214616 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fe1adf61-fa2b-447b-82ec-7856c02d40d0-bundle\") pod \"fe1adf61-fa2b-447b-82ec-7856c02d40d0\" (UID: \"fe1adf61-fa2b-447b-82ec-7856c02d40d0\") " Jul 06 20:14:54.214705 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:54.214672 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7nmb9\" (UniqueName: \"kubernetes.io/projected/fe1adf61-fa2b-447b-82ec-7856c02d40d0-kube-api-access-7nmb9\") pod \"fe1adf61-fa2b-447b-82ec-7856c02d40d0\" (UID: \"fe1adf61-fa2b-447b-82ec-7856c02d40d0\") " Jul 06 20:14:54.214705 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:54.214688 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fe1adf61-fa2b-447b-82ec-7856c02d40d0-util\") pod \"fe1adf61-fa2b-447b-82ec-7856c02d40d0\" (UID: \"fe1adf61-fa2b-447b-82ec-7856c02d40d0\") " Jul 06 20:14:54.215324 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:54.215287 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe1adf61-fa2b-447b-82ec-7856c02d40d0-bundle" (OuterVolumeSpecName: "bundle") pod "fe1adf61-fa2b-447b-82ec-7856c02d40d0" (UID: "fe1adf61-fa2b-447b-82ec-7856c02d40d0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:14:54.216824 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:54.216799 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe1adf61-fa2b-447b-82ec-7856c02d40d0-kube-api-access-7nmb9" (OuterVolumeSpecName: "kube-api-access-7nmb9") pod "fe1adf61-fa2b-447b-82ec-7856c02d40d0" (UID: "fe1adf61-fa2b-447b-82ec-7856c02d40d0"). InnerVolumeSpecName "kube-api-access-7nmb9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:14:54.219867 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:54.219843 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe1adf61-fa2b-447b-82ec-7856c02d40d0-util" (OuterVolumeSpecName: "util") pod "fe1adf61-fa2b-447b-82ec-7856c02d40d0" (UID: "fe1adf61-fa2b-447b-82ec-7856c02d40d0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:14:54.315919 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:54.315860 2551 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fe1adf61-fa2b-447b-82ec-7856c02d40d0-bundle\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:14:54.315919 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:54.315886 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7nmb9\" (UniqueName: \"kubernetes.io/projected/fe1adf61-fa2b-447b-82ec-7856c02d40d0-kube-api-access-7nmb9\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:14:54.315919 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:54.315909 2551 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fe1adf61-fa2b-447b-82ec-7856c02d40d0-util\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:14:54.931020 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:54.930993 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vhgl7" Jul 06 20:14:54.931020 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:54.930997 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vhgl7" event={"ID":"fe1adf61-fa2b-447b-82ec-7856c02d40d0","Type":"ContainerDied","Data":"5c5e2883d18452c3d3f8e8a09caed8b428efa98450854697b82c5defd176b4d0"} Jul 06 20:14:54.931235 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:54.931037 2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5c5e2883d18452c3d3f8e8a09caed8b428efa98450854697b82c5defd176b4d0" Jul 06 20:14:59.240466 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:59.240429 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7bctl"] Jul 06 20:14:59.240828 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:59.240729 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe1adf61-fa2b-447b-82ec-7856c02d40d0" containerName="extract" Jul 06 20:14:59.240828 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:59.240740 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe1adf61-fa2b-447b-82ec-7856c02d40d0" containerName="extract" Jul 06 20:14:59.240828 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:59.240758 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe1adf61-fa2b-447b-82ec-7856c02d40d0" containerName="util" Jul 06 20:14:59.240828 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:59.240764 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe1adf61-fa2b-447b-82ec-7856c02d40d0" containerName="util" Jul 06 20:14:59.240828 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:59.240772 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe1adf61-fa2b-447b-82ec-7856c02d40d0" containerName="pull" Jul 06 20:14:59.240828 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:59.240778 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe1adf61-fa2b-447b-82ec-7856c02d40d0" containerName="pull" Jul 06 20:14:59.240828 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:59.240829 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe1adf61-fa2b-447b-82ec-7856c02d40d0" containerName="extract" Jul 06 20:14:59.243014 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:59.242998 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7bctl" Jul 06 20:14:59.245741 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:59.245714 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 06 20:14:59.245882 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:59.245742 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-hsrkr\"" Jul 06 20:14:59.245882 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:59.245743 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 06 20:14:59.255670 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:59.255643 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7bctl"] Jul 06 20:14:59.361114 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:59.361075 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/64fa2128-6941-43c0-bc3a-d1a9bc1c0d25-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-7bctl\" (UID: \"64fa2128-6941-43c0-bc3a-d1a9bc1c0d25\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7bctl" Jul 06 20:14:59.361309 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:59.361125 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s52cw\" (UniqueName: \"kubernetes.io/projected/64fa2128-6941-43c0-bc3a-d1a9bc1c0d25-kube-api-access-s52cw\") pod \"cert-manager-operator-controller-manager-55c444bf86-7bctl\" (UID: \"64fa2128-6941-43c0-bc3a-d1a9bc1c0d25\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7bctl" Jul 06 20:14:59.462569 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:59.462533 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/64fa2128-6941-43c0-bc3a-d1a9bc1c0d25-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-7bctl\" (UID: \"64fa2128-6941-43c0-bc3a-d1a9bc1c0d25\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7bctl" Jul 06 20:14:59.462569 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:59.462578 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s52cw\" (UniqueName: \"kubernetes.io/projected/64fa2128-6941-43c0-bc3a-d1a9bc1c0d25-kube-api-access-s52cw\") pod \"cert-manager-operator-controller-manager-55c444bf86-7bctl\" (UID: \"64fa2128-6941-43c0-bc3a-d1a9bc1c0d25\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7bctl" Jul 06 20:14:59.463028 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:59.463002 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/64fa2128-6941-43c0-bc3a-d1a9bc1c0d25-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-7bctl\" (UID: \"64fa2128-6941-43c0-bc3a-d1a9bc1c0d25\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7bctl" Jul 06 20:14:59.476102 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:59.476071 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s52cw\" (UniqueName: \"kubernetes.io/projected/64fa2128-6941-43c0-bc3a-d1a9bc1c0d25-kube-api-access-s52cw\") pod \"cert-manager-operator-controller-manager-55c444bf86-7bctl\" (UID: \"64fa2128-6941-43c0-bc3a-d1a9bc1c0d25\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7bctl" Jul 06 20:14:59.552080 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:59.552050 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7bctl" Jul 06 20:14:59.687698 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:59.687673 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7bctl"] Jul 06 20:14:59.690559 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:14:59.690534 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64fa2128_6941_43c0_bc3a_d1a9bc1c0d25.slice/crio-1374f9aefbf3c348cd448957c916c1f5c7c3ef92d8f85bdc3d4252e866d0438e WatchSource:0}: Error finding container 1374f9aefbf3c348cd448957c916c1f5c7c3ef92d8f85bdc3d4252e866d0438e: Status 404 returned error can't find the container with id 1374f9aefbf3c348cd448957c916c1f5c7c3ef92d8f85bdc3d4252e866d0438e Jul 06 20:14:59.953771 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:14:59.953687 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7bctl" event={"ID":"64fa2128-6941-43c0-bc3a-d1a9bc1c0d25","Type":"ContainerStarted","Data":"1374f9aefbf3c348cd448957c916c1f5c7c3ef92d8f85bdc3d4252e866d0438e"} Jul 06 20:15:01.964234 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:01.964191 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7bctl" event={"ID":"64fa2128-6941-43c0-bc3a-d1a9bc1c0d25","Type":"ContainerStarted","Data":"35be1988467ca91b02848939cfb15bcaa8df2d3d73ded78144bc8b3cd09d982b"} Jul 06 20:15:01.986066 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:01.985998 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7bctl" podStartSLOduration=1.569453394 podStartE2EDuration="2.985980915s" totalImagesPullingTime="1.416527521s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:14:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:15:01.984544112 +0000 UTC m=+412.399743918" watchObservedRunningTime="2026-07-06 20:15:01.985980915 +0000 UTC m=+412.401180719" Jul 06 20:15:03.149167 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:03.149131 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmv45"] Jul 06 20:15:03.151626 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:03.151610 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmv45" Jul 06 20:15:03.154322 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:03.154298 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 06 20:15:03.154486 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:03.154323 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-wt4k5\"" Jul 06 20:15:03.155188 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:03.155170 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 06 20:15:03.164003 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:03.163978 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmv45"] Jul 06 20:15:03.193296 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:03.193257 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dknr\" (UniqueName: \"kubernetes.io/projected/3347133c-84df-4e8c-a5c6-ff34d1c24afe-kube-api-access-2dknr\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmv45\" (UID: \"3347133c-84df-4e8c-a5c6-ff34d1c24afe\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmv45" Jul 06 20:15:03.193296 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:03.193297 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3347133c-84df-4e8c-a5c6-ff34d1c24afe-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmv45\" (UID: \"3347133c-84df-4e8c-a5c6-ff34d1c24afe\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmv45" Jul 06 20:15:03.193547 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:03.193328 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3347133c-84df-4e8c-a5c6-ff34d1c24afe-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmv45\" (UID: \"3347133c-84df-4e8c-a5c6-ff34d1c24afe\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmv45" Jul 06 20:15:03.294129 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:03.294090 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dknr\" (UniqueName: \"kubernetes.io/projected/3347133c-84df-4e8c-a5c6-ff34d1c24afe-kube-api-access-2dknr\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmv45\" (UID: \"3347133c-84df-4e8c-a5c6-ff34d1c24afe\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmv45" Jul 06 20:15:03.294129 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:03.294130 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3347133c-84df-4e8c-a5c6-ff34d1c24afe-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmv45\" (UID: \"3347133c-84df-4e8c-a5c6-ff34d1c24afe\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmv45" Jul 06 20:15:03.294411 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:03.294168 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3347133c-84df-4e8c-a5c6-ff34d1c24afe-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmv45\" (UID: \"3347133c-84df-4e8c-a5c6-ff34d1c24afe\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmv45" Jul 06 20:15:03.294644 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:03.294620 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3347133c-84df-4e8c-a5c6-ff34d1c24afe-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmv45\" (UID: \"3347133c-84df-4e8c-a5c6-ff34d1c24afe\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmv45" Jul 06 20:15:03.294690 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:03.294629 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3347133c-84df-4e8c-a5c6-ff34d1c24afe-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmv45\" (UID: \"3347133c-84df-4e8c-a5c6-ff34d1c24afe\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmv45" Jul 06 20:15:03.302987 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:03.302955 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dknr\" (UniqueName: \"kubernetes.io/projected/3347133c-84df-4e8c-a5c6-ff34d1c24afe-kube-api-access-2dknr\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmv45\" (UID: \"3347133c-84df-4e8c-a5c6-ff34d1c24afe\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmv45" Jul 06 20:15:03.461346 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:03.461249 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmv45" Jul 06 20:15:03.601121 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:03.601090 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmv45"] Jul 06 20:15:03.604072 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:15:03.604043 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3347133c_84df_4e8c_a5c6_ff34d1c24afe.slice/crio-9d02ee9fea11f6c4e900991c0891859aba2b387fd64d4995cf4aea7bb86ac7a3 WatchSource:0}: Error finding container 9d02ee9fea11f6c4e900991c0891859aba2b387fd64d4995cf4aea7bb86ac7a3: Status 404 returned error can't find the container with id 9d02ee9fea11f6c4e900991c0891859aba2b387fd64d4995cf4aea7bb86ac7a3 Jul 06 20:15:03.975250 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:03.975155 2551 generic.go:356] "Generic (PLEG): container finished" podID="3347133c-84df-4e8c-a5c6-ff34d1c24afe" containerID="3e747f2bd169c64c8cf86cd7418ad253f6a542569b9ce691308273f9f7e753e1" exitCode=0 Jul 06 20:15:03.975397 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:03.975248 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmv45" event={"ID":"3347133c-84df-4e8c-a5c6-ff34d1c24afe","Type":"ContainerDied","Data":"3e747f2bd169c64c8cf86cd7418ad253f6a542569b9ce691308273f9f7e753e1"} Jul 06 20:15:03.975397 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:03.975293 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmv45" event={"ID":"3347133c-84df-4e8c-a5c6-ff34d1c24afe","Type":"ContainerStarted","Data":"9d02ee9fea11f6c4e900991c0891859aba2b387fd64d4995cf4aea7bb86ac7a3"} Jul 06 20:15:05.414493 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:05.414455 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-zbbpt"] Jul 06 20:15:05.420798 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:05.420765 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-zbbpt" Jul 06 20:15:05.423484 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:05.423461 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-5k474\"" Jul 06 20:15:05.423639 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:05.423480 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 06 20:15:05.424754 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:05.424578 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 06 20:15:05.427018 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:05.426992 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-zbbpt"] Jul 06 20:15:05.509617 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:05.509584 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1e3e2779-0f3e-4c0e-b22f-108d6247f42f-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-zbbpt\" (UID: \"1e3e2779-0f3e-4c0e-b22f-108d6247f42f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zbbpt" Jul 06 20:15:05.509846 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:05.509643 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fg6sw\" (UniqueName: \"kubernetes.io/projected/1e3e2779-0f3e-4c0e-b22f-108d6247f42f-kube-api-access-fg6sw\") pod \"cert-manager-webhook-6486ffd997-zbbpt\" (UID: \"1e3e2779-0f3e-4c0e-b22f-108d6247f42f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zbbpt" Jul 06 20:15:05.610739 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:05.610697 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1e3e2779-0f3e-4c0e-b22f-108d6247f42f-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-zbbpt\" (UID: \"1e3e2779-0f3e-4c0e-b22f-108d6247f42f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zbbpt" Jul 06 20:15:05.610958 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:05.610803 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fg6sw\" (UniqueName: \"kubernetes.io/projected/1e3e2779-0f3e-4c0e-b22f-108d6247f42f-kube-api-access-fg6sw\") pod \"cert-manager-webhook-6486ffd997-zbbpt\" (UID: \"1e3e2779-0f3e-4c0e-b22f-108d6247f42f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zbbpt" Jul 06 20:15:05.619505 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:05.619474 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1e3e2779-0f3e-4c0e-b22f-108d6247f42f-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-zbbpt\" (UID: \"1e3e2779-0f3e-4c0e-b22f-108d6247f42f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zbbpt" Jul 06 20:15:05.619657 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:05.619559 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fg6sw\" (UniqueName: \"kubernetes.io/projected/1e3e2779-0f3e-4c0e-b22f-108d6247f42f-kube-api-access-fg6sw\") pod \"cert-manager-webhook-6486ffd997-zbbpt\" (UID: \"1e3e2779-0f3e-4c0e-b22f-108d6247f42f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zbbpt" Jul 06 20:15:05.740256 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:05.740165 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-zbbpt" Jul 06 20:15:05.948208 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:05.948177 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-zbbpt"] Jul 06 20:15:05.951176 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:15:05.951147 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1e3e2779_0f3e_4c0e_b22f_108d6247f42f.slice/crio-984ddfb37c9a8819e23e5a1ec718b10577bfe07f615366b1df26add49fa05594 WatchSource:0}: Error finding container 984ddfb37c9a8819e23e5a1ec718b10577bfe07f615366b1df26add49fa05594: Status 404 returned error can't find the container with id 984ddfb37c9a8819e23e5a1ec718b10577bfe07f615366b1df26add49fa05594 Jul 06 20:15:05.986855 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:05.986812 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-zbbpt" event={"ID":"1e3e2779-0f3e-4c0e-b22f-108d6247f42f","Type":"ContainerStarted","Data":"984ddfb37c9a8819e23e5a1ec718b10577bfe07f615366b1df26add49fa05594"} Jul 06 20:15:05.988609 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:05.988571 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmv45" event={"ID":"3347133c-84df-4e8c-a5c6-ff34d1c24afe","Type":"ContainerStarted","Data":"fd17343ec5e199cadfe988393dfc30385688d5a0267e5794e2e80de532ba1adf"} Jul 06 20:15:06.997132 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:06.997035 2551 generic.go:356] "Generic (PLEG): container finished" podID="3347133c-84df-4e8c-a5c6-ff34d1c24afe" containerID="fd17343ec5e199cadfe988393dfc30385688d5a0267e5794e2e80de532ba1adf" exitCode=0 Jul 06 20:15:06.997132 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:06.997074 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmv45" event={"ID":"3347133c-84df-4e8c-a5c6-ff34d1c24afe","Type":"ContainerDied","Data":"fd17343ec5e199cadfe988393dfc30385688d5a0267e5794e2e80de532ba1adf"} Jul 06 20:15:08.005590 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:08.005555 2551 generic.go:356] "Generic (PLEG): container finished" podID="3347133c-84df-4e8c-a5c6-ff34d1c24afe" containerID="3b003b196ab4e4ef01d416863d8b88757c93641d5c088b99e65d04239ef15947" exitCode=0 Jul 06 20:15:08.006066 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:08.005643 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmv45" event={"ID":"3347133c-84df-4e8c-a5c6-ff34d1c24afe","Type":"ContainerDied","Data":"3b003b196ab4e4ef01d416863d8b88757c93641d5c088b99e65d04239ef15947"} Jul 06 20:15:09.012077 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:09.012043 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-zbbpt" event={"ID":"1e3e2779-0f3e-4c0e-b22f-108d6247f42f","Type":"ContainerStarted","Data":"cb30a8df58ef5b4a7b5d84c83015d736d52e92d64e0eb2a0c6ac0154d27fb39b"} Jul 06 20:15:09.012497 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:09.012172 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-zbbpt" Jul 06 20:15:09.039653 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:09.039597 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-zbbpt" podStartSLOduration=1.717777917 podStartE2EDuration="4.039581606s" totalImagesPullingTime="2.321803689s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:15:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:15:09.037973624 +0000 UTC m=+419.453173429" watchObservedRunningTime="2026-07-06 20:15:09.039581606 +0000 UTC m=+419.454781410" Jul 06 20:15:09.139202 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:09.139179 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmv45" Jul 06 20:15:09.236582 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:09.236532 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2dknr\" (UniqueName: \"kubernetes.io/projected/3347133c-84df-4e8c-a5c6-ff34d1c24afe-kube-api-access-2dknr\") pod \"3347133c-84df-4e8c-a5c6-ff34d1c24afe\" (UID: \"3347133c-84df-4e8c-a5c6-ff34d1c24afe\") " Jul 06 20:15:09.236582 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:09.236595 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3347133c-84df-4e8c-a5c6-ff34d1c24afe-bundle\") pod \"3347133c-84df-4e8c-a5c6-ff34d1c24afe\" (UID: \"3347133c-84df-4e8c-a5c6-ff34d1c24afe\") " Jul 06 20:15:09.236846 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:09.236621 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3347133c-84df-4e8c-a5c6-ff34d1c24afe-util\") pod \"3347133c-84df-4e8c-a5c6-ff34d1c24afe\" (UID: \"3347133c-84df-4e8c-a5c6-ff34d1c24afe\") " Jul 06 20:15:09.237174 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:09.237070 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3347133c-84df-4e8c-a5c6-ff34d1c24afe-bundle" (OuterVolumeSpecName: "bundle") pod "3347133c-84df-4e8c-a5c6-ff34d1c24afe" (UID: "3347133c-84df-4e8c-a5c6-ff34d1c24afe"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:15:09.238860 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:09.238839 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3347133c-84df-4e8c-a5c6-ff34d1c24afe-kube-api-access-2dknr" (OuterVolumeSpecName: "kube-api-access-2dknr") pod "3347133c-84df-4e8c-a5c6-ff34d1c24afe" (UID: "3347133c-84df-4e8c-a5c6-ff34d1c24afe"). InnerVolumeSpecName "kube-api-access-2dknr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:15:09.241216 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:09.241169 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3347133c-84df-4e8c-a5c6-ff34d1c24afe-util" (OuterVolumeSpecName: "util") pod "3347133c-84df-4e8c-a5c6-ff34d1c24afe" (UID: "3347133c-84df-4e8c-a5c6-ff34d1c24afe"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:15:09.338213 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:09.338125 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2dknr\" (UniqueName: \"kubernetes.io/projected/3347133c-84df-4e8c-a5c6-ff34d1c24afe-kube-api-access-2dknr\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:15:09.338213 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:09.338158 2551 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3347133c-84df-4e8c-a5c6-ff34d1c24afe-bundle\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:15:09.338213 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:09.338167 2551 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3347133c-84df-4e8c-a5c6-ff34d1c24afe-util\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:15:10.018324 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:10.018296 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmv45" Jul 06 20:15:10.018324 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:10.018315 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmv45" event={"ID":"3347133c-84df-4e8c-a5c6-ff34d1c24afe","Type":"ContainerDied","Data":"9d02ee9fea11f6c4e900991c0891859aba2b387fd64d4995cf4aea7bb86ac7a3"} Jul 06 20:15:10.018825 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:10.018366 2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9d02ee9fea11f6c4e900991c0891859aba2b387fd64d4995cf4aea7bb86ac7a3" Jul 06 20:15:15.021314 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:15.021278 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-zbbpt" Jul 06 20:15:19.222786 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:19.222750 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6n5wd"] Jul 06 20:15:19.223273 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:19.223075 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3347133c-84df-4e8c-a5c6-ff34d1c24afe" containerName="pull" Jul 06 20:15:19.223273 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:19.223087 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="3347133c-84df-4e8c-a5c6-ff34d1c24afe" containerName="pull" Jul 06 20:15:19.223273 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:19.223103 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3347133c-84df-4e8c-a5c6-ff34d1c24afe" containerName="extract" Jul 06 20:15:19.223273 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:19.223109 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="3347133c-84df-4e8c-a5c6-ff34d1c24afe" containerName="extract" Jul 06 20:15:19.223273 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:19.223120 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3347133c-84df-4e8c-a5c6-ff34d1c24afe" containerName="util" Jul 06 20:15:19.223273 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:19.223126 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="3347133c-84df-4e8c-a5c6-ff34d1c24afe" containerName="util" Jul 06 20:15:19.223273 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:19.223195 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="3347133c-84df-4e8c-a5c6-ff34d1c24afe" containerName="extract" Jul 06 20:15:19.254759 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:19.254722 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6n5wd"] Jul 06 20:15:19.254957 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:19.254831 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6n5wd" Jul 06 20:15:19.257816 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:19.257790 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 06 20:15:19.258048 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:19.258034 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-wt4k5\"" Jul 06 20:15:19.258239 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:19.258224 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 06 20:15:19.311818 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:19.311780 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86979\" (UniqueName: \"kubernetes.io/projected/e5560d38-60f3-4fab-83b4-1ef9041cbd6a-kube-api-access-86979\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6n5wd\" (UID: \"e5560d38-60f3-4fab-83b4-1ef9041cbd6a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6n5wd" Jul 06 20:15:19.311818 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:19.311815 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e5560d38-60f3-4fab-83b4-1ef9041cbd6a-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6n5wd\" (UID: \"e5560d38-60f3-4fab-83b4-1ef9041cbd6a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6n5wd" Jul 06 20:15:19.312073 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:19.311840 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e5560d38-60f3-4fab-83b4-1ef9041cbd6a-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6n5wd\" (UID: \"e5560d38-60f3-4fab-83b4-1ef9041cbd6a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6n5wd" Jul 06 20:15:19.412575 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:19.412542 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86979\" (UniqueName: \"kubernetes.io/projected/e5560d38-60f3-4fab-83b4-1ef9041cbd6a-kube-api-access-86979\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6n5wd\" (UID: \"e5560d38-60f3-4fab-83b4-1ef9041cbd6a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6n5wd" Jul 06 20:15:19.412575 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:19.412582 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e5560d38-60f3-4fab-83b4-1ef9041cbd6a-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6n5wd\" (UID: \"e5560d38-60f3-4fab-83b4-1ef9041cbd6a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6n5wd" Jul 06 20:15:19.412822 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:19.412604 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e5560d38-60f3-4fab-83b4-1ef9041cbd6a-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6n5wd\" (UID: \"e5560d38-60f3-4fab-83b4-1ef9041cbd6a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6n5wd" Jul 06 20:15:19.413054 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:19.413036 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e5560d38-60f3-4fab-83b4-1ef9041cbd6a-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6n5wd\" (UID: \"e5560d38-60f3-4fab-83b4-1ef9041cbd6a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6n5wd" Jul 06 20:15:19.413123 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:19.413039 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e5560d38-60f3-4fab-83b4-1ef9041cbd6a-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6n5wd\" (UID: \"e5560d38-60f3-4fab-83b4-1ef9041cbd6a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6n5wd" Jul 06 20:15:19.423166 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:19.423138 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-86979\" (UniqueName: \"kubernetes.io/projected/e5560d38-60f3-4fab-83b4-1ef9041cbd6a-kube-api-access-86979\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6n5wd\" (UID: \"e5560d38-60f3-4fab-83b4-1ef9041cbd6a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6n5wd" Jul 06 20:15:19.564774 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:19.564729 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6n5wd" Jul 06 20:15:19.720500 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:19.720477 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6n5wd"] Jul 06 20:15:19.723030 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:15:19.722996 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5560d38_60f3_4fab_83b4_1ef9041cbd6a.slice/crio-1c110e876bbd2242e1751ab2df593cb9c6c0db4c5a4209520ae6938c6dbec7b5 WatchSource:0}: Error finding container 1c110e876bbd2242e1751ab2df593cb9c6c0db4c5a4209520ae6938c6dbec7b5: Status 404 returned error can't find the container with id 1c110e876bbd2242e1751ab2df593cb9c6c0db4c5a4209520ae6938c6dbec7b5 Jul 06 20:15:20.064858 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:20.064818 2551 generic.go:356] "Generic (PLEG): container finished" podID="e5560d38-60f3-4fab-83b4-1ef9041cbd6a" containerID="df20ea69bdbcbe2a97adb8aabc4de59b4276177085a4caa9f153d9e50aab276a" exitCode=0 Jul 06 20:15:20.065225 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:20.064927 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6n5wd" event={"ID":"e5560d38-60f3-4fab-83b4-1ef9041cbd6a","Type":"ContainerDied","Data":"df20ea69bdbcbe2a97adb8aabc4de59b4276177085a4caa9f153d9e50aab276a"} Jul 06 20:15:20.065225 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:20.064978 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6n5wd" event={"ID":"e5560d38-60f3-4fab-83b4-1ef9041cbd6a","Type":"ContainerStarted","Data":"1c110e876bbd2242e1751ab2df593cb9c6c0db4c5a4209520ae6938c6dbec7b5"} Jul 06 20:15:22.075153 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:22.075114 2551 generic.go:356] "Generic (PLEG): container finished" podID="e5560d38-60f3-4fab-83b4-1ef9041cbd6a" containerID="aa4b797262d214bc96254e0ef5fc3b45003f4f621396a56665888fe8cc1f2d61" exitCode=0 Jul 06 20:15:22.075602 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:22.075199 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6n5wd" event={"ID":"e5560d38-60f3-4fab-83b4-1ef9041cbd6a","Type":"ContainerDied","Data":"aa4b797262d214bc96254e0ef5fc3b45003f4f621396a56665888fe8cc1f2d61"} Jul 06 20:15:23.081810 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:23.081774 2551 generic.go:356] "Generic (PLEG): container finished" podID="e5560d38-60f3-4fab-83b4-1ef9041cbd6a" containerID="7e96ad4ad6c31bb049f02195e7eec34513d880ea7c91c5ff3cdfa68c6b338f05" exitCode=0 Jul 06 20:15:23.082258 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:23.081865 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6n5wd" event={"ID":"e5560d38-60f3-4fab-83b4-1ef9041cbd6a","Type":"ContainerDied","Data":"7e96ad4ad6c31bb049f02195e7eec34513d880ea7c91c5ff3cdfa68c6b338f05"} Jul 06 20:15:23.572793 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:23.572759 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-ffprl"] Jul 06 20:15:23.586302 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:23.586270 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-ffprl" Jul 06 20:15:23.588768 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:23.588742 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-ffprl"] Jul 06 20:15:23.589083 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:23.589049 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-pmdpb\"" Jul 06 20:15:23.652099 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:23.652057 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5n468\" (UniqueName: \"kubernetes.io/projected/d922885b-872a-47ac-8bc1-96c6478473ec-kube-api-access-5n468\") pod \"cert-manager-55b4d954d5-ffprl\" (UID: \"d922885b-872a-47ac-8bc1-96c6478473ec\") " pod="cert-manager/cert-manager-55b4d954d5-ffprl" Jul 06 20:15:23.652298 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:23.652119 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d922885b-872a-47ac-8bc1-96c6478473ec-bound-sa-token\") pod \"cert-manager-55b4d954d5-ffprl\" (UID: \"d922885b-872a-47ac-8bc1-96c6478473ec\") " pod="cert-manager/cert-manager-55b4d954d5-ffprl" Jul 06 20:15:23.752769 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:23.752731 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5n468\" (UniqueName: \"kubernetes.io/projected/d922885b-872a-47ac-8bc1-96c6478473ec-kube-api-access-5n468\") pod \"cert-manager-55b4d954d5-ffprl\" (UID: \"d922885b-872a-47ac-8bc1-96c6478473ec\") " pod="cert-manager/cert-manager-55b4d954d5-ffprl" Jul 06 20:15:23.752994 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:23.752785 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d922885b-872a-47ac-8bc1-96c6478473ec-bound-sa-token\") pod \"cert-manager-55b4d954d5-ffprl\" (UID: \"d922885b-872a-47ac-8bc1-96c6478473ec\") " pod="cert-manager/cert-manager-55b4d954d5-ffprl" Jul 06 20:15:23.765594 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:23.765566 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5n468\" (UniqueName: \"kubernetes.io/projected/d922885b-872a-47ac-8bc1-96c6478473ec-kube-api-access-5n468\") pod \"cert-manager-55b4d954d5-ffprl\" (UID: \"d922885b-872a-47ac-8bc1-96c6478473ec\") " pod="cert-manager/cert-manager-55b4d954d5-ffprl" Jul 06 20:15:23.767050 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:23.767033 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d922885b-872a-47ac-8bc1-96c6478473ec-bound-sa-token\") pod \"cert-manager-55b4d954d5-ffprl\" (UID: \"d922885b-872a-47ac-8bc1-96c6478473ec\") " pod="cert-manager/cert-manager-55b4d954d5-ffprl" Jul 06 20:15:23.896704 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:23.896609 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-ffprl" Jul 06 20:15:24.035750 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:24.035715 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-ffprl"] Jul 06 20:15:24.039017 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:15:24.038985 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd922885b_872a_47ac_8bc1_96c6478473ec.slice/crio-cf95eef0718787ab247a10a690187c5dc3610fd36dc56a87efff8900561c3403 WatchSource:0}: Error finding container cf95eef0718787ab247a10a690187c5dc3610fd36dc56a87efff8900561c3403: Status 404 returned error can't find the container with id cf95eef0718787ab247a10a690187c5dc3610fd36dc56a87efff8900561c3403 Jul 06 20:15:24.088637 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:24.088608 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-ffprl" event={"ID":"d922885b-872a-47ac-8bc1-96c6478473ec","Type":"ContainerStarted","Data":"cf95eef0718787ab247a10a690187c5dc3610fd36dc56a87efff8900561c3403"} Jul 06 20:15:24.206692 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:24.206667 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6n5wd" Jul 06 20:15:24.257417 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:24.257379 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-86979\" (UniqueName: \"kubernetes.io/projected/e5560d38-60f3-4fab-83b4-1ef9041cbd6a-kube-api-access-86979\") pod \"e5560d38-60f3-4fab-83b4-1ef9041cbd6a\" (UID: \"e5560d38-60f3-4fab-83b4-1ef9041cbd6a\") " Jul 06 20:15:24.257618 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:24.257469 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e5560d38-60f3-4fab-83b4-1ef9041cbd6a-bundle\") pod \"e5560d38-60f3-4fab-83b4-1ef9041cbd6a\" (UID: \"e5560d38-60f3-4fab-83b4-1ef9041cbd6a\") " Jul 06 20:15:24.257689 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:24.257612 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e5560d38-60f3-4fab-83b4-1ef9041cbd6a-util\") pod \"e5560d38-60f3-4fab-83b4-1ef9041cbd6a\" (UID: \"e5560d38-60f3-4fab-83b4-1ef9041cbd6a\") " Jul 06 20:15:24.258276 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:24.258252 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5560d38-60f3-4fab-83b4-1ef9041cbd6a-bundle" (OuterVolumeSpecName: "bundle") pod "e5560d38-60f3-4fab-83b4-1ef9041cbd6a" (UID: "e5560d38-60f3-4fab-83b4-1ef9041cbd6a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:15:24.260021 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:24.259997 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5560d38-60f3-4fab-83b4-1ef9041cbd6a-kube-api-access-86979" (OuterVolumeSpecName: "kube-api-access-86979") pod "e5560d38-60f3-4fab-83b4-1ef9041cbd6a" (UID: "e5560d38-60f3-4fab-83b4-1ef9041cbd6a"). InnerVolumeSpecName "kube-api-access-86979". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:15:24.264271 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:24.264233 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5560d38-60f3-4fab-83b4-1ef9041cbd6a-util" (OuterVolumeSpecName: "util") pod "e5560d38-60f3-4fab-83b4-1ef9041cbd6a" (UID: "e5560d38-60f3-4fab-83b4-1ef9041cbd6a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:15:24.358869 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:24.358818 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-86979\" (UniqueName: \"kubernetes.io/projected/e5560d38-60f3-4fab-83b4-1ef9041cbd6a-kube-api-access-86979\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:15:24.358869 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:24.358859 2551 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e5560d38-60f3-4fab-83b4-1ef9041cbd6a-bundle\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:15:24.358869 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:24.358873 2551 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e5560d38-60f3-4fab-83b4-1ef9041cbd6a-util\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:15:25.094681 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:25.094584 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-ffprl" event={"ID":"d922885b-872a-47ac-8bc1-96c6478473ec","Type":"ContainerStarted","Data":"99ea4bce73d3c5b6616306e29cc5e6b941d97779df5937d503d98b40d1087a27"} Jul 06 20:15:25.096438 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:25.096411 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6n5wd" event={"ID":"e5560d38-60f3-4fab-83b4-1ef9041cbd6a","Type":"ContainerDied","Data":"1c110e876bbd2242e1751ab2df593cb9c6c0db4c5a4209520ae6938c6dbec7b5"} Jul 06 20:15:25.096438 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:25.096431 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6n5wd" Jul 06 20:15:25.096585 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:25.096490 2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1c110e876bbd2242e1751ab2df593cb9c6c0db4c5a4209520ae6938c6dbec7b5" Jul 06 20:15:25.114909 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:25.114847 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-ffprl" podStartSLOduration=2.11483296 podStartE2EDuration="2.11483296s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:15:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:15:25.113027751 +0000 UTC m=+435.528227597" watchObservedRunningTime="2026-07-06 20:15:25.11483296 +0000 UTC m=+435.530032765" Jul 06 20:15:36.388355 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:36.388278 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7f6455564d-rwbgc"] Jul 06 20:15:36.388742 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:36.388614 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5560d38-60f3-4fab-83b4-1ef9041cbd6a" containerName="pull" Jul 06 20:15:36.388742 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:36.388625 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5560d38-60f3-4fab-83b4-1ef9041cbd6a" containerName="pull" Jul 06 20:15:36.388742 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:36.388637 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5560d38-60f3-4fab-83b4-1ef9041cbd6a" containerName="extract" Jul 06 20:15:36.388742 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:36.388644 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5560d38-60f3-4fab-83b4-1ef9041cbd6a" containerName="extract" Jul 06 20:15:36.388742 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:36.388655 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5560d38-60f3-4fab-83b4-1ef9041cbd6a" containerName="util" Jul 06 20:15:36.388742 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:36.388661 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5560d38-60f3-4fab-83b4-1ef9041cbd6a" containerName="util" Jul 06 20:15:36.388742 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:36.388744 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5560d38-60f3-4fab-83b4-1ef9041cbd6a" containerName="extract" Jul 06 20:15:36.390757 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:36.390739 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7f6455564d-rwbgc" Jul 06 20:15:36.394375 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:36.394350 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 06 20:15:36.396012 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:36.395990 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 06 20:15:36.396428 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:36.396407 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 06 20:15:36.397098 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:36.397077 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 06 20:15:36.397098 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:36.397090 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-mlfx8\"" Jul 06 20:15:36.422331 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:36.422296 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7f6455564d-rwbgc"] Jul 06 20:15:36.461670 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:36.461635 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4p4m\" (UniqueName: \"kubernetes.io/projected/944d69e8-62db-410d-b9e6-a9e476aef550-kube-api-access-h4p4m\") pod \"opendatahub-operator-controller-manager-7f6455564d-rwbgc\" (UID: \"944d69e8-62db-410d-b9e6-a9e476aef550\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6455564d-rwbgc" Jul 06 20:15:36.461843 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:36.461691 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/944d69e8-62db-410d-b9e6-a9e476aef550-webhook-cert\") pod \"opendatahub-operator-controller-manager-7f6455564d-rwbgc\" (UID: \"944d69e8-62db-410d-b9e6-a9e476aef550\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6455564d-rwbgc" Jul 06 20:15:36.461843 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:36.461724 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/944d69e8-62db-410d-b9e6-a9e476aef550-tmp\") pod \"opendatahub-operator-controller-manager-7f6455564d-rwbgc\" (UID: \"944d69e8-62db-410d-b9e6-a9e476aef550\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6455564d-rwbgc" Jul 06 20:15:36.461938 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:36.461840 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/944d69e8-62db-410d-b9e6-a9e476aef550-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7f6455564d-rwbgc\" (UID: \"944d69e8-62db-410d-b9e6-a9e476aef550\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6455564d-rwbgc" Jul 06 20:15:36.461938 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:36.461888 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/944d69e8-62db-410d-b9e6-a9e476aef550-manifests\") pod \"opendatahub-operator-controller-manager-7f6455564d-rwbgc\" (UID: \"944d69e8-62db-410d-b9e6-a9e476aef550\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6455564d-rwbgc" Jul 06 20:15:36.562862 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:36.562815 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/944d69e8-62db-410d-b9e6-a9e476aef550-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7f6455564d-rwbgc\" (UID: \"944d69e8-62db-410d-b9e6-a9e476aef550\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6455564d-rwbgc" Jul 06 20:15:36.563105 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:36.562880 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/944d69e8-62db-410d-b9e6-a9e476aef550-manifests\") pod \"opendatahub-operator-controller-manager-7f6455564d-rwbgc\" (UID: \"944d69e8-62db-410d-b9e6-a9e476aef550\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6455564d-rwbgc" Jul 06 20:15:36.563105 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:36.562940 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4p4m\" (UniqueName: \"kubernetes.io/projected/944d69e8-62db-410d-b9e6-a9e476aef550-kube-api-access-h4p4m\") pod \"opendatahub-operator-controller-manager-7f6455564d-rwbgc\" (UID: \"944d69e8-62db-410d-b9e6-a9e476aef550\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6455564d-rwbgc" Jul 06 20:15:36.563105 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:36.562975 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/944d69e8-62db-410d-b9e6-a9e476aef550-webhook-cert\") pod \"opendatahub-operator-controller-manager-7f6455564d-rwbgc\" (UID: \"944d69e8-62db-410d-b9e6-a9e476aef550\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6455564d-rwbgc" Jul 06 20:15:36.563105 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:36.563027 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/944d69e8-62db-410d-b9e6-a9e476aef550-tmp\") pod \"opendatahub-operator-controller-manager-7f6455564d-rwbgc\" (UID: \"944d69e8-62db-410d-b9e6-a9e476aef550\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6455564d-rwbgc" Jul 06 20:15:36.563355 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:36.563328 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/944d69e8-62db-410d-b9e6-a9e476aef550-manifests\") pod \"opendatahub-operator-controller-manager-7f6455564d-rwbgc\" (UID: \"944d69e8-62db-410d-b9e6-a9e476aef550\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6455564d-rwbgc" Jul 06 20:15:36.565302 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:36.565274 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/944d69e8-62db-410d-b9e6-a9e476aef550-tmp\") pod \"opendatahub-operator-controller-manager-7f6455564d-rwbgc\" (UID: \"944d69e8-62db-410d-b9e6-a9e476aef550\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6455564d-rwbgc" Jul 06 20:15:36.565525 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:36.565506 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/944d69e8-62db-410d-b9e6-a9e476aef550-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7f6455564d-rwbgc\" (UID: \"944d69e8-62db-410d-b9e6-a9e476aef550\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6455564d-rwbgc" Jul 06 20:15:36.565568 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:36.565527 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/944d69e8-62db-410d-b9e6-a9e476aef550-webhook-cert\") pod \"opendatahub-operator-controller-manager-7f6455564d-rwbgc\" (UID: \"944d69e8-62db-410d-b9e6-a9e476aef550\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6455564d-rwbgc" Jul 06 20:15:36.578480 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:36.578446 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4p4m\" (UniqueName: \"kubernetes.io/projected/944d69e8-62db-410d-b9e6-a9e476aef550-kube-api-access-h4p4m\") pod \"opendatahub-operator-controller-manager-7f6455564d-rwbgc\" (UID: \"944d69e8-62db-410d-b9e6-a9e476aef550\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6455564d-rwbgc" Jul 06 20:15:36.701423 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:36.701326 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7f6455564d-rwbgc" Jul 06 20:15:36.840633 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:36.840601 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7f6455564d-rwbgc"] Jul 06 20:15:36.842499 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:15:36.842475 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod944d69e8_62db_410d_b9e6_a9e476aef550.slice/crio-75b7c9639e627a14edbd86c8b4596e49bd7663f6df0544f45bef5b82355d7323 WatchSource:0}: Error finding container 75b7c9639e627a14edbd86c8b4596e49bd7663f6df0544f45bef5b82355d7323: Status 404 returned error can't find the container with id 75b7c9639e627a14edbd86c8b4596e49bd7663f6df0544f45bef5b82355d7323 Jul 06 20:15:37.151204 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:37.151165 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7f6455564d-rwbgc" event={"ID":"944d69e8-62db-410d-b9e6-a9e476aef550","Type":"ContainerStarted","Data":"75b7c9639e627a14edbd86c8b4596e49bd7663f6df0544f45bef5b82355d7323"} Jul 06 20:15:41.177059 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:41.177024 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7f6455564d-rwbgc" event={"ID":"944d69e8-62db-410d-b9e6-a9e476aef550","Type":"ContainerStarted","Data":"cebae19ccf3fa758bdfc21445a9c3e25f011b7348f3fee9bcd1c0191700fb492"} Jul 06 20:15:42.182937 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:42.182884 2551 generic.go:356] "Generic (PLEG): container finished" podID="944d69e8-62db-410d-b9e6-a9e476aef550" containerID="cebae19ccf3fa758bdfc21445a9c3e25f011b7348f3fee9bcd1c0191700fb492" exitCode=0 Jul 06 20:15:42.182937 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:42.182923 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7f6455564d-rwbgc" event={"ID":"944d69e8-62db-410d-b9e6-a9e476aef550","Type":"ContainerDied","Data":"cebae19ccf3fa758bdfc21445a9c3e25f011b7348f3fee9bcd1c0191700fb492"} Jul 06 20:15:43.189990 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:43.189948 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7f6455564d-rwbgc" event={"ID":"944d69e8-62db-410d-b9e6-a9e476aef550","Type":"ContainerStarted","Data":"b40bad3b58e7f1a2723ac650cdcac7b48289ab12c746e6de58fbe6b2a42b9f16"} Jul 06 20:15:43.190429 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:43.190075 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7f6455564d-rwbgc" Jul 06 20:15:43.226753 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:43.226693 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7f6455564d-rwbgc" podStartSLOduration=2.490270051 podStartE2EDuration="7.226675419s" totalImagesPullingTime="3.67125748s" totalInitContainerRuntime="1.065147888s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:15:36 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:15:43.223328925 +0000 UTC m=+453.638528751" watchObservedRunningTime="2026-07-06 20:15:43.226675419 +0000 UTC m=+453.641875291" Jul 06 20:15:54.196490 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:54.196455 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7f6455564d-rwbgc" Jul 06 20:15:58.157251 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:58.157214 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-68b56f95f5-6krcp"] Jul 06 20:15:58.173095 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:58.173051 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-68b56f95f5-6krcp" Jul 06 20:15:58.201467 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:58.201424 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 06 20:15:58.201467 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:58.201424 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 06 20:15:58.208707 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:58.208674 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-68b56f95f5-6krcp"] Jul 06 20:15:58.283268 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:58.283229 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-r5svr"] Jul 06 20:15:58.285972 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:58.285946 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-r5svr" Jul 06 20:15:58.289645 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:58.289616 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 06 20:15:58.290163 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:58.290146 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-bv782\"" Jul 06 20:15:58.319876 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:58.319840 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-r5svr"] Jul 06 20:15:58.362179 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:58.362137 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fad5f926-d0b6-43b0-b124-92a9d7478625-tmp\") pod \"kube-auth-proxy-68b56f95f5-6krcp\" (UID: \"fad5f926-d0b6-43b0-b124-92a9d7478625\") " pod="openshift-ingress/kube-auth-proxy-68b56f95f5-6krcp" Jul 06 20:15:58.362179 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:58.362183 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fad5f926-d0b6-43b0-b124-92a9d7478625-tls-certs\") pod \"kube-auth-proxy-68b56f95f5-6krcp\" (UID: \"fad5f926-d0b6-43b0-b124-92a9d7478625\") " pod="openshift-ingress/kube-auth-proxy-68b56f95f5-6krcp" Jul 06 20:15:58.362390 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:58.362212 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrlfx\" (UniqueName: \"kubernetes.io/projected/fad5f926-d0b6-43b0-b124-92a9d7478625-kube-api-access-rrlfx\") pod \"kube-auth-proxy-68b56f95f5-6krcp\" (UID: \"fad5f926-d0b6-43b0-b124-92a9d7478625\") " pod="openshift-ingress/kube-auth-proxy-68b56f95f5-6krcp" Jul 06 20:15:58.462795 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:58.462703 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dbbc\" (UniqueName: \"kubernetes.io/projected/e7240fb9-24ab-412a-b7dc-d12f95a61060-kube-api-access-2dbbc\") pod \"odh-model-controller-7998c67dc4-r5svr\" (UID: \"e7240fb9-24ab-412a-b7dc-d12f95a61060\") " pod="opendatahub/odh-model-controller-7998c67dc4-r5svr" Jul 06 20:15:58.462795 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:58.462748 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e7240fb9-24ab-412a-b7dc-d12f95a61060-cert\") pod \"odh-model-controller-7998c67dc4-r5svr\" (UID: \"e7240fb9-24ab-412a-b7dc-d12f95a61060\") " pod="opendatahub/odh-model-controller-7998c67dc4-r5svr" Jul 06 20:15:58.463028 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:58.462805 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fad5f926-d0b6-43b0-b124-92a9d7478625-tmp\") pod \"kube-auth-proxy-68b56f95f5-6krcp\" (UID: \"fad5f926-d0b6-43b0-b124-92a9d7478625\") " pod="openshift-ingress/kube-auth-proxy-68b56f95f5-6krcp" Jul 06 20:15:58.463028 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:58.462849 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fad5f926-d0b6-43b0-b124-92a9d7478625-tls-certs\") pod \"kube-auth-proxy-68b56f95f5-6krcp\" (UID: \"fad5f926-d0b6-43b0-b124-92a9d7478625\") " pod="openshift-ingress/kube-auth-proxy-68b56f95f5-6krcp" Jul 06 20:15:58.463028 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:58.462868 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrlfx\" (UniqueName: \"kubernetes.io/projected/fad5f926-d0b6-43b0-b124-92a9d7478625-kube-api-access-rrlfx\") pod \"kube-auth-proxy-68b56f95f5-6krcp\" (UID: \"fad5f926-d0b6-43b0-b124-92a9d7478625\") " pod="openshift-ingress/kube-auth-proxy-68b56f95f5-6krcp" Jul 06 20:15:58.465272 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:58.465250 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fad5f926-d0b6-43b0-b124-92a9d7478625-tmp\") pod \"kube-auth-proxy-68b56f95f5-6krcp\" (UID: \"fad5f926-d0b6-43b0-b124-92a9d7478625\") " pod="openshift-ingress/kube-auth-proxy-68b56f95f5-6krcp" Jul 06 20:15:58.465402 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:58.465384 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fad5f926-d0b6-43b0-b124-92a9d7478625-tls-certs\") pod \"kube-auth-proxy-68b56f95f5-6krcp\" (UID: \"fad5f926-d0b6-43b0-b124-92a9d7478625\") " pod="openshift-ingress/kube-auth-proxy-68b56f95f5-6krcp" Jul 06 20:15:58.531677 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:58.531647 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrlfx\" (UniqueName: \"kubernetes.io/projected/fad5f926-d0b6-43b0-b124-92a9d7478625-kube-api-access-rrlfx\") pod \"kube-auth-proxy-68b56f95f5-6krcp\" (UID: \"fad5f926-d0b6-43b0-b124-92a9d7478625\") " pod="openshift-ingress/kube-auth-proxy-68b56f95f5-6krcp" Jul 06 20:15:58.563410 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:58.563373 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dbbc\" (UniqueName: \"kubernetes.io/projected/e7240fb9-24ab-412a-b7dc-d12f95a61060-kube-api-access-2dbbc\") pod \"odh-model-controller-7998c67dc4-r5svr\" (UID: \"e7240fb9-24ab-412a-b7dc-d12f95a61060\") " pod="opendatahub/odh-model-controller-7998c67dc4-r5svr" Jul 06 20:15:58.563629 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:58.563436 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e7240fb9-24ab-412a-b7dc-d12f95a61060-cert\") pod \"odh-model-controller-7998c67dc4-r5svr\" (UID: \"e7240fb9-24ab-412a-b7dc-d12f95a61060\") " pod="opendatahub/odh-model-controller-7998c67dc4-r5svr" Jul 06 20:15:58.563629 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:15:58.563562 2551 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 06 20:15:58.563743 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:15:58.563651 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7240fb9-24ab-412a-b7dc-d12f95a61060-cert podName:e7240fb9-24ab-412a-b7dc-d12f95a61060 nodeName:}" failed. No retries permitted until 2026-07-06 20:15:59.0636303 +0000 UTC m=+469.478830087 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/e7240fb9-24ab-412a-b7dc-d12f95a61060-cert") pod "odh-model-controller-7998c67dc4-r5svr" (UID: "e7240fb9-24ab-412a-b7dc-d12f95a61060") : secret "odh-model-controller-webhook-cert" not found Jul 06 20:15:58.595561 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:58.595528 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dbbc\" (UniqueName: \"kubernetes.io/projected/e7240fb9-24ab-412a-b7dc-d12f95a61060-kube-api-access-2dbbc\") pod \"odh-model-controller-7998c67dc4-r5svr\" (UID: \"e7240fb9-24ab-412a-b7dc-d12f95a61060\") " pod="opendatahub/odh-model-controller-7998c67dc4-r5svr" Jul 06 20:15:58.783853 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:58.783814 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-68b56f95f5-6krcp" Jul 06 20:15:58.936291 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:58.936264 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-68b56f95f5-6krcp"] Jul 06 20:15:58.938469 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:15:58.938440 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfad5f926_d0b6_43b0_b124_92a9d7478625.slice/crio-9af24d8b577b70a10104f71c50e2be36f156d1c4f408ecf07ccd8268369553e3 WatchSource:0}: Error finding container 9af24d8b577b70a10104f71c50e2be36f156d1c4f408ecf07ccd8268369553e3: Status 404 returned error can't find the container with id 9af24d8b577b70a10104f71c50e2be36f156d1c4f408ecf07ccd8268369553e3 Jul 06 20:15:59.067788 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:59.067688 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e7240fb9-24ab-412a-b7dc-d12f95a61060-cert\") pod \"odh-model-controller-7998c67dc4-r5svr\" (UID: \"e7240fb9-24ab-412a-b7dc-d12f95a61060\") " pod="opendatahub/odh-model-controller-7998c67dc4-r5svr" Jul 06 20:15:59.070186 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:59.070159 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e7240fb9-24ab-412a-b7dc-d12f95a61060-cert\") pod \"odh-model-controller-7998c67dc4-r5svr\" (UID: \"e7240fb9-24ab-412a-b7dc-d12f95a61060\") " pod="opendatahub/odh-model-controller-7998c67dc4-r5svr" Jul 06 20:15:59.196637 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:59.196598 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-r5svr" Jul 06 20:15:59.264205 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:59.264172 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-68b56f95f5-6krcp" event={"ID":"fad5f926-d0b6-43b0-b124-92a9d7478625","Type":"ContainerStarted","Data":"9af24d8b577b70a10104f71c50e2be36f156d1c4f408ecf07ccd8268369553e3"} Jul 06 20:15:59.348985 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:15:59.348945 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-r5svr"] Jul 06 20:15:59.350662 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:15:59.350634 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7240fb9_24ab_412a_b7dc_d12f95a61060.slice/crio-5d91ebc104a5e8a3e7a8f619e8e6a3f43dbbf69cf0a5bb841e3860c504fa39a8 WatchSource:0}: Error finding container 5d91ebc104a5e8a3e7a8f619e8e6a3f43dbbf69cf0a5bb841e3860c504fa39a8: Status 404 returned error can't find the container with id 5d91ebc104a5e8a3e7a8f619e8e6a3f43dbbf69cf0a5bb841e3860c504fa39a8 Jul 06 20:16:00.038371 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:00.038335 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-gp85h"] Jul 06 20:16:00.041085 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:00.041059 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-gp85h" Jul 06 20:16:00.045372 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:00.045351 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-h2ldq\"" Jul 06 20:16:00.058068 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:00.058038 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 06 20:16:00.149471 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:00.149432 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-gp85h"] Jul 06 20:16:00.176753 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:00.176716 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmbjp\" (UniqueName: \"kubernetes.io/projected/3fe7b2ec-32e7-4b5e-bfad-7f0f1ff752c2-kube-api-access-zmbjp\") pod \"model-serving-api-54d89bd5fc-gp85h\" (UID: \"3fe7b2ec-32e7-4b5e-bfad-7f0f1ff752c2\") " pod="opendatahub/model-serving-api-54d89bd5fc-gp85h" Jul 06 20:16:00.177034 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:00.176762 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3fe7b2ec-32e7-4b5e-bfad-7f0f1ff752c2-tls-certs\") pod \"model-serving-api-54d89bd5fc-gp85h\" (UID: \"3fe7b2ec-32e7-4b5e-bfad-7f0f1ff752c2\") " pod="opendatahub/model-serving-api-54d89bd5fc-gp85h" Jul 06 20:16:00.270994 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:00.270951 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-r5svr" event={"ID":"e7240fb9-24ab-412a-b7dc-d12f95a61060","Type":"ContainerStarted","Data":"5d91ebc104a5e8a3e7a8f619e8e6a3f43dbbf69cf0a5bb841e3860c504fa39a8"} Jul 06 20:16:00.277484 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:00.277446 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmbjp\" (UniqueName: \"kubernetes.io/projected/3fe7b2ec-32e7-4b5e-bfad-7f0f1ff752c2-kube-api-access-zmbjp\") pod \"model-serving-api-54d89bd5fc-gp85h\" (UID: \"3fe7b2ec-32e7-4b5e-bfad-7f0f1ff752c2\") " pod="opendatahub/model-serving-api-54d89bd5fc-gp85h" Jul 06 20:16:00.277675 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:00.277506 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3fe7b2ec-32e7-4b5e-bfad-7f0f1ff752c2-tls-certs\") pod \"model-serving-api-54d89bd5fc-gp85h\" (UID: \"3fe7b2ec-32e7-4b5e-bfad-7f0f1ff752c2\") " pod="opendatahub/model-serving-api-54d89bd5fc-gp85h" Jul 06 20:16:00.280317 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:00.280288 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3fe7b2ec-32e7-4b5e-bfad-7f0f1ff752c2-tls-certs\") pod \"model-serving-api-54d89bd5fc-gp85h\" (UID: \"3fe7b2ec-32e7-4b5e-bfad-7f0f1ff752c2\") " pod="opendatahub/model-serving-api-54d89bd5fc-gp85h" Jul 06 20:16:00.303601 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:00.303524 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmbjp\" (UniqueName: \"kubernetes.io/projected/3fe7b2ec-32e7-4b5e-bfad-7f0f1ff752c2-kube-api-access-zmbjp\") pod \"model-serving-api-54d89bd5fc-gp85h\" (UID: \"3fe7b2ec-32e7-4b5e-bfad-7f0f1ff752c2\") " pod="opendatahub/model-serving-api-54d89bd5fc-gp85h" Jul 06 20:16:00.353059 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:00.353017 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-gp85h" Jul 06 20:16:00.580647 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:00.580562 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-x2f4v"] Jul 06 20:16:00.584378 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:00.584352 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-x2f4v" Jul 06 20:16:00.587151 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:00.587129 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 06 20:16:00.588495 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:00.588449 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 06 20:16:00.602202 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:00.601397 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-qpfbw\"" Jul 06 20:16:00.602202 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:00.601642 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 06 20:16:00.602202 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:00.601817 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 06 20:16:00.606311 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:00.606282 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-gp85h"] Jul 06 20:16:00.624011 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:00.623936 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-x2f4v"] Jul 06 20:16:00.782110 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:00.782070 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/234ae895-a749-43ab-8d7a-4ad2098e79c8-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-x2f4v\" (UID: \"234ae895-a749-43ab-8d7a-4ad2098e79c8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x2f4v" Jul 06 20:16:00.782283 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:00.782121 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/234ae895-a749-43ab-8d7a-4ad2098e79c8-cacerts\") pod \"istiod-openshift-gateway-7574459656-x2f4v\" (UID: \"234ae895-a749-43ab-8d7a-4ad2098e79c8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x2f4v" Jul 06 20:16:00.782283 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:00.782150 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xsmkz\" (UniqueName: \"kubernetes.io/projected/234ae895-a749-43ab-8d7a-4ad2098e79c8-kube-api-access-xsmkz\") pod \"istiod-openshift-gateway-7574459656-x2f4v\" (UID: \"234ae895-a749-43ab-8d7a-4ad2098e79c8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x2f4v" Jul 06 20:16:00.782283 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:00.782214 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/234ae895-a749-43ab-8d7a-4ad2098e79c8-local-certs\") pod \"istiod-openshift-gateway-7574459656-x2f4v\" (UID: \"234ae895-a749-43ab-8d7a-4ad2098e79c8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x2f4v" Jul 06 20:16:00.782283 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:00.782243 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/234ae895-a749-43ab-8d7a-4ad2098e79c8-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-x2f4v\" (UID: \"234ae895-a749-43ab-8d7a-4ad2098e79c8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x2f4v" Jul 06 20:16:00.782417 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:00.782290 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/234ae895-a749-43ab-8d7a-4ad2098e79c8-istio-token\") pod \"istiod-openshift-gateway-7574459656-x2f4v\" (UID: \"234ae895-a749-43ab-8d7a-4ad2098e79c8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x2f4v" Jul 06 20:16:00.782417 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:00.782350 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/234ae895-a749-43ab-8d7a-4ad2098e79c8-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-x2f4v\" (UID: \"234ae895-a749-43ab-8d7a-4ad2098e79c8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x2f4v" Jul 06 20:16:00.883338 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:00.883240 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/234ae895-a749-43ab-8d7a-4ad2098e79c8-istio-token\") pod \"istiod-openshift-gateway-7574459656-x2f4v\" (UID: \"234ae895-a749-43ab-8d7a-4ad2098e79c8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x2f4v" Jul 06 20:16:00.883512 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:00.883343 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/234ae895-a749-43ab-8d7a-4ad2098e79c8-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-x2f4v\" (UID: \"234ae895-a749-43ab-8d7a-4ad2098e79c8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x2f4v" Jul 06 20:16:00.883512 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:00.883390 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/234ae895-a749-43ab-8d7a-4ad2098e79c8-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-x2f4v\" (UID: \"234ae895-a749-43ab-8d7a-4ad2098e79c8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x2f4v" Jul 06 20:16:00.883512 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:00.883433 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/234ae895-a749-43ab-8d7a-4ad2098e79c8-cacerts\") pod \"istiod-openshift-gateway-7574459656-x2f4v\" (UID: \"234ae895-a749-43ab-8d7a-4ad2098e79c8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x2f4v" Jul 06 20:16:00.883512 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:00.883456 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xsmkz\" (UniqueName: \"kubernetes.io/projected/234ae895-a749-43ab-8d7a-4ad2098e79c8-kube-api-access-xsmkz\") pod \"istiod-openshift-gateway-7574459656-x2f4v\" (UID: \"234ae895-a749-43ab-8d7a-4ad2098e79c8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x2f4v" Jul 06 20:16:00.883709 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:00.883604 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/234ae895-a749-43ab-8d7a-4ad2098e79c8-local-certs\") pod \"istiod-openshift-gateway-7574459656-x2f4v\" (UID: \"234ae895-a749-43ab-8d7a-4ad2098e79c8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x2f4v" Jul 06 20:16:00.883709 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:00.883650 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/234ae895-a749-43ab-8d7a-4ad2098e79c8-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-x2f4v\" (UID: \"234ae895-a749-43ab-8d7a-4ad2098e79c8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x2f4v" Jul 06 20:16:00.884199 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:00.884145 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/234ae895-a749-43ab-8d7a-4ad2098e79c8-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-x2f4v\" (UID: \"234ae895-a749-43ab-8d7a-4ad2098e79c8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x2f4v" Jul 06 20:16:00.886581 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:00.886508 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/234ae895-a749-43ab-8d7a-4ad2098e79c8-local-certs\") pod \"istiod-openshift-gateway-7574459656-x2f4v\" (UID: \"234ae895-a749-43ab-8d7a-4ad2098e79c8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x2f4v" Jul 06 20:16:00.886724 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:00.886607 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/234ae895-a749-43ab-8d7a-4ad2098e79c8-cacerts\") pod \"istiod-openshift-gateway-7574459656-x2f4v\" (UID: \"234ae895-a749-43ab-8d7a-4ad2098e79c8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x2f4v" Jul 06 20:16:00.886724 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:00.886693 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/234ae895-a749-43ab-8d7a-4ad2098e79c8-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-x2f4v\" (UID: \"234ae895-a749-43ab-8d7a-4ad2098e79c8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x2f4v" Jul 06 20:16:00.886842 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:00.886741 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/234ae895-a749-43ab-8d7a-4ad2098e79c8-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-x2f4v\" (UID: \"234ae895-a749-43ab-8d7a-4ad2098e79c8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x2f4v" Jul 06 20:16:00.896849 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:00.896812 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/234ae895-a749-43ab-8d7a-4ad2098e79c8-istio-token\") pod \"istiod-openshift-gateway-7574459656-x2f4v\" (UID: \"234ae895-a749-43ab-8d7a-4ad2098e79c8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x2f4v" Jul 06 20:16:00.897488 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:00.897443 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xsmkz\" (UniqueName: \"kubernetes.io/projected/234ae895-a749-43ab-8d7a-4ad2098e79c8-kube-api-access-xsmkz\") pod \"istiod-openshift-gateway-7574459656-x2f4v\" (UID: \"234ae895-a749-43ab-8d7a-4ad2098e79c8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x2f4v" Jul 06 20:16:00.899363 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:00.899338 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-x2f4v" Jul 06 20:16:01.094758 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:01.094727 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-x2f4v"] Jul 06 20:16:01.096617 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:16:01.096586 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod234ae895_a749_43ab_8d7a_4ad2098e79c8.slice/crio-74019feb2b37e865103232e7b1cc9a0b3bd57054b332bb8a60c7f83c2342d037 WatchSource:0}: Error finding container 74019feb2b37e865103232e7b1cc9a0b3bd57054b332bb8a60c7f83c2342d037: Status 404 returned error can't find the container with id 74019feb2b37e865103232e7b1cc9a0b3bd57054b332bb8a60c7f83c2342d037 Jul 06 20:16:01.278073 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:01.278012 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-x2f4v" event={"ID":"234ae895-a749-43ab-8d7a-4ad2098e79c8","Type":"ContainerStarted","Data":"74019feb2b37e865103232e7b1cc9a0b3bd57054b332bb8a60c7f83c2342d037"} Jul 06 20:16:01.279545 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:01.279503 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-gp85h" event={"ID":"3fe7b2ec-32e7-4b5e-bfad-7f0f1ff752c2","Type":"ContainerStarted","Data":"53ffcdfb410c35602b5380cc206759bb495f2075cfac765f360daaa83c492a3d"} Jul 06 20:16:04.855960 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:04.855906 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:16:04.856305 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:04.856007 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:16:05.308065 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:05.308027 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-x2f4v" event={"ID":"234ae895-a749-43ab-8d7a-4ad2098e79c8","Type":"ContainerStarted","Data":"78ea86a0f98c2e4e2fe4ff0c8e82b4f171a9df7217d94a966401613ea3182cbf"} Jul 06 20:16:05.308306 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:05.308223 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-x2f4v" Jul 06 20:16:05.310002 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:05.309968 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-gp85h" event={"ID":"3fe7b2ec-32e7-4b5e-bfad-7f0f1ff752c2","Type":"ContainerStarted","Data":"75503c7d6523ac8077a5e5b854392ba2709071ec31612d154c389f40bac1d754"} Jul 06 20:16:05.310115 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:05.310045 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-gp85h" Jul 06 20:16:05.310570 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:05.310541 2551 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-x2f4v container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 06 20:16:05.310823 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:05.310613 2551 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-x2f4v" podUID="234ae895-a749-43ab-8d7a-4ad2098e79c8" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 20:16:05.311829 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:05.311798 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-68b56f95f5-6krcp" event={"ID":"fad5f926-d0b6-43b0-b124-92a9d7478625","Type":"ContainerStarted","Data":"542a8883b5a8c6f4b0e1d05b2fe1858341a116a8467b9f860db07c4c7509839d"} Jul 06 20:16:05.313354 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:05.313332 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-r5svr" event={"ID":"e7240fb9-24ab-412a-b7dc-d12f95a61060","Type":"ContainerStarted","Data":"897c98b9aa6412ddb15ee326a6abf6493ebec3ef197b9da22a4d922a4f20f350"} Jul 06 20:16:05.313545 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:05.313528 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-r5svr" Jul 06 20:16:05.352150 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:05.352083 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-x2f4v" podStartSLOduration=1.5956460890000002 podStartE2EDuration="5.352058767s" totalImagesPullingTime="3.756412678s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:16:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:16:05.348972949 +0000 UTC m=+475.764172775" watchObservedRunningTime="2026-07-06 20:16:05.352058767 +0000 UTC m=+475.767258590" Jul 06 20:16:05.374452 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:05.374399 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-68b56f95f5-6krcp" podStartSLOduration=1.470834198 podStartE2EDuration="7.374381023s" totalImagesPullingTime="5.903546825s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:15:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:16:05.371044749 +0000 UTC m=+475.786244601" watchObservedRunningTime="2026-07-06 20:16:05.374381023 +0000 UTC m=+475.789580880" Jul 06 20:16:05.404258 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:05.404158 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-r5svr" podStartSLOduration=1.9008212439999999 podStartE2EDuration="7.404134477s" totalImagesPullingTime="5.503313233s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:15:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:16:05.401090569 +0000 UTC m=+475.816290374" watchObservedRunningTime="2026-07-06 20:16:05.404134477 +0000 UTC m=+475.819334284" Jul 06 20:16:05.432560 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:05.432493 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-gp85h" podStartSLOduration=2.193956807 podStartE2EDuration="6.432476759s" totalImagesPullingTime="4.238519952s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:15:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:16:05.429210527 +0000 UTC m=+475.844410352" watchObservedRunningTime="2026-07-06 20:16:05.432476759 +0000 UTC m=+475.847676564" Jul 06 20:16:06.319075 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:06.319028 2551 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-x2f4v container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 06 20:16:06.319485 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:06.319101 2551 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-x2f4v" podUID="234ae895-a749-43ab-8d7a-4ad2098e79c8" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 20:16:07.035444 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:07.035410 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xwrmt"] Jul 06 20:16:07.038601 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:07.038581 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xwrmt" Jul 06 20:16:07.041791 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:07.041762 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-4tdr5\"" Jul 06 20:16:07.055905 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:07.055858 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xwrmt"] Jul 06 20:16:07.144961 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:07.144921 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6dd6cbd6-1df4-4233-bca9-6e44587c8759-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678xwrmt\" (UID: \"6dd6cbd6-1df4-4233-bca9-6e44587c8759\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xwrmt" Jul 06 20:16:07.145118 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:07.144964 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6dd6cbd6-1df4-4233-bca9-6e44587c8759-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678xwrmt\" (UID: \"6dd6cbd6-1df4-4233-bca9-6e44587c8759\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xwrmt" Jul 06 20:16:07.145118 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:07.145021 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6dd6cbd6-1df4-4233-bca9-6e44587c8759-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678xwrmt\" (UID: \"6dd6cbd6-1df4-4233-bca9-6e44587c8759\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xwrmt" Jul 06 20:16:07.145118 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:07.145058 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpf89\" (UniqueName: \"kubernetes.io/projected/6dd6cbd6-1df4-4233-bca9-6e44587c8759-kube-api-access-wpf89\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678xwrmt\" (UID: \"6dd6cbd6-1df4-4233-bca9-6e44587c8759\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xwrmt" Jul 06 20:16:07.145118 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:07.145107 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6dd6cbd6-1df4-4233-bca9-6e44587c8759-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678xwrmt\" (UID: \"6dd6cbd6-1df4-4233-bca9-6e44587c8759\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xwrmt" Jul 06 20:16:07.145262 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:07.145169 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6dd6cbd6-1df4-4233-bca9-6e44587c8759-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678xwrmt\" (UID: \"6dd6cbd6-1df4-4233-bca9-6e44587c8759\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xwrmt" Jul 06 20:16:07.145262 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:07.145193 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dd6cbd6-1df4-4233-bca9-6e44587c8759-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678xwrmt\" (UID: \"6dd6cbd6-1df4-4233-bca9-6e44587c8759\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xwrmt" Jul 06 20:16:07.145262 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:07.145227 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6dd6cbd6-1df4-4233-bca9-6e44587c8759-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678xwrmt\" (UID: \"6dd6cbd6-1df4-4233-bca9-6e44587c8759\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xwrmt" Jul 06 20:16:07.145262 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:07.145254 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dd6cbd6-1df4-4233-bca9-6e44587c8759-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678xwrmt\" (UID: \"6dd6cbd6-1df4-4233-bca9-6e44587c8759\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xwrmt" Jul 06 20:16:07.246193 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:07.246148 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6dd6cbd6-1df4-4233-bca9-6e44587c8759-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678xwrmt\" (UID: \"6dd6cbd6-1df4-4233-bca9-6e44587c8759\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xwrmt" Jul 06 20:16:07.246193 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:07.246195 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6dd6cbd6-1df4-4233-bca9-6e44587c8759-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678xwrmt\" (UID: \"6dd6cbd6-1df4-4233-bca9-6e44587c8759\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xwrmt" Jul 06 20:16:07.246475 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:07.246239 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6dd6cbd6-1df4-4233-bca9-6e44587c8759-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678xwrmt\" (UID: \"6dd6cbd6-1df4-4233-bca9-6e44587c8759\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xwrmt" Jul 06 20:16:07.246475 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:07.246266 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpf89\" (UniqueName: \"kubernetes.io/projected/6dd6cbd6-1df4-4233-bca9-6e44587c8759-kube-api-access-wpf89\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678xwrmt\" (UID: \"6dd6cbd6-1df4-4233-bca9-6e44587c8759\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xwrmt" Jul 06 20:16:07.246475 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:07.246318 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6dd6cbd6-1df4-4233-bca9-6e44587c8759-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678xwrmt\" (UID: \"6dd6cbd6-1df4-4233-bca9-6e44587c8759\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xwrmt" Jul 06 20:16:07.246621 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:07.246506 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6dd6cbd6-1df4-4233-bca9-6e44587c8759-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678xwrmt\" (UID: \"6dd6cbd6-1df4-4233-bca9-6e44587c8759\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xwrmt" Jul 06 20:16:07.246621 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:07.246569 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dd6cbd6-1df4-4233-bca9-6e44587c8759-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678xwrmt\" (UID: \"6dd6cbd6-1df4-4233-bca9-6e44587c8759\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xwrmt" Jul 06 20:16:07.246722 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:07.246620 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6dd6cbd6-1df4-4233-bca9-6e44587c8759-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678xwrmt\" (UID: \"6dd6cbd6-1df4-4233-bca9-6e44587c8759\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xwrmt" Jul 06 20:16:07.246722 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:07.246667 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dd6cbd6-1df4-4233-bca9-6e44587c8759-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678xwrmt\" (UID: \"6dd6cbd6-1df4-4233-bca9-6e44587c8759\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xwrmt" Jul 06 20:16:07.246722 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:07.246683 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6dd6cbd6-1df4-4233-bca9-6e44587c8759-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678xwrmt\" (UID: \"6dd6cbd6-1df4-4233-bca9-6e44587c8759\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xwrmt" Jul 06 20:16:07.247026 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:07.247003 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6dd6cbd6-1df4-4233-bca9-6e44587c8759-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678xwrmt\" (UID: \"6dd6cbd6-1df4-4233-bca9-6e44587c8759\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xwrmt" Jul 06 20:16:07.247120 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:07.247001 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dd6cbd6-1df4-4233-bca9-6e44587c8759-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678xwrmt\" (UID: \"6dd6cbd6-1df4-4233-bca9-6e44587c8759\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xwrmt" Jul 06 20:16:07.247120 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:07.247086 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dd6cbd6-1df4-4233-bca9-6e44587c8759-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678xwrmt\" (UID: \"6dd6cbd6-1df4-4233-bca9-6e44587c8759\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xwrmt" Jul 06 20:16:07.247321 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:07.247298 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6dd6cbd6-1df4-4233-bca9-6e44587c8759-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678xwrmt\" (UID: \"6dd6cbd6-1df4-4233-bca9-6e44587c8759\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xwrmt" Jul 06 20:16:07.248536 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:07.248519 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6dd6cbd6-1df4-4233-bca9-6e44587c8759-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678xwrmt\" (UID: \"6dd6cbd6-1df4-4233-bca9-6e44587c8759\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xwrmt" Jul 06 20:16:07.248843 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:07.248824 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6dd6cbd6-1df4-4233-bca9-6e44587c8759-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678xwrmt\" (UID: \"6dd6cbd6-1df4-4233-bca9-6e44587c8759\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xwrmt" Jul 06 20:16:07.283362 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:07.283326 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpf89\" (UniqueName: \"kubernetes.io/projected/6dd6cbd6-1df4-4233-bca9-6e44587c8759-kube-api-access-wpf89\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678xwrmt\" (UID: \"6dd6cbd6-1df4-4233-bca9-6e44587c8759\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xwrmt" Jul 06 20:16:07.307921 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:07.307824 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6dd6cbd6-1df4-4233-bca9-6e44587c8759-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678xwrmt\" (UID: \"6dd6cbd6-1df4-4233-bca9-6e44587c8759\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xwrmt" Jul 06 20:16:07.348738 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:07.348705 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xwrmt" Jul 06 20:16:07.602437 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:07.602412 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xwrmt"] Jul 06 20:16:07.604447 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:16:07.604413 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dd6cbd6_1df4_4233_bca9_6e44587c8759.slice/crio-7ce6a3874ab3071b1f81e88830343491a2288e280f258e0c74a6a2f52d7085fc WatchSource:0}: Error finding container 7ce6a3874ab3071b1f81e88830343491a2288e280f258e0c74a6a2f52d7085fc: Status 404 returned error can't find the container with id 7ce6a3874ab3071b1f81e88830343491a2288e280f258e0c74a6a2f52d7085fc Jul 06 20:16:08.329950 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:08.329916 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xwrmt" event={"ID":"6dd6cbd6-1df4-4233-bca9-6e44587c8759","Type":"ContainerStarted","Data":"7ce6a3874ab3071b1f81e88830343491a2288e280f258e0c74a6a2f52d7085fc"} Jul 06 20:16:09.319406 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:09.319373 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-x2f4v" Jul 06 20:16:10.825992 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:10.825954 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-7qd2m"] Jul 06 20:16:10.828576 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:10.828550 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7qd2m" Jul 06 20:16:10.834485 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:10.834459 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 06 20:16:10.835077 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:10.834862 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-4kpzq\"" Jul 06 20:16:10.873926 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:10.873878 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-7qd2m"] Jul 06 20:16:10.882927 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:10.882866 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c0ee877c-70e6-4960-9924-c3aeb9f21455-cert\") pod \"kserve-controller-manager-d6f4b65b8-7qd2m\" (UID: \"c0ee877c-70e6-4960-9924-c3aeb9f21455\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7qd2m" Jul 06 20:16:10.883116 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:10.882930 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjhh5\" (UniqueName: \"kubernetes.io/projected/c0ee877c-70e6-4960-9924-c3aeb9f21455-kube-api-access-cjhh5\") pod \"kserve-controller-manager-d6f4b65b8-7qd2m\" (UID: \"c0ee877c-70e6-4960-9924-c3aeb9f21455\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7qd2m" Jul 06 20:16:10.903795 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:10.903751 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-6nlkr"] Jul 06 20:16:10.906653 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:10.906626 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6nlkr" Jul 06 20:16:10.910013 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:10.909949 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-tsdlw\"" Jul 06 20:16:10.910152 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:10.910102 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 06 20:16:10.947660 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:10.947623 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-6nlkr"] Jul 06 20:16:10.984571 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:10.984532 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c0ee877c-70e6-4960-9924-c3aeb9f21455-cert\") pod \"kserve-controller-manager-d6f4b65b8-7qd2m\" (UID: \"c0ee877c-70e6-4960-9924-c3aeb9f21455\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7qd2m" Jul 06 20:16:10.984571 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:10.984576 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjhh5\" (UniqueName: \"kubernetes.io/projected/c0ee877c-70e6-4960-9924-c3aeb9f21455-kube-api-access-cjhh5\") pod \"kserve-controller-manager-d6f4b65b8-7qd2m\" (UID: \"c0ee877c-70e6-4960-9924-c3aeb9f21455\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7qd2m" Jul 06 20:16:10.984805 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:10.984725 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a6ef5c09-60d4-44fd-b968-f0bd1770147d-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-6nlkr\" (UID: \"a6ef5c09-60d4-44fd-b968-f0bd1770147d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6nlkr" Jul 06 20:16:10.984805 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:10.984797 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjx8w\" (UniqueName: \"kubernetes.io/projected/a6ef5c09-60d4-44fd-b968-f0bd1770147d-kube-api-access-tjx8w\") pod \"llmisvc-controller-manager-777c6bd7d5-6nlkr\" (UID: \"a6ef5c09-60d4-44fd-b968-f0bd1770147d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6nlkr" Jul 06 20:16:10.987536 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:10.987507 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c0ee877c-70e6-4960-9924-c3aeb9f21455-cert\") pod \"kserve-controller-manager-d6f4b65b8-7qd2m\" (UID: \"c0ee877c-70e6-4960-9924-c3aeb9f21455\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7qd2m" Jul 06 20:16:11.004012 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:11.003979 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjhh5\" (UniqueName: \"kubernetes.io/projected/c0ee877c-70e6-4960-9924-c3aeb9f21455-kube-api-access-cjhh5\") pod \"kserve-controller-manager-d6f4b65b8-7qd2m\" (UID: \"c0ee877c-70e6-4960-9924-c3aeb9f21455\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7qd2m" Jul 06 20:16:11.086010 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:11.085887 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a6ef5c09-60d4-44fd-b968-f0bd1770147d-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-6nlkr\" (UID: \"a6ef5c09-60d4-44fd-b968-f0bd1770147d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6nlkr" Jul 06 20:16:11.086166 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:11.086011 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjx8w\" (UniqueName: \"kubernetes.io/projected/a6ef5c09-60d4-44fd-b968-f0bd1770147d-kube-api-access-tjx8w\") pod \"llmisvc-controller-manager-777c6bd7d5-6nlkr\" (UID: \"a6ef5c09-60d4-44fd-b968-f0bd1770147d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6nlkr" Jul 06 20:16:11.088572 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:11.088546 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a6ef5c09-60d4-44fd-b968-f0bd1770147d-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-6nlkr\" (UID: \"a6ef5c09-60d4-44fd-b968-f0bd1770147d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6nlkr" Jul 06 20:16:11.115276 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:11.115246 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjx8w\" (UniqueName: \"kubernetes.io/projected/a6ef5c09-60d4-44fd-b968-f0bd1770147d-kube-api-access-tjx8w\") pod \"llmisvc-controller-manager-777c6bd7d5-6nlkr\" (UID: \"a6ef5c09-60d4-44fd-b968-f0bd1770147d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6nlkr" Jul 06 20:16:11.141311 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:11.141277 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7qd2m" Jul 06 20:16:11.220532 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:11.220489 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6nlkr" Jul 06 20:16:11.616880 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:11.616842 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-x2f4v"] Jul 06 20:16:11.617120 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:11.617093 2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-x2f4v" podUID="234ae895-a749-43ab-8d7a-4ad2098e79c8" containerName="discovery" containerID="cri-o://78ea86a0f98c2e4e2fe4ff0c8e82b4f171a9df7217d94a966401613ea3182cbf" gracePeriod=30 Jul 06 20:16:12.099999 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:12.099526 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:16:12.099999 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:12.099631 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:16:12.099999 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:12.099671 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:16:12.217833 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:12.217810 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-x2f4v" Jul 06 20:16:12.297772 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:12.297662 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/234ae895-a749-43ab-8d7a-4ad2098e79c8-istio-token\") pod \"234ae895-a749-43ab-8d7a-4ad2098e79c8\" (UID: \"234ae895-a749-43ab-8d7a-4ad2098e79c8\") " Jul 06 20:16:12.297772 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:12.297719 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/234ae895-a749-43ab-8d7a-4ad2098e79c8-istio-kubeconfig\") pod \"234ae895-a749-43ab-8d7a-4ad2098e79c8\" (UID: \"234ae895-a749-43ab-8d7a-4ad2098e79c8\") " Jul 06 20:16:12.297772 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:12.297738 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/234ae895-a749-43ab-8d7a-4ad2098e79c8-cacerts\") pod \"234ae895-a749-43ab-8d7a-4ad2098e79c8\" (UID: \"234ae895-a749-43ab-8d7a-4ad2098e79c8\") " Jul 06 20:16:12.298150 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:12.297869 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/234ae895-a749-43ab-8d7a-4ad2098e79c8-istio-csr-dns-cert\") pod \"234ae895-a749-43ab-8d7a-4ad2098e79c8\" (UID: \"234ae895-a749-43ab-8d7a-4ad2098e79c8\") " Jul 06 20:16:12.298150 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:12.297936 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/234ae895-a749-43ab-8d7a-4ad2098e79c8-istio-csr-ca-configmap\") pod \"234ae895-a749-43ab-8d7a-4ad2098e79c8\" (UID: \"234ae895-a749-43ab-8d7a-4ad2098e79c8\") " Jul 06 20:16:12.298150 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:12.297969 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/234ae895-a749-43ab-8d7a-4ad2098e79c8-local-certs\") pod \"234ae895-a749-43ab-8d7a-4ad2098e79c8\" (UID: \"234ae895-a749-43ab-8d7a-4ad2098e79c8\") " Jul 06 20:16:12.298150 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:12.298049 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xsmkz\" (UniqueName: \"kubernetes.io/projected/234ae895-a749-43ab-8d7a-4ad2098e79c8-kube-api-access-xsmkz\") pod \"234ae895-a749-43ab-8d7a-4ad2098e79c8\" (UID: \"234ae895-a749-43ab-8d7a-4ad2098e79c8\") " Jul 06 20:16:12.298403 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:12.298374 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/234ae895-a749-43ab-8d7a-4ad2098e79c8-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "234ae895-a749-43ab-8d7a-4ad2098e79c8" (UID: "234ae895-a749-43ab-8d7a-4ad2098e79c8"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:16:12.300383 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:12.300348 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/234ae895-a749-43ab-8d7a-4ad2098e79c8-istio-token" (OuterVolumeSpecName: "istio-token") pod "234ae895-a749-43ab-8d7a-4ad2098e79c8" (UID: "234ae895-a749-43ab-8d7a-4ad2098e79c8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:16:12.300493 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:12.300429 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/234ae895-a749-43ab-8d7a-4ad2098e79c8-cacerts" (OuterVolumeSpecName: "cacerts") pod "234ae895-a749-43ab-8d7a-4ad2098e79c8" (UID: "234ae895-a749-43ab-8d7a-4ad2098e79c8"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 20:16:12.300493 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:12.300450 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/234ae895-a749-43ab-8d7a-4ad2098e79c8-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "234ae895-a749-43ab-8d7a-4ad2098e79c8" (UID: "234ae895-a749-43ab-8d7a-4ad2098e79c8"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 20:16:12.300579 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:12.300524 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/234ae895-a749-43ab-8d7a-4ad2098e79c8-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "234ae895-a749-43ab-8d7a-4ad2098e79c8" (UID: "234ae895-a749-43ab-8d7a-4ad2098e79c8"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 20:16:12.300708 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:12.300688 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/234ae895-a749-43ab-8d7a-4ad2098e79c8-kube-api-access-xsmkz" (OuterVolumeSpecName: "kube-api-access-xsmkz") pod "234ae895-a749-43ab-8d7a-4ad2098e79c8" (UID: "234ae895-a749-43ab-8d7a-4ad2098e79c8"). InnerVolumeSpecName "kube-api-access-xsmkz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:16:12.300795 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:12.300776 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/234ae895-a749-43ab-8d7a-4ad2098e79c8-local-certs" (OuterVolumeSpecName: "local-certs") pod "234ae895-a749-43ab-8d7a-4ad2098e79c8" (UID: "234ae895-a749-43ab-8d7a-4ad2098e79c8"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:16:12.354098 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:12.354056 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xwrmt" event={"ID":"6dd6cbd6-1df4-4233-bca9-6e44587c8759","Type":"ContainerStarted","Data":"bf1addab79ebc0e4f5ea62e97bc7a6569a7f017b7270cdbec1b3f7cc4ad58a54"} Jul 06 20:16:12.355589 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:12.355556 2551 generic.go:356] "Generic (PLEG): container finished" podID="234ae895-a749-43ab-8d7a-4ad2098e79c8" containerID="78ea86a0f98c2e4e2fe4ff0c8e82b4f171a9df7217d94a966401613ea3182cbf" exitCode=0 Jul 06 20:16:12.355753 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:12.355606 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-x2f4v" Jul 06 20:16:12.355753 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:12.355629 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-x2f4v" event={"ID":"234ae895-a749-43ab-8d7a-4ad2098e79c8","Type":"ContainerDied","Data":"78ea86a0f98c2e4e2fe4ff0c8e82b4f171a9df7217d94a966401613ea3182cbf"} Jul 06 20:16:12.355753 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:12.355674 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-x2f4v" event={"ID":"234ae895-a749-43ab-8d7a-4ad2098e79c8","Type":"ContainerDied","Data":"74019feb2b37e865103232e7b1cc9a0b3bd57054b332bb8a60c7f83c2342d037"} Jul 06 20:16:12.355753 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:12.355696 2551 scope.go:117] "RemoveContainer" containerID="78ea86a0f98c2e4e2fe4ff0c8e82b4f171a9df7217d94a966401613ea3182cbf" Jul 06 20:16:12.364708 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:12.364680 2551 scope.go:117] "RemoveContainer" containerID="78ea86a0f98c2e4e2fe4ff0c8e82b4f171a9df7217d94a966401613ea3182cbf" Jul 06 20:16:12.365126 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:16:12.365106 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78ea86a0f98c2e4e2fe4ff0c8e82b4f171a9df7217d94a966401613ea3182cbf\": container with ID starting with 78ea86a0f98c2e4e2fe4ff0c8e82b4f171a9df7217d94a966401613ea3182cbf not found: ID does not exist" containerID="78ea86a0f98c2e4e2fe4ff0c8e82b4f171a9df7217d94a966401613ea3182cbf" Jul 06 20:16:12.365219 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:12.365134 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78ea86a0f98c2e4e2fe4ff0c8e82b4f171a9df7217d94a966401613ea3182cbf"} err="failed to get container status \"78ea86a0f98c2e4e2fe4ff0c8e82b4f171a9df7217d94a966401613ea3182cbf\": rpc error: code = NotFound desc = could not find container \"78ea86a0f98c2e4e2fe4ff0c8e82b4f171a9df7217d94a966401613ea3182cbf\": container with ID starting with 78ea86a0f98c2e4e2fe4ff0c8e82b4f171a9df7217d94a966401613ea3182cbf not found: ID does not exist" Jul 06 20:16:12.399963 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:12.399930 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xsmkz\" (UniqueName: \"kubernetes.io/projected/234ae895-a749-43ab-8d7a-4ad2098e79c8-kube-api-access-xsmkz\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:16:12.399963 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:12.399956 2551 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/234ae895-a749-43ab-8d7a-4ad2098e79c8-istio-token\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:16:12.399963 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:12.399967 2551 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/234ae895-a749-43ab-8d7a-4ad2098e79c8-istio-kubeconfig\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:16:12.399963 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:12.399975 2551 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/234ae895-a749-43ab-8d7a-4ad2098e79c8-cacerts\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:16:12.399963 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:12.399985 2551 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/234ae895-a749-43ab-8d7a-4ad2098e79c8-istio-csr-dns-cert\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:16:12.400329 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:12.399994 2551 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/234ae895-a749-43ab-8d7a-4ad2098e79c8-istio-csr-ca-configmap\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:16:12.400329 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:12.400020 2551 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/234ae895-a749-43ab-8d7a-4ad2098e79c8-local-certs\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:16:12.440589 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:12.440537 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xwrmt" podStartSLOduration=0.947873621 podStartE2EDuration="5.440519015s" totalImagesPullingTime="4.492645394s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:16:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:16:12.421162295 +0000 UTC m=+482.836362110" watchObservedRunningTime="2026-07-06 20:16:12.440519015 +0000 UTC m=+482.855718820" Jul 06 20:16:12.484578 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:12.484548 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-7qd2m"] Jul 06 20:16:12.486648 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:16:12.486621 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0ee877c_70e6_4960_9924_c3aeb9f21455.slice/crio-b38631b2a0461a7f5301cb1e356e4b19e661cf2aba78ed8f3b62bf08414ed38d WatchSource:0}: Error finding container b38631b2a0461a7f5301cb1e356e4b19e661cf2aba78ed8f3b62bf08414ed38d: Status 404 returned error can't find the container with id b38631b2a0461a7f5301cb1e356e4b19e661cf2aba78ed8f3b62bf08414ed38d Jul 06 20:16:12.498761 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:12.498704 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-x2f4v"] Jul 06 20:16:12.500042 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:16:12.500014 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda6ef5c09_60d4_44fd_b968_f0bd1770147d.slice/crio-7700d3cf557a1afb13fdbbf5283c5160fe5cb20ac9da208830810b634758f6b4 WatchSource:0}: Error finding container 7700d3cf557a1afb13fdbbf5283c5160fe5cb20ac9da208830810b634758f6b4: Status 404 returned error can't find the container with id 7700d3cf557a1afb13fdbbf5283c5160fe5cb20ac9da208830810b634758f6b4 Jul 06 20:16:12.500779 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:12.500758 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-6nlkr"] Jul 06 20:16:12.515638 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:12.515606 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-x2f4v"] Jul 06 20:16:13.349435 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:13.349350 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xwrmt" Jul 06 20:16:13.354296 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:13.354269 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xwrmt" Jul 06 20:16:13.365575 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:13.365539 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6nlkr" event={"ID":"a6ef5c09-60d4-44fd-b968-f0bd1770147d","Type":"ContainerStarted","Data":"7700d3cf557a1afb13fdbbf5283c5160fe5cb20ac9da208830810b634758f6b4"} Jul 06 20:16:13.366794 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:13.366764 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7qd2m" event={"ID":"c0ee877c-70e6-4960-9924-c3aeb9f21455","Type":"ContainerStarted","Data":"b38631b2a0461a7f5301cb1e356e4b19e661cf2aba78ed8f3b62bf08414ed38d"} Jul 06 20:16:13.367032 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:13.367015 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xwrmt" Jul 06 20:16:13.368118 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:13.368098 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678xwrmt" Jul 06 20:16:14.165688 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:14.165652 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="234ae895-a749-43ab-8d7a-4ad2098e79c8" path="/var/lib/kubelet/pods/234ae895-a749-43ab-8d7a-4ad2098e79c8/volumes" Jul 06 20:16:16.321032 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:16.320999 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-r5svr" Jul 06 20:16:16.322693 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:16.322673 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-gp85h" Jul 06 20:16:16.385873 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:16.385826 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7qd2m" event={"ID":"c0ee877c-70e6-4960-9924-c3aeb9f21455","Type":"ContainerStarted","Data":"5980636767d695740db537d9050fbc855505179df78a25c329643301c2f8cd99"} Jul 06 20:16:16.386086 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:16.385979 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7qd2m" Jul 06 20:16:16.448820 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:16.448635 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7qd2m" podStartSLOduration=3.005033082 podStartE2EDuration="6.448613877s" totalImagesPullingTime="3.443580795s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:16:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:16:16.446383651 +0000 UTC m=+486.861583456" watchObservedRunningTime="2026-07-06 20:16:16.448613877 +0000 UTC m=+486.863813683" Jul 06 20:16:17.392250 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:17.392215 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6nlkr" event={"ID":"a6ef5c09-60d4-44fd-b968-f0bd1770147d","Type":"ContainerStarted","Data":"e7aabe9510942135ab9bc6ad26b5324b355b76b2d7a0fde95c9b0c93ab14f09c"} Jul 06 20:16:17.392701 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:17.392356 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6nlkr" Jul 06 20:16:17.426146 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:17.426089 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6nlkr" podStartSLOduration=3.32831256 podStartE2EDuration="7.426072263s" totalImagesPullingTime="4.097759703s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:16:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:16:17.423158315 +0000 UTC m=+487.838358121" watchObservedRunningTime="2026-07-06 20:16:17.426072263 +0000 UTC m=+487.841272067" Jul 06 20:16:28.088421 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:28.088384 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-85d867cd45-829fc"] Jul 06 20:16:28.088883 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:28.088802 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="234ae895-a749-43ab-8d7a-4ad2098e79c8" containerName="discovery" Jul 06 20:16:28.088883 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:28.088814 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="234ae895-a749-43ab-8d7a-4ad2098e79c8" containerName="discovery" Jul 06 20:16:28.088883 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:28.088872 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="234ae895-a749-43ab-8d7a-4ad2098e79c8" containerName="discovery" Jul 06 20:16:28.092107 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:28.092091 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85d867cd45-829fc" Jul 06 20:16:28.095647 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:28.095616 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 06 20:16:28.095957 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:28.095941 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 06 20:16:28.096257 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:28.096243 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-jnph4\"" Jul 06 20:16:28.096311 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:28.096274 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 06 20:16:28.096575 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:28.096562 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 06 20:16:28.096625 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:28.096583 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 06 20:16:28.096947 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:28.096933 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 06 20:16:28.097137 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:28.097124 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 06 20:16:28.103277 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:28.103259 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 06 20:16:28.108802 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:28.108780 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85d867cd45-829fc"] Jul 06 20:16:28.258080 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:28.258045 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/febd04fa-ce6e-48ce-9d9e-461fc9885088-service-ca\") pod \"console-85d867cd45-829fc\" (UID: \"febd04fa-ce6e-48ce-9d9e-461fc9885088\") " pod="openshift-console/console-85d867cd45-829fc" Jul 06 20:16:28.258080 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:28.258082 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjt29\" (UniqueName: \"kubernetes.io/projected/febd04fa-ce6e-48ce-9d9e-461fc9885088-kube-api-access-fjt29\") pod \"console-85d867cd45-829fc\" (UID: \"febd04fa-ce6e-48ce-9d9e-461fc9885088\") " pod="openshift-console/console-85d867cd45-829fc" Jul 06 20:16:28.258295 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:28.258104 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/febd04fa-ce6e-48ce-9d9e-461fc9885088-oauth-serving-cert\") pod \"console-85d867cd45-829fc\" (UID: \"febd04fa-ce6e-48ce-9d9e-461fc9885088\") " pod="openshift-console/console-85d867cd45-829fc" Jul 06 20:16:28.258295 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:28.258219 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/febd04fa-ce6e-48ce-9d9e-461fc9885088-trusted-ca-bundle\") pod \"console-85d867cd45-829fc\" (UID: \"febd04fa-ce6e-48ce-9d9e-461fc9885088\") " pod="openshift-console/console-85d867cd45-829fc" Jul 06 20:16:28.258295 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:28.258258 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/febd04fa-ce6e-48ce-9d9e-461fc9885088-console-serving-cert\") pod \"console-85d867cd45-829fc\" (UID: \"febd04fa-ce6e-48ce-9d9e-461fc9885088\") " pod="openshift-console/console-85d867cd45-829fc" Jul 06 20:16:28.258394 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:28.258341 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/febd04fa-ce6e-48ce-9d9e-461fc9885088-console-config\") pod \"console-85d867cd45-829fc\" (UID: \"febd04fa-ce6e-48ce-9d9e-461fc9885088\") " pod="openshift-console/console-85d867cd45-829fc" Jul 06 20:16:28.258394 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:28.258367 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/febd04fa-ce6e-48ce-9d9e-461fc9885088-console-oauth-config\") pod \"console-85d867cd45-829fc\" (UID: \"febd04fa-ce6e-48ce-9d9e-461fc9885088\") " pod="openshift-console/console-85d867cd45-829fc" Jul 06 20:16:28.359720 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:28.359636 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/febd04fa-ce6e-48ce-9d9e-461fc9885088-console-config\") pod \"console-85d867cd45-829fc\" (UID: \"febd04fa-ce6e-48ce-9d9e-461fc9885088\") " pod="openshift-console/console-85d867cd45-829fc" Jul 06 20:16:28.359720 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:28.359672 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/febd04fa-ce6e-48ce-9d9e-461fc9885088-console-oauth-config\") pod \"console-85d867cd45-829fc\" (UID: \"febd04fa-ce6e-48ce-9d9e-461fc9885088\") " pod="openshift-console/console-85d867cd45-829fc" Jul 06 20:16:28.359720 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:28.359705 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/febd04fa-ce6e-48ce-9d9e-461fc9885088-service-ca\") pod \"console-85d867cd45-829fc\" (UID: \"febd04fa-ce6e-48ce-9d9e-461fc9885088\") " pod="openshift-console/console-85d867cd45-829fc" Jul 06 20:16:28.360015 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:28.359733 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjt29\" (UniqueName: \"kubernetes.io/projected/febd04fa-ce6e-48ce-9d9e-461fc9885088-kube-api-access-fjt29\") pod \"console-85d867cd45-829fc\" (UID: \"febd04fa-ce6e-48ce-9d9e-461fc9885088\") " pod="openshift-console/console-85d867cd45-829fc" Jul 06 20:16:28.360015 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:28.359760 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/febd04fa-ce6e-48ce-9d9e-461fc9885088-oauth-serving-cert\") pod \"console-85d867cd45-829fc\" (UID: \"febd04fa-ce6e-48ce-9d9e-461fc9885088\") " pod="openshift-console/console-85d867cd45-829fc" Jul 06 20:16:28.360015 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:28.359806 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/febd04fa-ce6e-48ce-9d9e-461fc9885088-trusted-ca-bundle\") pod \"console-85d867cd45-829fc\" (UID: \"febd04fa-ce6e-48ce-9d9e-461fc9885088\") " pod="openshift-console/console-85d867cd45-829fc" Jul 06 20:16:28.360015 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:28.359876 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/febd04fa-ce6e-48ce-9d9e-461fc9885088-console-serving-cert\") pod \"console-85d867cd45-829fc\" (UID: \"febd04fa-ce6e-48ce-9d9e-461fc9885088\") " pod="openshift-console/console-85d867cd45-829fc" Jul 06 20:16:28.360534 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:28.360510 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/febd04fa-ce6e-48ce-9d9e-461fc9885088-console-config\") pod \"console-85d867cd45-829fc\" (UID: \"febd04fa-ce6e-48ce-9d9e-461fc9885088\") " pod="openshift-console/console-85d867cd45-829fc" Jul 06 20:16:28.360651 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:28.360510 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/febd04fa-ce6e-48ce-9d9e-461fc9885088-service-ca\") pod \"console-85d867cd45-829fc\" (UID: \"febd04fa-ce6e-48ce-9d9e-461fc9885088\") " pod="openshift-console/console-85d867cd45-829fc" Jul 06 20:16:28.360651 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:28.360640 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/febd04fa-ce6e-48ce-9d9e-461fc9885088-trusted-ca-bundle\") pod \"console-85d867cd45-829fc\" (UID: \"febd04fa-ce6e-48ce-9d9e-461fc9885088\") " pod="openshift-console/console-85d867cd45-829fc" Jul 06 20:16:28.360731 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:28.360657 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/febd04fa-ce6e-48ce-9d9e-461fc9885088-oauth-serving-cert\") pod \"console-85d867cd45-829fc\" (UID: \"febd04fa-ce6e-48ce-9d9e-461fc9885088\") " pod="openshift-console/console-85d867cd45-829fc" Jul 06 20:16:28.362283 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:28.362265 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/febd04fa-ce6e-48ce-9d9e-461fc9885088-console-oauth-config\") pod \"console-85d867cd45-829fc\" (UID: \"febd04fa-ce6e-48ce-9d9e-461fc9885088\") " pod="openshift-console/console-85d867cd45-829fc" Jul 06 20:16:28.362388 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:28.362371 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/febd04fa-ce6e-48ce-9d9e-461fc9885088-console-serving-cert\") pod \"console-85d867cd45-829fc\" (UID: \"febd04fa-ce6e-48ce-9d9e-461fc9885088\") " pod="openshift-console/console-85d867cd45-829fc" Jul 06 20:16:28.371312 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:28.371281 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjt29\" (UniqueName: \"kubernetes.io/projected/febd04fa-ce6e-48ce-9d9e-461fc9885088-kube-api-access-fjt29\") pod \"console-85d867cd45-829fc\" (UID: \"febd04fa-ce6e-48ce-9d9e-461fc9885088\") " pod="openshift-console/console-85d867cd45-829fc" Jul 06 20:16:28.403455 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:28.403419 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85d867cd45-829fc" Jul 06 20:16:28.545010 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:28.544979 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85d867cd45-829fc"] Jul 06 20:16:28.546612 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:16:28.546588 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfebd04fa_ce6e_48ce_9d9e_461fc9885088.slice/crio-b4ac42df78ebd54c2a6c2bc7ca534b1ac8d089fee4d7726e5efefec5e11aadab WatchSource:0}: Error finding container b4ac42df78ebd54c2a6c2bc7ca534b1ac8d089fee4d7726e5efefec5e11aadab: Status 404 returned error can't find the container with id b4ac42df78ebd54c2a6c2bc7ca534b1ac8d089fee4d7726e5efefec5e11aadab Jul 06 20:16:29.457319 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:29.457283 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85d867cd45-829fc" event={"ID":"febd04fa-ce6e-48ce-9d9e-461fc9885088","Type":"ContainerStarted","Data":"c53a94ba898a3717be8bf270056abefd6fa77c89cd0ad4290dc19e597328bcbc"} Jul 06 20:16:29.457736 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:29.457331 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85d867cd45-829fc" event={"ID":"febd04fa-ce6e-48ce-9d9e-461fc9885088","Type":"ContainerStarted","Data":"b4ac42df78ebd54c2a6c2bc7ca534b1ac8d089fee4d7726e5efefec5e11aadab"} Jul 06 20:16:29.488315 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:29.488261 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-85d867cd45-829fc" podStartSLOduration=1.488245275 podStartE2EDuration="1.488245275s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:16:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:16:29.486708158 +0000 UTC m=+499.901907963" watchObservedRunningTime="2026-07-06 20:16:29.488245275 +0000 UTC m=+499.903445079" Jul 06 20:16:38.403551 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:38.403512 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-85d867cd45-829fc" Jul 06 20:16:38.404128 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:38.403597 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-85d867cd45-829fc" Jul 06 20:16:38.408417 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:38.408390 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-85d867cd45-829fc" Jul 06 20:16:38.506017 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:38.505991 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-85d867cd45-829fc" Jul 06 20:16:47.397802 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:47.397772 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7qd2m" Jul 06 20:16:48.400572 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:48.400539 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6nlkr" Jul 06 20:16:58.948679 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:58.948643 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6t2nb"] Jul 06 20:16:58.967937 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:58.967862 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6t2nb"] Jul 06 20:16:58.968125 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:58.968059 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-6t2nb" Jul 06 20:16:58.971154 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:58.971108 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 06 20:16:58.971987 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:58.971958 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-vx2l2\"" Jul 06 20:16:58.971987 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:58.971979 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 06 20:16:59.030195 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:59.030157 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szmkn\" (UniqueName: \"kubernetes.io/projected/0233ed4d-0601-49e9-86ea-abce553d89c8-kube-api-access-szmkn\") pod \"kuadrant-operator-catalog-6t2nb\" (UID: \"0233ed4d-0601-49e9-86ea-abce553d89c8\") " pod="kuadrant-system/kuadrant-operator-catalog-6t2nb" Jul 06 20:16:59.131599 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:59.131562 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szmkn\" (UniqueName: \"kubernetes.io/projected/0233ed4d-0601-49e9-86ea-abce553d89c8-kube-api-access-szmkn\") pod \"kuadrant-operator-catalog-6t2nb\" (UID: \"0233ed4d-0601-49e9-86ea-abce553d89c8\") " pod="kuadrant-system/kuadrant-operator-catalog-6t2nb" Jul 06 20:16:59.140495 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:59.140461 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-szmkn\" (UniqueName: \"kubernetes.io/projected/0233ed4d-0601-49e9-86ea-abce553d89c8-kube-api-access-szmkn\") pod \"kuadrant-operator-catalog-6t2nb\" (UID: \"0233ed4d-0601-49e9-86ea-abce553d89c8\") " pod="kuadrant-system/kuadrant-operator-catalog-6t2nb" Jul 06 20:16:59.279689 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:59.279655 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-6t2nb" Jul 06 20:16:59.378461 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:59.378425 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6t2nb"] Jul 06 20:16:59.416558 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:59.416517 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6t2nb"] Jul 06 20:16:59.420282 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:16:59.420240 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0233ed4d_0601_49e9_86ea_abce553d89c8.slice/crio-a85230080d9fcc561b07ec28e728dfa1d7a8e8a2007b9413b85ed26f2431fd81 WatchSource:0}: Error finding container a85230080d9fcc561b07ec28e728dfa1d7a8e8a2007b9413b85ed26f2431fd81: Status 404 returned error can't find the container with id a85230080d9fcc561b07ec28e728dfa1d7a8e8a2007b9413b85ed26f2431fd81 Jul 06 20:16:59.507978 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:59.507949 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6zck8"] Jul 06 20:16:59.511108 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:59.511091 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-6zck8" Jul 06 20:16:59.521675 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:59.521651 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6zck8"] Jul 06 20:16:59.607076 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:59.606990 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-6t2nb" event={"ID":"0233ed4d-0601-49e9-86ea-abce553d89c8","Type":"ContainerStarted","Data":"a85230080d9fcc561b07ec28e728dfa1d7a8e8a2007b9413b85ed26f2431fd81"} Jul 06 20:16:59.637495 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:59.637459 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndb5v\" (UniqueName: \"kubernetes.io/projected/286a63f8-e79e-434e-b82b-214061777c43-kube-api-access-ndb5v\") pod \"kuadrant-operator-catalog-6zck8\" (UID: \"286a63f8-e79e-434e-b82b-214061777c43\") " pod="kuadrant-system/kuadrant-operator-catalog-6zck8" Jul 06 20:16:59.738101 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:59.738069 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndb5v\" (UniqueName: \"kubernetes.io/projected/286a63f8-e79e-434e-b82b-214061777c43-kube-api-access-ndb5v\") pod \"kuadrant-operator-catalog-6zck8\" (UID: \"286a63f8-e79e-434e-b82b-214061777c43\") " pod="kuadrant-system/kuadrant-operator-catalog-6zck8" Jul 06 20:16:59.746470 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:59.746434 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndb5v\" (UniqueName: \"kubernetes.io/projected/286a63f8-e79e-434e-b82b-214061777c43-kube-api-access-ndb5v\") pod \"kuadrant-operator-catalog-6zck8\" (UID: \"286a63f8-e79e-434e-b82b-214061777c43\") " pod="kuadrant-system/kuadrant-operator-catalog-6zck8" Jul 06 20:16:59.821215 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:59.821176 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-6zck8" Jul 06 20:16:59.975206 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:16:59.975181 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6zck8"] Jul 06 20:16:59.992003 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:16:59.991970 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod286a63f8_e79e_434e_b82b_214061777c43.slice/crio-51bcde61a0c90b935a2f6b5bb261ca25db95985cca65ec87d6ae6d2b15f6bb47 WatchSource:0}: Error finding container 51bcde61a0c90b935a2f6b5bb261ca25db95985cca65ec87d6ae6d2b15f6bb47: Status 404 returned error can't find the container with id 51bcde61a0c90b935a2f6b5bb261ca25db95985cca65ec87d6ae6d2b15f6bb47 Jul 06 20:17:00.615081 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:00.615049 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-6zck8" event={"ID":"286a63f8-e79e-434e-b82b-214061777c43","Type":"ContainerStarted","Data":"51bcde61a0c90b935a2f6b5bb261ca25db95985cca65ec87d6ae6d2b15f6bb47"} Jul 06 20:17:01.624317 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:01.624282 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-6zck8" event={"ID":"286a63f8-e79e-434e-b82b-214061777c43","Type":"ContainerStarted","Data":"8529504f8f11a44034d93a0521785a9d5b07aa42fb0d7b1ccd02a25339a1a568"} Jul 06 20:17:01.625976 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:01.625940 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-6t2nb" event={"ID":"0233ed4d-0601-49e9-86ea-abce553d89c8","Type":"ContainerStarted","Data":"76eb1e69cbb90abd9f2b0e6ef6f81d336df090395ee968bcc2bf5fd08780d2bb"} Jul 06 20:17:01.626115 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:01.625995 2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-6t2nb" podUID="0233ed4d-0601-49e9-86ea-abce553d89c8" containerName="registry-server" containerID="cri-o://76eb1e69cbb90abd9f2b0e6ef6f81d336df090395ee968bcc2bf5fd08780d2bb" gracePeriod=2 Jul 06 20:17:01.649146 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:01.649078 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-6zck8" podStartSLOduration=1.124632958 podStartE2EDuration="2.64906087s" totalImagesPullingTime="1.524427912s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:16:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:17:01.646924931 +0000 UTC m=+532.062124758" watchObservedRunningTime="2026-07-06 20:17:01.64906087 +0000 UTC m=+532.064260675" Jul 06 20:17:01.664136 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:01.664081 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-6t2nb" podStartSLOduration=1.570045544 podStartE2EDuration="3.664063216s" totalImagesPullingTime="2.094017672s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:16:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:17:01.662197029 +0000 UTC m=+532.077396846" watchObservedRunningTime="2026-07-06 20:17:01.664063216 +0000 UTC m=+532.079263024" Jul 06 20:17:01.866316 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:01.866291 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-6t2nb_0233ed4d-0601-49e9-86ea-abce553d89c8/registry-server/0.log" Jul 06 20:17:01.867678 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:01.867662 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-6t2nb" Jul 06 20:17:01.961021 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:01.960988 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-szmkn\" (UniqueName: \"kubernetes.io/projected/0233ed4d-0601-49e9-86ea-abce553d89c8-kube-api-access-szmkn\") pod \"0233ed4d-0601-49e9-86ea-abce553d89c8\" (UID: \"0233ed4d-0601-49e9-86ea-abce553d89c8\") " Jul 06 20:17:01.963220 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:01.963189 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0233ed4d-0601-49e9-86ea-abce553d89c8-kube-api-access-szmkn" (OuterVolumeSpecName: "kube-api-access-szmkn") pod "0233ed4d-0601-49e9-86ea-abce553d89c8" (UID: "0233ed4d-0601-49e9-86ea-abce553d89c8"). InnerVolumeSpecName "kube-api-access-szmkn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:17:02.062104 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:02.062011 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-szmkn\" (UniqueName: \"kubernetes.io/projected/0233ed4d-0601-49e9-86ea-abce553d89c8-kube-api-access-szmkn\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:17:02.632948 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:02.632886 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-6t2nb_0233ed4d-0601-49e9-86ea-abce553d89c8/registry-server/0.log" Jul 06 20:17:02.633388 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:02.632958 2551 generic.go:356] "Generic (PLEG): container finished" podID="0233ed4d-0601-49e9-86ea-abce553d89c8" containerID="76eb1e69cbb90abd9f2b0e6ef6f81d336df090395ee968bcc2bf5fd08780d2bb" exitCode=1 Jul 06 20:17:02.633388 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:02.632990 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-6t2nb" event={"ID":"0233ed4d-0601-49e9-86ea-abce553d89c8","Type":"ContainerDied","Data":"76eb1e69cbb90abd9f2b0e6ef6f81d336df090395ee968bcc2bf5fd08780d2bb"} Jul 06 20:17:02.633388 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:02.633013 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-6t2nb" Jul 06 20:17:02.633388 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:02.633047 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-6t2nb" event={"ID":"0233ed4d-0601-49e9-86ea-abce553d89c8","Type":"ContainerDied","Data":"a85230080d9fcc561b07ec28e728dfa1d7a8e8a2007b9413b85ed26f2431fd81"} Jul 06 20:17:02.633388 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:02.633073 2551 scope.go:117] "RemoveContainer" containerID="76eb1e69cbb90abd9f2b0e6ef6f81d336df090395ee968bcc2bf5fd08780d2bb" Jul 06 20:17:02.642625 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:02.642603 2551 scope.go:117] "RemoveContainer" containerID="76eb1e69cbb90abd9f2b0e6ef6f81d336df090395ee968bcc2bf5fd08780d2bb" Jul 06 20:17:02.642939 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:17:02.642921 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76eb1e69cbb90abd9f2b0e6ef6f81d336df090395ee968bcc2bf5fd08780d2bb\": container with ID starting with 76eb1e69cbb90abd9f2b0e6ef6f81d336df090395ee968bcc2bf5fd08780d2bb not found: ID does not exist" containerID="76eb1e69cbb90abd9f2b0e6ef6f81d336df090395ee968bcc2bf5fd08780d2bb" Jul 06 20:17:02.643067 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:02.642949 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76eb1e69cbb90abd9f2b0e6ef6f81d336df090395ee968bcc2bf5fd08780d2bb"} err="failed to get container status \"76eb1e69cbb90abd9f2b0e6ef6f81d336df090395ee968bcc2bf5fd08780d2bb\": rpc error: code = NotFound desc = could not find container \"76eb1e69cbb90abd9f2b0e6ef6f81d336df090395ee968bcc2bf5fd08780d2bb\": container with ID starting with 76eb1e69cbb90abd9f2b0e6ef6f81d336df090395ee968bcc2bf5fd08780d2bb not found: ID does not exist" Jul 06 20:17:02.652617 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:02.652581 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6t2nb"] Jul 06 20:17:02.655971 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:02.655937 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6t2nb"] Jul 06 20:17:04.165134 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:04.165099 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0233ed4d-0601-49e9-86ea-abce553d89c8" path="/var/lib/kubelet/pods/0233ed4d-0601-49e9-86ea-abce553d89c8/volumes" Jul 06 20:17:09.821718 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:09.821675 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-6zck8" Jul 06 20:17:09.822281 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:09.821751 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-6zck8" Jul 06 20:17:09.845397 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:09.845368 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-6zck8" Jul 06 20:17:10.697531 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:10.697503 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-6zck8" Jul 06 20:17:14.521866 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:14.521825 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732g7dd"] Jul 06 20:17:14.522294 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:14.522242 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0233ed4d-0601-49e9-86ea-abce553d89c8" containerName="registry-server" Jul 06 20:17:14.522294 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:14.522253 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="0233ed4d-0601-49e9-86ea-abce553d89c8" containerName="registry-server" Jul 06 20:17:14.522365 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:14.522317 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="0233ed4d-0601-49e9-86ea-abce553d89c8" containerName="registry-server" Jul 06 20:17:14.525512 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:14.525494 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732g7dd" Jul 06 20:17:14.527909 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:14.527876 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-slmrl\"" Jul 06 20:17:14.534123 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:14.534095 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732g7dd"] Jul 06 20:17:14.681751 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:14.681715 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a46a28db-8e4c-458a-ba7b-c5c5ea614b97-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732g7dd\" (UID: \"a46a28db-8e4c-458a-ba7b-c5c5ea614b97\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732g7dd" Jul 06 20:17:14.681983 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:14.681770 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a46a28db-8e4c-458a-ba7b-c5c5ea614b97-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732g7dd\" (UID: \"a46a28db-8e4c-458a-ba7b-c5c5ea614b97\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732g7dd" Jul 06 20:17:14.681983 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:14.681805 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ld6c4\" (UniqueName: \"kubernetes.io/projected/a46a28db-8e4c-458a-ba7b-c5c5ea614b97-kube-api-access-ld6c4\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732g7dd\" (UID: \"a46a28db-8e4c-458a-ba7b-c5c5ea614b97\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732g7dd" Jul 06 20:17:14.783228 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:14.783121 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ld6c4\" (UniqueName: \"kubernetes.io/projected/a46a28db-8e4c-458a-ba7b-c5c5ea614b97-kube-api-access-ld6c4\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732g7dd\" (UID: \"a46a28db-8e4c-458a-ba7b-c5c5ea614b97\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732g7dd" Jul 06 20:17:14.783228 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:14.783220 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a46a28db-8e4c-458a-ba7b-c5c5ea614b97-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732g7dd\" (UID: \"a46a28db-8e4c-458a-ba7b-c5c5ea614b97\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732g7dd" Jul 06 20:17:14.783477 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:14.783261 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a46a28db-8e4c-458a-ba7b-c5c5ea614b97-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732g7dd\" (UID: \"a46a28db-8e4c-458a-ba7b-c5c5ea614b97\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732g7dd" Jul 06 20:17:14.783618 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:14.783601 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a46a28db-8e4c-458a-ba7b-c5c5ea614b97-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732g7dd\" (UID: \"a46a28db-8e4c-458a-ba7b-c5c5ea614b97\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732g7dd" Jul 06 20:17:14.783670 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:14.783633 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a46a28db-8e4c-458a-ba7b-c5c5ea614b97-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732g7dd\" (UID: \"a46a28db-8e4c-458a-ba7b-c5c5ea614b97\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732g7dd" Jul 06 20:17:14.792081 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:14.792053 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ld6c4\" (UniqueName: \"kubernetes.io/projected/a46a28db-8e4c-458a-ba7b-c5c5ea614b97-kube-api-access-ld6c4\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732g7dd\" (UID: \"a46a28db-8e4c-458a-ba7b-c5c5ea614b97\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732g7dd" Jul 06 20:17:14.835623 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:14.835586 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732g7dd" Jul 06 20:17:14.974551 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:14.974528 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732g7dd"] Jul 06 20:17:14.976619 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:17:14.976587 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda46a28db_8e4c_458a_ba7b_c5c5ea614b97.slice/crio-cbe40d788aa6914b0dde07893f606bf8ebdfbf68c8b867ddc2a45cc06f729d2a WatchSource:0}: Error finding container cbe40d788aa6914b0dde07893f606bf8ebdfbf68c8b867ddc2a45cc06f729d2a: Status 404 returned error can't find the container with id cbe40d788aa6914b0dde07893f606bf8ebdfbf68c8b867ddc2a45cc06f729d2a Jul 06 20:17:15.341152 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:15.341062 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bc9fh"] Jul 06 20:17:15.343705 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:15.343688 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bc9fh" Jul 06 20:17:15.353211 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:15.353173 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bc9fh"] Jul 06 20:17:15.488511 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:15.488470 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3bf83d9c-23f6-4f11-8c0f-d70f27447216-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bc9fh\" (UID: \"3bf83d9c-23f6-4f11-8c0f-d70f27447216\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bc9fh" Jul 06 20:17:15.488707 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:15.488600 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3bf83d9c-23f6-4f11-8c0f-d70f27447216-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bc9fh\" (UID: \"3bf83d9c-23f6-4f11-8c0f-d70f27447216\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bc9fh" Jul 06 20:17:15.488707 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:15.488632 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbbfr\" (UniqueName: \"kubernetes.io/projected/3bf83d9c-23f6-4f11-8c0f-d70f27447216-kube-api-access-cbbfr\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bc9fh\" (UID: \"3bf83d9c-23f6-4f11-8c0f-d70f27447216\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bc9fh" Jul 06 20:17:15.590028 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:15.589987 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3bf83d9c-23f6-4f11-8c0f-d70f27447216-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bc9fh\" (UID: \"3bf83d9c-23f6-4f11-8c0f-d70f27447216\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bc9fh" Jul 06 20:17:15.590426 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:15.590092 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3bf83d9c-23f6-4f11-8c0f-d70f27447216-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bc9fh\" (UID: \"3bf83d9c-23f6-4f11-8c0f-d70f27447216\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bc9fh" Jul 06 20:17:15.590426 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:15.590113 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbbfr\" (UniqueName: \"kubernetes.io/projected/3bf83d9c-23f6-4f11-8c0f-d70f27447216-kube-api-access-cbbfr\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bc9fh\" (UID: \"3bf83d9c-23f6-4f11-8c0f-d70f27447216\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bc9fh" Jul 06 20:17:15.590501 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:15.590462 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3bf83d9c-23f6-4f11-8c0f-d70f27447216-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bc9fh\" (UID: \"3bf83d9c-23f6-4f11-8c0f-d70f27447216\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bc9fh" Jul 06 20:17:15.590535 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:15.590500 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3bf83d9c-23f6-4f11-8c0f-d70f27447216-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bc9fh\" (UID: \"3bf83d9c-23f6-4f11-8c0f-d70f27447216\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bc9fh" Jul 06 20:17:15.599472 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:15.599416 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbbfr\" (UniqueName: \"kubernetes.io/projected/3bf83d9c-23f6-4f11-8c0f-d70f27447216-kube-api-access-cbbfr\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bc9fh\" (UID: \"3bf83d9c-23f6-4f11-8c0f-d70f27447216\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bc9fh" Jul 06 20:17:15.654393 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:15.654351 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bc9fh" Jul 06 20:17:15.701989 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:15.701955 2551 generic.go:356] "Generic (PLEG): container finished" podID="a46a28db-8e4c-458a-ba7b-c5c5ea614b97" containerID="bddee6266a9c876e327867f2a5068e90ab5973363ff6d8ba1d5703d21a3dc9e3" exitCode=0 Jul 06 20:17:15.702191 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:15.702035 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732g7dd" event={"ID":"a46a28db-8e4c-458a-ba7b-c5c5ea614b97","Type":"ContainerDied","Data":"bddee6266a9c876e327867f2a5068e90ab5973363ff6d8ba1d5703d21a3dc9e3"} Jul 06 20:17:15.702191 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:15.702086 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732g7dd" event={"ID":"a46a28db-8e4c-458a-ba7b-c5c5ea614b97","Type":"ContainerStarted","Data":"cbe40d788aa6914b0dde07893f606bf8ebdfbf68c8b867ddc2a45cc06f729d2a"} Jul 06 20:17:15.810729 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:15.810698 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bc9fh"] Jul 06 20:17:15.812268 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:17:15.812241 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bf83d9c_23f6_4f11_8c0f_d70f27447216.slice/crio-57d39a8a25ff269bd0c928bc8f57bd514c3cc7107ae0fcced6654277e5a1c237 WatchSource:0}: Error finding container 57d39a8a25ff269bd0c928bc8f57bd514c3cc7107ae0fcced6654277e5a1c237: Status 404 returned error can't find the container with id 57d39a8a25ff269bd0c928bc8f57bd514c3cc7107ae0fcced6654277e5a1c237 Jul 06 20:17:16.129344 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:16.129258 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqd6k"] Jul 06 20:17:16.136836 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:16.133684 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqd6k" Jul 06 20:17:16.140504 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:16.140472 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqd6k"] Jul 06 20:17:16.297144 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:16.297110 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5ea8e837-257b-4fde-915b-96062fc3bb8f-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqd6k\" (UID: \"5ea8e837-257b-4fde-915b-96062fc3bb8f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqd6k" Jul 06 20:17:16.297144 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:16.297155 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v79zz\" (UniqueName: \"kubernetes.io/projected/5ea8e837-257b-4fde-915b-96062fc3bb8f-kube-api-access-v79zz\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqd6k\" (UID: \"5ea8e837-257b-4fde-915b-96062fc3bb8f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqd6k" Jul 06 20:17:16.297371 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:16.297219 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5ea8e837-257b-4fde-915b-96062fc3bb8f-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqd6k\" (UID: \"5ea8e837-257b-4fde-915b-96062fc3bb8f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqd6k" Jul 06 20:17:16.398582 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:16.398478 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5ea8e837-257b-4fde-915b-96062fc3bb8f-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqd6k\" (UID: \"5ea8e837-257b-4fde-915b-96062fc3bb8f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqd6k" Jul 06 20:17:16.398582 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:16.398521 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v79zz\" (UniqueName: \"kubernetes.io/projected/5ea8e837-257b-4fde-915b-96062fc3bb8f-kube-api-access-v79zz\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqd6k\" (UID: \"5ea8e837-257b-4fde-915b-96062fc3bb8f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqd6k" Jul 06 20:17:16.398582 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:16.398540 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5ea8e837-257b-4fde-915b-96062fc3bb8f-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqd6k\" (UID: \"5ea8e837-257b-4fde-915b-96062fc3bb8f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqd6k" Jul 06 20:17:16.399000 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:16.398982 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5ea8e837-257b-4fde-915b-96062fc3bb8f-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqd6k\" (UID: \"5ea8e837-257b-4fde-915b-96062fc3bb8f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqd6k" Jul 06 20:17:16.399071 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:16.398983 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5ea8e837-257b-4fde-915b-96062fc3bb8f-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqd6k\" (UID: \"5ea8e837-257b-4fde-915b-96062fc3bb8f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqd6k" Jul 06 20:17:16.407440 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:16.407410 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v79zz\" (UniqueName: \"kubernetes.io/projected/5ea8e837-257b-4fde-915b-96062fc3bb8f-kube-api-access-v79zz\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqd6k\" (UID: \"5ea8e837-257b-4fde-915b-96062fc3bb8f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqd6k" Jul 06 20:17:16.447157 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:16.447122 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqd6k" Jul 06 20:17:16.591085 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:16.591060 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqd6k"] Jul 06 20:17:16.594162 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:17:16.594125 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ea8e837_257b_4fde_915b_96062fc3bb8f.slice/crio-3fc9591940b14e131d8bad5f928c398c8eeabc2d34a6f4ad5564f30b774bbb18 WatchSource:0}: Error finding container 3fc9591940b14e131d8bad5f928c398c8eeabc2d34a6f4ad5564f30b774bbb18: Status 404 returned error can't find the container with id 3fc9591940b14e131d8bad5f928c398c8eeabc2d34a6f4ad5564f30b774bbb18 Jul 06 20:17:16.708821 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:16.708786 2551 generic.go:356] "Generic (PLEG): container finished" podID="a46a28db-8e4c-458a-ba7b-c5c5ea614b97" containerID="36b115aae4aa405316f4c79265b61acb670b3b8c4d888b162295ca363c578c89" exitCode=0 Jul 06 20:17:16.709024 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:16.708876 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732g7dd" event={"ID":"a46a28db-8e4c-458a-ba7b-c5c5ea614b97","Type":"ContainerDied","Data":"36b115aae4aa405316f4c79265b61acb670b3b8c4d888b162295ca363c578c89"} Jul 06 20:17:16.710574 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:16.710549 2551 generic.go:356] "Generic (PLEG): container finished" podID="3bf83d9c-23f6-4f11-8c0f-d70f27447216" containerID="9821d86d7222f1d42e36b32d0aebac02ac755e1e97a914ab91cbd0bbd3fa2b5d" exitCode=0 Jul 06 20:17:16.710680 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:16.710626 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bc9fh" event={"ID":"3bf83d9c-23f6-4f11-8c0f-d70f27447216","Type":"ContainerDied","Data":"9821d86d7222f1d42e36b32d0aebac02ac755e1e97a914ab91cbd0bbd3fa2b5d"} Jul 06 20:17:16.710738 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:16.710682 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bc9fh" event={"ID":"3bf83d9c-23f6-4f11-8c0f-d70f27447216","Type":"ContainerStarted","Data":"57d39a8a25ff269bd0c928bc8f57bd514c3cc7107ae0fcced6654277e5a1c237"} Jul 06 20:17:16.712776 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:16.712753 2551 generic.go:356] "Generic (PLEG): container finished" podID="5ea8e837-257b-4fde-915b-96062fc3bb8f" containerID="95b88ab5851908e36aca4343f75500d0332f9fdae4bd08ff048ed6be66498fa2" exitCode=0 Jul 06 20:17:16.712881 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:16.712815 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqd6k" event={"ID":"5ea8e837-257b-4fde-915b-96062fc3bb8f","Type":"ContainerDied","Data":"95b88ab5851908e36aca4343f75500d0332f9fdae4bd08ff048ed6be66498fa2"} Jul 06 20:17:16.712881 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:16.712864 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqd6k" event={"ID":"5ea8e837-257b-4fde-915b-96062fc3bb8f","Type":"ContainerStarted","Data":"3fc9591940b14e131d8bad5f928c398c8eeabc2d34a6f4ad5564f30b774bbb18"} Jul 06 20:17:16.736186 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:16.736157 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027c5q"] Jul 06 20:17:16.739799 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:16.739783 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027c5q" Jul 06 20:17:16.749723 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:16.749695 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027c5q"] Jul 06 20:17:16.903870 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:16.903834 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/61c662a3-a1eb-4fcb-9861-747c57829e38-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027c5q\" (UID: \"61c662a3-a1eb-4fcb-9861-747c57829e38\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027c5q" Jul 06 20:17:16.904119 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:16.903960 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jk6mb\" (UniqueName: \"kubernetes.io/projected/61c662a3-a1eb-4fcb-9861-747c57829e38-kube-api-access-jk6mb\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027c5q\" (UID: \"61c662a3-a1eb-4fcb-9861-747c57829e38\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027c5q" Jul 06 20:17:16.904119 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:16.903982 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/61c662a3-a1eb-4fcb-9861-747c57829e38-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027c5q\" (UID: \"61c662a3-a1eb-4fcb-9861-747c57829e38\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027c5q" Jul 06 20:17:17.005156 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:17.005124 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jk6mb\" (UniqueName: \"kubernetes.io/projected/61c662a3-a1eb-4fcb-9861-747c57829e38-kube-api-access-jk6mb\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027c5q\" (UID: \"61c662a3-a1eb-4fcb-9861-747c57829e38\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027c5q" Jul 06 20:17:17.005316 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:17.005164 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/61c662a3-a1eb-4fcb-9861-747c57829e38-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027c5q\" (UID: \"61c662a3-a1eb-4fcb-9861-747c57829e38\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027c5q" Jul 06 20:17:17.005316 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:17.005242 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/61c662a3-a1eb-4fcb-9861-747c57829e38-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027c5q\" (UID: \"61c662a3-a1eb-4fcb-9861-747c57829e38\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027c5q" Jul 06 20:17:17.005598 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:17.005580 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/61c662a3-a1eb-4fcb-9861-747c57829e38-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027c5q\" (UID: \"61c662a3-a1eb-4fcb-9861-747c57829e38\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027c5q" Jul 06 20:17:17.005643 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:17.005608 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/61c662a3-a1eb-4fcb-9861-747c57829e38-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027c5q\" (UID: \"61c662a3-a1eb-4fcb-9861-747c57829e38\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027c5q" Jul 06 20:17:17.017103 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:17.017056 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jk6mb\" (UniqueName: \"kubernetes.io/projected/61c662a3-a1eb-4fcb-9861-747c57829e38-kube-api-access-jk6mb\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027c5q\" (UID: \"61c662a3-a1eb-4fcb-9861-747c57829e38\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027c5q" Jul 06 20:17:17.080552 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:17.080505 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027c5q" Jul 06 20:17:17.225043 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:17.225014 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027c5q"] Jul 06 20:17:17.227538 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:17:17.227513 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61c662a3_a1eb_4fcb_9861_747c57829e38.slice/crio-112dccbe6f078d5fcdefcec61a1d5200c30098e94ee34f1c85d360465b505ae1 WatchSource:0}: Error finding container 112dccbe6f078d5fcdefcec61a1d5200c30098e94ee34f1c85d360465b505ae1: Status 404 returned error can't find the container with id 112dccbe6f078d5fcdefcec61a1d5200c30098e94ee34f1c85d360465b505ae1 Jul 06 20:17:17.719667 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:17.719627 2551 generic.go:356] "Generic (PLEG): container finished" podID="5ea8e837-257b-4fde-915b-96062fc3bb8f" containerID="e0be45176351aafe3c1163459a20626c8184f247a358b9be459c1563c496e28a" exitCode=0 Jul 06 20:17:17.720105 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:17.719699 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqd6k" event={"ID":"5ea8e837-257b-4fde-915b-96062fc3bb8f","Type":"ContainerDied","Data":"e0be45176351aafe3c1163459a20626c8184f247a358b9be459c1563c496e28a"} Jul 06 20:17:17.722064 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:17.722042 2551 generic.go:356] "Generic (PLEG): container finished" podID="a46a28db-8e4c-458a-ba7b-c5c5ea614b97" containerID="a86cd59f8b78b994742cb39f3cc077dea9da0002995aab86e7612ad1d3aacd33" exitCode=0 Jul 06 20:17:17.722160 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:17.722132 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732g7dd" event={"ID":"a46a28db-8e4c-458a-ba7b-c5c5ea614b97","Type":"ContainerDied","Data":"a86cd59f8b78b994742cb39f3cc077dea9da0002995aab86e7612ad1d3aacd33"} Jul 06 20:17:17.723608 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:17.723588 2551 generic.go:356] "Generic (PLEG): container finished" podID="61c662a3-a1eb-4fcb-9861-747c57829e38" containerID="478250acc4adeaceded51c5d0aa043ba2e67bb5c4aac1624d915668fa7fd7034" exitCode=0 Jul 06 20:17:17.723695 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:17.723627 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027c5q" event={"ID":"61c662a3-a1eb-4fcb-9861-747c57829e38","Type":"ContainerDied","Data":"478250acc4adeaceded51c5d0aa043ba2e67bb5c4aac1624d915668fa7fd7034"} Jul 06 20:17:17.723695 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:17.723662 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027c5q" event={"ID":"61c662a3-a1eb-4fcb-9861-747c57829e38","Type":"ContainerStarted","Data":"112dccbe6f078d5fcdefcec61a1d5200c30098e94ee34f1c85d360465b505ae1"} Jul 06 20:17:18.731478 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:18.731376 2551 generic.go:356] "Generic (PLEG): container finished" podID="5ea8e837-257b-4fde-915b-96062fc3bb8f" containerID="57de92d49be2b545592ddccc3422bdcf4b6ec24038b5f44319603162f830e64c" exitCode=0 Jul 06 20:17:18.731478 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:18.731422 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqd6k" event={"ID":"5ea8e837-257b-4fde-915b-96062fc3bb8f","Type":"ContainerDied","Data":"57de92d49be2b545592ddccc3422bdcf4b6ec24038b5f44319603162f830e64c"} Jul 06 20:17:18.733367 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:18.733337 2551 generic.go:356] "Generic (PLEG): container finished" podID="3bf83d9c-23f6-4f11-8c0f-d70f27447216" containerID="88e615f69190f8ab2dd23cffdac564d0345aa8c99a78663ec72add2f190e4209" exitCode=0 Jul 06 20:17:18.733477 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:18.733397 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bc9fh" event={"ID":"3bf83d9c-23f6-4f11-8c0f-d70f27447216","Type":"ContainerDied","Data":"88e615f69190f8ab2dd23cffdac564d0345aa8c99a78663ec72add2f190e4209"} Jul 06 20:17:18.735265 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:18.735237 2551 generic.go:356] "Generic (PLEG): container finished" podID="61c662a3-a1eb-4fcb-9861-747c57829e38" containerID="6b8bde2396e21f7aeb3fa6adae1a7d71e1d14b5488bd26ac93397c32022647ba" exitCode=0 Jul 06 20:17:18.735357 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:18.735272 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027c5q" event={"ID":"61c662a3-a1eb-4fcb-9861-747c57829e38","Type":"ContainerDied","Data":"6b8bde2396e21f7aeb3fa6adae1a7d71e1d14b5488bd26ac93397c32022647ba"} Jul 06 20:17:18.894731 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:18.894698 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732g7dd" Jul 06 20:17:19.023530 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:19.023489 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ld6c4\" (UniqueName: \"kubernetes.io/projected/a46a28db-8e4c-458a-ba7b-c5c5ea614b97-kube-api-access-ld6c4\") pod \"a46a28db-8e4c-458a-ba7b-c5c5ea614b97\" (UID: \"a46a28db-8e4c-458a-ba7b-c5c5ea614b97\") " Jul 06 20:17:19.023715 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:19.023567 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a46a28db-8e4c-458a-ba7b-c5c5ea614b97-util\") pod \"a46a28db-8e4c-458a-ba7b-c5c5ea614b97\" (UID: \"a46a28db-8e4c-458a-ba7b-c5c5ea614b97\") " Jul 06 20:17:19.023715 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:19.023644 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a46a28db-8e4c-458a-ba7b-c5c5ea614b97-bundle\") pod \"a46a28db-8e4c-458a-ba7b-c5c5ea614b97\" (UID: \"a46a28db-8e4c-458a-ba7b-c5c5ea614b97\") " Jul 06 20:17:19.024254 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:19.024230 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a46a28db-8e4c-458a-ba7b-c5c5ea614b97-bundle" (OuterVolumeSpecName: "bundle") pod "a46a28db-8e4c-458a-ba7b-c5c5ea614b97" (UID: "a46a28db-8e4c-458a-ba7b-c5c5ea614b97"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:17:19.025810 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:19.025783 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a46a28db-8e4c-458a-ba7b-c5c5ea614b97-kube-api-access-ld6c4" (OuterVolumeSpecName: "kube-api-access-ld6c4") pod "a46a28db-8e4c-458a-ba7b-c5c5ea614b97" (UID: "a46a28db-8e4c-458a-ba7b-c5c5ea614b97"). InnerVolumeSpecName "kube-api-access-ld6c4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:17:19.029019 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:19.028996 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a46a28db-8e4c-458a-ba7b-c5c5ea614b97-util" (OuterVolumeSpecName: "util") pod "a46a28db-8e4c-458a-ba7b-c5c5ea614b97" (UID: "a46a28db-8e4c-458a-ba7b-c5c5ea614b97"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:17:19.125047 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:19.125012 2551 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a46a28db-8e4c-458a-ba7b-c5c5ea614b97-bundle\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:17:19.125047 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:19.125039 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ld6c4\" (UniqueName: \"kubernetes.io/projected/a46a28db-8e4c-458a-ba7b-c5c5ea614b97-kube-api-access-ld6c4\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:17:19.125047 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:19.125049 2551 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a46a28db-8e4c-458a-ba7b-c5c5ea614b97-util\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:17:19.743137 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:19.743109 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732g7dd" Jul 06 20:17:19.743137 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:19.743122 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732g7dd" event={"ID":"a46a28db-8e4c-458a-ba7b-c5c5ea614b97","Type":"ContainerDied","Data":"cbe40d788aa6914b0dde07893f606bf8ebdfbf68c8b867ddc2a45cc06f729d2a"} Jul 06 20:17:19.743630 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:19.743169 2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cbe40d788aa6914b0dde07893f606bf8ebdfbf68c8b867ddc2a45cc06f729d2a" Jul 06 20:17:19.745281 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:19.745256 2551 generic.go:356] "Generic (PLEG): container finished" podID="3bf83d9c-23f6-4f11-8c0f-d70f27447216" containerID="a15b2a6787dcd7135111aef206a6f0b7c22cb7674b3375879455e8d5cf690438" exitCode=0 Jul 06 20:17:19.745428 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:19.745289 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bc9fh" event={"ID":"3bf83d9c-23f6-4f11-8c0f-d70f27447216","Type":"ContainerDied","Data":"a15b2a6787dcd7135111aef206a6f0b7c22cb7674b3375879455e8d5cf690438"} Jul 06 20:17:19.747391 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:19.747365 2551 generic.go:356] "Generic (PLEG): container finished" podID="61c662a3-a1eb-4fcb-9861-747c57829e38" containerID="b3d482355a756d782aa298e06312080dd4f35385f728008cfef3ebb4537b11e2" exitCode=0 Jul 06 20:17:19.747477 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:19.747439 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027c5q" event={"ID":"61c662a3-a1eb-4fcb-9861-747c57829e38","Type":"ContainerDied","Data":"b3d482355a756d782aa298e06312080dd4f35385f728008cfef3ebb4537b11e2"} Jul 06 20:17:19.889696 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:19.889671 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqd6k" Jul 06 20:17:19.931944 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:19.931912 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v79zz\" (UniqueName: \"kubernetes.io/projected/5ea8e837-257b-4fde-915b-96062fc3bb8f-kube-api-access-v79zz\") pod \"5ea8e837-257b-4fde-915b-96062fc3bb8f\" (UID: \"5ea8e837-257b-4fde-915b-96062fc3bb8f\") " Jul 06 20:17:19.932118 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:19.931985 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5ea8e837-257b-4fde-915b-96062fc3bb8f-util\") pod \"5ea8e837-257b-4fde-915b-96062fc3bb8f\" (UID: \"5ea8e837-257b-4fde-915b-96062fc3bb8f\") " Jul 06 20:17:19.932118 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:19.932032 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5ea8e837-257b-4fde-915b-96062fc3bb8f-bundle\") pod \"5ea8e837-257b-4fde-915b-96062fc3bb8f\" (UID: \"5ea8e837-257b-4fde-915b-96062fc3bb8f\") " Jul 06 20:17:19.932517 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:19.932490 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ea8e837-257b-4fde-915b-96062fc3bb8f-bundle" (OuterVolumeSpecName: "bundle") pod "5ea8e837-257b-4fde-915b-96062fc3bb8f" (UID: "5ea8e837-257b-4fde-915b-96062fc3bb8f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:17:19.934224 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:19.934189 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ea8e837-257b-4fde-915b-96062fc3bb8f-kube-api-access-v79zz" (OuterVolumeSpecName: "kube-api-access-v79zz") pod "5ea8e837-257b-4fde-915b-96062fc3bb8f" (UID: "5ea8e837-257b-4fde-915b-96062fc3bb8f"). InnerVolumeSpecName "kube-api-access-v79zz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:17:19.937730 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:19.937704 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ea8e837-257b-4fde-915b-96062fc3bb8f-util" (OuterVolumeSpecName: "util") pod "5ea8e837-257b-4fde-915b-96062fc3bb8f" (UID: "5ea8e837-257b-4fde-915b-96062fc3bb8f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:17:20.032936 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:20.032857 2551 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5ea8e837-257b-4fde-915b-96062fc3bb8f-bundle\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:17:20.032936 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:20.032930 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v79zz\" (UniqueName: \"kubernetes.io/projected/5ea8e837-257b-4fde-915b-96062fc3bb8f-kube-api-access-v79zz\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:17:20.032936 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:20.032942 2551 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5ea8e837-257b-4fde-915b-96062fc3bb8f-util\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:17:20.754956 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:20.754915 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqd6k" event={"ID":"5ea8e837-257b-4fde-915b-96062fc3bb8f","Type":"ContainerDied","Data":"3fc9591940b14e131d8bad5f928c398c8eeabc2d34a6f4ad5564f30b774bbb18"} Jul 06 20:17:20.755380 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:20.754965 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqd6k" Jul 06 20:17:20.755380 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:20.754976 2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3fc9591940b14e131d8bad5f928c398c8eeabc2d34a6f4ad5564f30b774bbb18" Jul 06 20:17:20.893128 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:20.893101 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bc9fh" Jul 06 20:17:20.925518 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:20.925494 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027c5q" Jul 06 20:17:20.940297 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:20.940265 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/61c662a3-a1eb-4fcb-9861-747c57829e38-bundle\") pod \"61c662a3-a1eb-4fcb-9861-747c57829e38\" (UID: \"61c662a3-a1eb-4fcb-9861-747c57829e38\") " Jul 06 20:17:20.940492 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:20.940316 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3bf83d9c-23f6-4f11-8c0f-d70f27447216-util\") pod \"3bf83d9c-23f6-4f11-8c0f-d70f27447216\" (UID: \"3bf83d9c-23f6-4f11-8c0f-d70f27447216\") " Jul 06 20:17:20.940492 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:20.940382 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3bf83d9c-23f6-4f11-8c0f-d70f27447216-bundle\") pod \"3bf83d9c-23f6-4f11-8c0f-d70f27447216\" (UID: \"3bf83d9c-23f6-4f11-8c0f-d70f27447216\") " Jul 06 20:17:20.940492 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:20.940412 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cbbfr\" (UniqueName: \"kubernetes.io/projected/3bf83d9c-23f6-4f11-8c0f-d70f27447216-kube-api-access-cbbfr\") pod \"3bf83d9c-23f6-4f11-8c0f-d70f27447216\" (UID: \"3bf83d9c-23f6-4f11-8c0f-d70f27447216\") " Jul 06 20:17:20.940492 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:20.940436 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jk6mb\" (UniqueName: \"kubernetes.io/projected/61c662a3-a1eb-4fcb-9861-747c57829e38-kube-api-access-jk6mb\") pod \"61c662a3-a1eb-4fcb-9861-747c57829e38\" (UID: \"61c662a3-a1eb-4fcb-9861-747c57829e38\") " Jul 06 20:17:20.940492 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:20.940468 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/61c662a3-a1eb-4fcb-9861-747c57829e38-util\") pod \"61c662a3-a1eb-4fcb-9861-747c57829e38\" (UID: \"61c662a3-a1eb-4fcb-9861-747c57829e38\") " Jul 06 20:17:20.941059 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:20.941026 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61c662a3-a1eb-4fcb-9861-747c57829e38-bundle" (OuterVolumeSpecName: "bundle") pod "61c662a3-a1eb-4fcb-9861-747c57829e38" (UID: "61c662a3-a1eb-4fcb-9861-747c57829e38"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:17:20.941853 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:20.941561 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bf83d9c-23f6-4f11-8c0f-d70f27447216-bundle" (OuterVolumeSpecName: "bundle") pod "3bf83d9c-23f6-4f11-8c0f-d70f27447216" (UID: "3bf83d9c-23f6-4f11-8c0f-d70f27447216"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:17:20.943847 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:20.943804 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61c662a3-a1eb-4fcb-9861-747c57829e38-kube-api-access-jk6mb" (OuterVolumeSpecName: "kube-api-access-jk6mb") pod "61c662a3-a1eb-4fcb-9861-747c57829e38" (UID: "61c662a3-a1eb-4fcb-9861-747c57829e38"). InnerVolumeSpecName "kube-api-access-jk6mb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:17:20.944006 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:20.943913 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3bf83d9c-23f6-4f11-8c0f-d70f27447216-kube-api-access-cbbfr" (OuterVolumeSpecName: "kube-api-access-cbbfr") pod "3bf83d9c-23f6-4f11-8c0f-d70f27447216" (UID: "3bf83d9c-23f6-4f11-8c0f-d70f27447216"). InnerVolumeSpecName "kube-api-access-cbbfr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:17:20.948512 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:20.948479 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61c662a3-a1eb-4fcb-9861-747c57829e38-util" (OuterVolumeSpecName: "util") pod "61c662a3-a1eb-4fcb-9861-747c57829e38" (UID: "61c662a3-a1eb-4fcb-9861-747c57829e38"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:17:20.949345 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:20.949320 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bf83d9c-23f6-4f11-8c0f-d70f27447216-util" (OuterVolumeSpecName: "util") pod "3bf83d9c-23f6-4f11-8c0f-d70f27447216" (UID: "3bf83d9c-23f6-4f11-8c0f-d70f27447216"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:17:21.042268 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:21.042177 2551 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3bf83d9c-23f6-4f11-8c0f-d70f27447216-bundle\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:17:21.042268 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:21.042213 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cbbfr\" (UniqueName: \"kubernetes.io/projected/3bf83d9c-23f6-4f11-8c0f-d70f27447216-kube-api-access-cbbfr\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:17:21.042268 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:21.042223 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jk6mb\" (UniqueName: \"kubernetes.io/projected/61c662a3-a1eb-4fcb-9861-747c57829e38-kube-api-access-jk6mb\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:17:21.042268 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:21.042231 2551 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/61c662a3-a1eb-4fcb-9861-747c57829e38-util\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:17:21.042268 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:21.042239 2551 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/61c662a3-a1eb-4fcb-9861-747c57829e38-bundle\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:17:21.042268 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:21.042246 2551 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3bf83d9c-23f6-4f11-8c0f-d70f27447216-util\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:17:21.762914 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:21.762860 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bc9fh" Jul 06 20:17:21.762914 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:21.762853 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bc9fh" event={"ID":"3bf83d9c-23f6-4f11-8c0f-d70f27447216","Type":"ContainerDied","Data":"57d39a8a25ff269bd0c928bc8f57bd514c3cc7107ae0fcced6654277e5a1c237"} Jul 06 20:17:21.763393 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:21.762947 2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="57d39a8a25ff269bd0c928bc8f57bd514c3cc7107ae0fcced6654277e5a1c237" Jul 06 20:17:21.771988 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:21.771962 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027c5q" Jul 06 20:17:21.772399 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:21.771994 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027c5q" event={"ID":"61c662a3-a1eb-4fcb-9861-747c57829e38","Type":"ContainerDied","Data":"112dccbe6f078d5fcdefcec61a1d5200c30098e94ee34f1c85d360465b505ae1"} Jul 06 20:17:21.772399 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:21.772079 2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="112dccbe6f078d5fcdefcec61a1d5200c30098e94ee34f1c85d360465b505ae1" Jul 06 20:17:31.415002 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:31.414968 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bqrxx"] Jul 06 20:17:31.415485 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:31.415344 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61c662a3-a1eb-4fcb-9861-747c57829e38" containerName="pull" Jul 06 20:17:31.415485 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:31.415356 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="61c662a3-a1eb-4fcb-9861-747c57829e38" containerName="pull" Jul 06 20:17:31.415485 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:31.415368 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3bf83d9c-23f6-4f11-8c0f-d70f27447216" containerName="util" Jul 06 20:17:31.415485 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:31.415373 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bf83d9c-23f6-4f11-8c0f-d70f27447216" containerName="util" Jul 06 20:17:31.415485 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:31.415379 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a46a28db-8e4c-458a-ba7b-c5c5ea614b97" containerName="extract" Jul 06 20:17:31.415485 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:31.415386 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="a46a28db-8e4c-458a-ba7b-c5c5ea614b97" containerName="extract" Jul 06 20:17:31.415485 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:31.415393 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61c662a3-a1eb-4fcb-9861-747c57829e38" containerName="extract" Jul 06 20:17:31.415485 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:31.415398 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="61c662a3-a1eb-4fcb-9861-747c57829e38" containerName="extract" Jul 06 20:17:31.415485 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:31.415406 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3bf83d9c-23f6-4f11-8c0f-d70f27447216" containerName="pull" Jul 06 20:17:31.415485 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:31.415410 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bf83d9c-23f6-4f11-8c0f-d70f27447216" containerName="pull" Jul 06 20:17:31.415485 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:31.415422 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a46a28db-8e4c-458a-ba7b-c5c5ea614b97" containerName="pull" Jul 06 20:17:31.415485 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:31.415427 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="a46a28db-8e4c-458a-ba7b-c5c5ea614b97" containerName="pull" Jul 06 20:17:31.415485 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:31.415433 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ea8e837-257b-4fde-915b-96062fc3bb8f" containerName="util" Jul 06 20:17:31.415485 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:31.415437 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ea8e837-257b-4fde-915b-96062fc3bb8f" containerName="util" Jul 06 20:17:31.415485 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:31.415442 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61c662a3-a1eb-4fcb-9861-747c57829e38" containerName="util" Jul 06 20:17:31.415485 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:31.415447 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="61c662a3-a1eb-4fcb-9861-747c57829e38" containerName="util" Jul 06 20:17:31.415485 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:31.415452 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3bf83d9c-23f6-4f11-8c0f-d70f27447216" containerName="extract" Jul 06 20:17:31.415485 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:31.415456 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bf83d9c-23f6-4f11-8c0f-d70f27447216" containerName="extract" Jul 06 20:17:31.415485 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:31.415463 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ea8e837-257b-4fde-915b-96062fc3bb8f" containerName="pull" Jul 06 20:17:31.415485 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:31.415469 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ea8e837-257b-4fde-915b-96062fc3bb8f" containerName="pull" Jul 06 20:17:31.415485 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:31.415484 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a46a28db-8e4c-458a-ba7b-c5c5ea614b97" containerName="util" Jul 06 20:17:31.415485 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:31.415489 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="a46a28db-8e4c-458a-ba7b-c5c5ea614b97" containerName="util" Jul 06 20:17:31.415485 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:31.415494 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ea8e837-257b-4fde-915b-96062fc3bb8f" containerName="extract" Jul 06 20:17:31.415485 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:31.415499 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ea8e837-257b-4fde-915b-96062fc3bb8f" containerName="extract" Jul 06 20:17:31.416302 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:31.415552 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ea8e837-257b-4fde-915b-96062fc3bb8f" containerName="extract" Jul 06 20:17:31.416302 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:31.415562 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="61c662a3-a1eb-4fcb-9861-747c57829e38" containerName="extract" Jul 06 20:17:31.416302 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:31.415567 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="3bf83d9c-23f6-4f11-8c0f-d70f27447216" containerName="extract" Jul 06 20:17:31.416302 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:31.415574 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="a46a28db-8e4c-458a-ba7b-c5c5ea614b97" containerName="extract" Jul 06 20:17:31.422933 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:31.422886 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bqrxx" Jul 06 20:17:31.427416 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:31.427385 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-qrxb5\"" Jul 06 20:17:31.435434 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:31.435402 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bqrxx"] Jul 06 20:17:31.540072 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:31.540020 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlnbf\" (UniqueName: \"kubernetes.io/projected/0426da4f-e8cb-46f6-a4cc-ea85903c6328-kube-api-access-zlnbf\") pod \"limitador-operator-controller-manager-859d7b78bd-bqrxx\" (UID: \"0426da4f-e8cb-46f6-a4cc-ea85903c6328\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bqrxx" Jul 06 20:17:31.640611 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:31.640579 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlnbf\" (UniqueName: \"kubernetes.io/projected/0426da4f-e8cb-46f6-a4cc-ea85903c6328-kube-api-access-zlnbf\") pod \"limitador-operator-controller-manager-859d7b78bd-bqrxx\" (UID: \"0426da4f-e8cb-46f6-a4cc-ea85903c6328\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bqrxx" Jul 06 20:17:31.656242 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:31.656201 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlnbf\" (UniqueName: \"kubernetes.io/projected/0426da4f-e8cb-46f6-a4cc-ea85903c6328-kube-api-access-zlnbf\") pod \"limitador-operator-controller-manager-859d7b78bd-bqrxx\" (UID: \"0426da4f-e8cb-46f6-a4cc-ea85903c6328\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bqrxx" Jul 06 20:17:31.739623 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:31.739512 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bqrxx" Jul 06 20:17:31.898298 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:31.898268 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bqrxx"] Jul 06 20:17:31.900258 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:17:31.900230 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0426da4f_e8cb_46f6_a4cc_ea85903c6328.slice/crio-b33614e46979437bf70c0a0403139bd92af58e8c0d5930d6efc19760ac791f1d WatchSource:0}: Error finding container b33614e46979437bf70c0a0403139bd92af58e8c0d5930d6efc19760ac791f1d: Status 404 returned error can't find the container with id b33614e46979437bf70c0a0403139bd92af58e8c0d5930d6efc19760ac791f1d Jul 06 20:17:32.835101 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:32.835063 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bqrxx" event={"ID":"0426da4f-e8cb-46f6-a4cc-ea85903c6328","Type":"ContainerStarted","Data":"b33614e46979437bf70c0a0403139bd92af58e8c0d5930d6efc19760ac791f1d"} Jul 06 20:17:34.850726 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:34.850680 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bqrxx" event={"ID":"0426da4f-e8cb-46f6-a4cc-ea85903c6328","Type":"ContainerStarted","Data":"e41db1f5dd944f310453803b548d0c2b9088b95f7547e550e3b4cd3ae62d57e8"} Jul 06 20:17:34.851159 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:34.850761 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bqrxx" Jul 06 20:17:34.877337 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:34.877286 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bqrxx" podStartSLOduration=2.001466792 podStartE2EDuration="3.877268966s" totalImagesPullingTime="1.875802174s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:17:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:17:34.875758771 +0000 UTC m=+565.290958575" watchObservedRunningTime="2026-07-06 20:17:34.877268966 +0000 UTC m=+565.292468771" Jul 06 20:17:37.109022 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:37.108980 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ww8ms"] Jul 06 20:17:37.112926 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:37.112884 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ww8ms" Jul 06 20:17:37.117406 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:37.117375 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-xx9kf\"" Jul 06 20:17:37.132287 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:37.132253 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ww8ms"] Jul 06 20:17:37.191141 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:37.191105 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8978ba4-094f-4189-ada0-c881870d8a2f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ww8ms\" (UID: \"c8978ba4-094f-4189-ada0-c881870d8a2f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ww8ms" Jul 06 20:17:37.191332 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:37.191159 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pptg7\" (UniqueName: \"kubernetes.io/projected/c8978ba4-094f-4189-ada0-c881870d8a2f-kube-api-access-pptg7\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ww8ms\" (UID: \"c8978ba4-094f-4189-ada0-c881870d8a2f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ww8ms" Jul 06 20:17:37.292093 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:37.292053 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pptg7\" (UniqueName: \"kubernetes.io/projected/c8978ba4-094f-4189-ada0-c881870d8a2f-kube-api-access-pptg7\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ww8ms\" (UID: \"c8978ba4-094f-4189-ada0-c881870d8a2f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ww8ms" Jul 06 20:17:37.292318 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:37.292173 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8978ba4-094f-4189-ada0-c881870d8a2f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ww8ms\" (UID: \"c8978ba4-094f-4189-ada0-c881870d8a2f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ww8ms" Jul 06 20:17:37.292520 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:37.292502 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8978ba4-094f-4189-ada0-c881870d8a2f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ww8ms\" (UID: \"c8978ba4-094f-4189-ada0-c881870d8a2f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ww8ms" Jul 06 20:17:37.320445 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:37.320403 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pptg7\" (UniqueName: \"kubernetes.io/projected/c8978ba4-094f-4189-ada0-c881870d8a2f-kube-api-access-pptg7\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ww8ms\" (UID: \"c8978ba4-094f-4189-ada0-c881870d8a2f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ww8ms" Jul 06 20:17:37.426071 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:37.425974 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ww8ms" Jul 06 20:17:37.582170 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:17:37.582136 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8978ba4_094f_4189_ada0_c881870d8a2f.slice/crio-4b37398930c599ca0a251a9b2485ca3ca73ae46e892ed03d53880224c5e47a71 WatchSource:0}: Error finding container 4b37398930c599ca0a251a9b2485ca3ca73ae46e892ed03d53880224c5e47a71: Status 404 returned error can't find the container with id 4b37398930c599ca0a251a9b2485ca3ca73ae46e892ed03d53880224c5e47a71 Jul 06 20:17:37.582769 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:37.582750 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ww8ms"] Jul 06 20:17:37.866723 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:37.866686 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ww8ms" event={"ID":"c8978ba4-094f-4189-ada0-c881870d8a2f","Type":"ContainerStarted","Data":"4b37398930c599ca0a251a9b2485ca3ca73ae46e892ed03d53880224c5e47a71"} Jul 06 20:17:43.020562 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:43.020526 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-smk9d"] Jul 06 20:17:43.023566 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:43.023549 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-smk9d" Jul 06 20:17:43.027181 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:43.027161 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-7k64q\"" Jul 06 20:17:43.027533 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:43.027518 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 06 20:17:43.047765 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:43.047735 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-smk9d"] Jul 06 20:17:43.148048 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:43.148008 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s574z\" (UniqueName: \"kubernetes.io/projected/f48a1ab3-cef0-4cc0-8041-a5d97e784452-kube-api-access-s574z\") pod \"dns-operator-controller-manager-864dd78c4c-smk9d\" (UID: \"f48a1ab3-cef0-4cc0-8041-a5d97e784452\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-smk9d" Jul 06 20:17:43.249139 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:43.249100 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s574z\" (UniqueName: \"kubernetes.io/projected/f48a1ab3-cef0-4cc0-8041-a5d97e784452-kube-api-access-s574z\") pod \"dns-operator-controller-manager-864dd78c4c-smk9d\" (UID: \"f48a1ab3-cef0-4cc0-8041-a5d97e784452\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-smk9d" Jul 06 20:17:43.265579 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:43.265544 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s574z\" (UniqueName: \"kubernetes.io/projected/f48a1ab3-cef0-4cc0-8041-a5d97e784452-kube-api-access-s574z\") pod \"dns-operator-controller-manager-864dd78c4c-smk9d\" (UID: \"f48a1ab3-cef0-4cc0-8041-a5d97e784452\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-smk9d" Jul 06 20:17:43.334291 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:43.334193 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-smk9d" Jul 06 20:17:44.495168 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:44.495145 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-smk9d"] Jul 06 20:17:44.496563 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:17:44.496535 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf48a1ab3_cef0_4cc0_8041_a5d97e784452.slice/crio-f89c7c5362b939fe7f816376ae1f0c54e773b2698b14417d54592b59a3116a71 WatchSource:0}: Error finding container f89c7c5362b939fe7f816376ae1f0c54e773b2698b14417d54592b59a3116a71: Status 404 returned error can't find the container with id f89c7c5362b939fe7f816376ae1f0c54e773b2698b14417d54592b59a3116a71 Jul 06 20:17:44.909827 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:44.909781 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-smk9d" event={"ID":"f48a1ab3-cef0-4cc0-8041-a5d97e784452","Type":"ContainerStarted","Data":"f89c7c5362b939fe7f816376ae1f0c54e773b2698b14417d54592b59a3116a71"} Jul 06 20:17:44.911510 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:44.911477 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ww8ms" event={"ID":"c8978ba4-094f-4189-ada0-c881870d8a2f","Type":"ContainerStarted","Data":"52f4ec42be227a39bae42d46b8e4d3875da919b000b474ca88ab2fe05c4eaff0"} Jul 06 20:17:44.911657 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:44.911535 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ww8ms" Jul 06 20:17:44.969377 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:44.969318 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ww8ms" podStartSLOduration=1.166335968 podStartE2EDuration="7.969300382s" totalImagesPullingTime="6.802964414s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:17:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:17:44.967073748 +0000 UTC m=+575.382273574" watchObservedRunningTime="2026-07-06 20:17:44.969300382 +0000 UTC m=+575.384500187" Jul 06 20:17:45.859312 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:45.859281 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bqrxx" Jul 06 20:17:47.709789 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:47.709756 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-gz2pn"] Jul 06 20:17:47.713513 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:47.713489 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gz2pn" Jul 06 20:17:47.716937 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:47.716914 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 06 20:17:47.717454 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:47.717431 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 06 20:17:47.717533 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:47.717519 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-slmrl\"" Jul 06 20:17:47.731583 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:47.731558 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-gz2pn"] Jul 06 20:17:47.794906 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:47.794871 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vvk9\" (UniqueName: \"kubernetes.io/projected/09faf81c-561c-4eaf-86d1-a6e5856291c8-kube-api-access-5vvk9\") pod \"kuadrant-console-plugin-66b5774985-gz2pn\" (UID: \"09faf81c-561c-4eaf-86d1-a6e5856291c8\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gz2pn" Jul 06 20:17:47.795098 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:47.794925 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/09faf81c-561c-4eaf-86d1-a6e5856291c8-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-gz2pn\" (UID: \"09faf81c-561c-4eaf-86d1-a6e5856291c8\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gz2pn" Jul 06 20:17:47.795098 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:47.795085 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/09faf81c-561c-4eaf-86d1-a6e5856291c8-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-gz2pn\" (UID: \"09faf81c-561c-4eaf-86d1-a6e5856291c8\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gz2pn" Jul 06 20:17:47.896001 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:47.895967 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/09faf81c-561c-4eaf-86d1-a6e5856291c8-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-gz2pn\" (UID: \"09faf81c-561c-4eaf-86d1-a6e5856291c8\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gz2pn" Jul 06 20:17:47.896176 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:47.896021 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vvk9\" (UniqueName: \"kubernetes.io/projected/09faf81c-561c-4eaf-86d1-a6e5856291c8-kube-api-access-5vvk9\") pod \"kuadrant-console-plugin-66b5774985-gz2pn\" (UID: \"09faf81c-561c-4eaf-86d1-a6e5856291c8\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gz2pn" Jul 06 20:17:47.896176 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:47.896053 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/09faf81c-561c-4eaf-86d1-a6e5856291c8-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-gz2pn\" (UID: \"09faf81c-561c-4eaf-86d1-a6e5856291c8\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gz2pn" Jul 06 20:17:47.896176 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:17:47.896131 2551 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 06 20:17:47.896303 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:17:47.896192 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09faf81c-561c-4eaf-86d1-a6e5856291c8-plugin-serving-cert podName:09faf81c-561c-4eaf-86d1-a6e5856291c8 nodeName:}" failed. No retries permitted until 2026-07-06 20:17:48.396176204 +0000 UTC m=+578.811375991 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/09faf81c-561c-4eaf-86d1-a6e5856291c8-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-gz2pn" (UID: "09faf81c-561c-4eaf-86d1-a6e5856291c8") : secret "plugin-serving-cert" not found Jul 06 20:17:47.896957 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:47.896932 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/09faf81c-561c-4eaf-86d1-a6e5856291c8-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-gz2pn\" (UID: \"09faf81c-561c-4eaf-86d1-a6e5856291c8\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gz2pn" Jul 06 20:17:47.924340 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:47.924308 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vvk9\" (UniqueName: \"kubernetes.io/projected/09faf81c-561c-4eaf-86d1-a6e5856291c8-kube-api-access-5vvk9\") pod \"kuadrant-console-plugin-66b5774985-gz2pn\" (UID: \"09faf81c-561c-4eaf-86d1-a6e5856291c8\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gz2pn" Jul 06 20:17:47.935464 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:47.935424 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-smk9d" event={"ID":"f48a1ab3-cef0-4cc0-8041-a5d97e784452","Type":"ContainerStarted","Data":"0f2344b88c5fcd622b2a7b3b514cc29c7ab47e0e958bd8d1f708a55202b2f5d6"} Jul 06 20:17:47.935645 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:47.935588 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-smk9d" Jul 06 20:17:47.964839 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:47.964725 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-smk9d" podStartSLOduration=3.564047532 podStartE2EDuration="5.964707123s" totalImagesPullingTime="2.400659591s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:17:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:17:47.962408162 +0000 UTC m=+578.377607995" watchObservedRunningTime="2026-07-06 20:17:47.964707123 +0000 UTC m=+578.379906928" Jul 06 20:17:48.402743 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:48.402712 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/09faf81c-561c-4eaf-86d1-a6e5856291c8-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-gz2pn\" (UID: \"09faf81c-561c-4eaf-86d1-a6e5856291c8\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gz2pn" Jul 06 20:17:48.405247 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:48.405220 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/09faf81c-561c-4eaf-86d1-a6e5856291c8-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-gz2pn\" (UID: \"09faf81c-561c-4eaf-86d1-a6e5856291c8\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gz2pn" Jul 06 20:17:48.623313 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:48.623277 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gz2pn" Jul 06 20:17:48.776975 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:17:48.776875 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod09faf81c_561c_4eaf_86d1_a6e5856291c8.slice/crio-b461ea1ca71692866354443fa40c54e4cca94630cc383b5e4963179619e1fb2c WatchSource:0}: Error finding container b461ea1ca71692866354443fa40c54e4cca94630cc383b5e4963179619e1fb2c: Status 404 returned error can't find the container with id b461ea1ca71692866354443fa40c54e4cca94630cc383b5e4963179619e1fb2c Jul 06 20:17:48.781711 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:48.781688 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-gz2pn"] Jul 06 20:17:48.942342 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:48.942241 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gz2pn" event={"ID":"09faf81c-561c-4eaf-86d1-a6e5856291c8","Type":"ContainerStarted","Data":"b461ea1ca71692866354443fa40c54e4cca94630cc383b5e4963179619e1fb2c"} Jul 06 20:17:55.922673 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:55.922636 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ww8ms" Jul 06 20:17:58.079766 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:58.079727 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ww8ms"] Jul 06 20:17:58.080279 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:58.079986 2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ww8ms" podUID="c8978ba4-094f-4189-ada0-c881870d8a2f" containerName="manager" containerID="cri-o://52f4ec42be227a39bae42d46b8e4d3875da919b000b474ca88ab2fe05c4eaff0" gracePeriod=2 Jul 06 20:17:58.105595 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:58.105564 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ww8ms"] Jul 06 20:17:58.107727 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:17:58.107684 2551 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"c8978ba4-094f-4189-ada0-c881870d8a2f\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-ww8ms\": pods \"kuadrant-operator-controller-manager-8684c9ff49-ww8ms\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ww8ms" Jul 06 20:17:58.126464 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:58.126429 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bqrxx"] Jul 06 20:17:58.126737 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:58.126698 2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bqrxx" podUID="0426da4f-e8cb-46f6-a4cc-ea85903c6328" containerName="manager" containerID="cri-o://e41db1f5dd944f310453803b548d0c2b9088b95f7547e550e3b4cd3ae62d57e8" gracePeriod=2 Jul 06 20:17:58.144446 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:58.144402 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-724k9"] Jul 06 20:17:58.145334 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:58.145310 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8978ba4-094f-4189-ada0-c881870d8a2f" containerName="manager" Jul 06 20:17:58.145539 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:58.145480 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8978ba4-094f-4189-ada0-c881870d8a2f" containerName="manager" Jul 06 20:17:58.149942 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:17:58.145868 2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-ww8ms\" is forbidden: User \"system:node:ip-10-0-137-178.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-178.ec2.internal' and this object" podUID="c8978ba4-094f-4189-ada0-c881870d8a2f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ww8ms" Jul 06 20:17:58.150236 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:58.150217 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8978ba4-094f-4189-ada0-c881870d8a2f" containerName="manager" Jul 06 20:17:58.154550 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:58.154271 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-724k9" Jul 06 20:17:58.154774 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:58.154147 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bqrxx"] Jul 06 20:17:58.163800 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:17:58.163762 2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-ww8ms\" is forbidden: User \"system:node:ip-10-0-137-178.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-178.ec2.internal' and this object" podUID="c8978ba4-094f-4189-ada0-c881870d8a2f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ww8ms" Jul 06 20:17:58.174962 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:58.174927 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-724k9"] Jul 06 20:17:58.190700 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:58.190618 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zjvkk"] Jul 06 20:17:58.191204 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:58.191183 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0426da4f-e8cb-46f6-a4cc-ea85903c6328" containerName="manager" Jul 06 20:17:58.191320 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:58.191207 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="0426da4f-e8cb-46f6-a4cc-ea85903c6328" containerName="manager" Jul 06 20:17:58.191378 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:58.191330 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="0426da4f-e8cb-46f6-a4cc-ea85903c6328" containerName="manager" Jul 06 20:17:58.194738 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:58.194714 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zjvkk" Jul 06 20:17:58.199440 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:58.199411 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a405a81e-39e2-4461-83f4-f1bef15b5fd7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-724k9\" (UID: \"a405a81e-39e2-4461-83f4-f1bef15b5fd7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-724k9" Jul 06 20:17:58.199626 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:58.199598 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgmtn\" (UniqueName: \"kubernetes.io/projected/a405a81e-39e2-4461-83f4-f1bef15b5fd7-kube-api-access-zgmtn\") pod \"kuadrant-operator-controller-manager-8684c9ff49-724k9\" (UID: \"a405a81e-39e2-4461-83f4-f1bef15b5fd7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-724k9" Jul 06 20:17:58.199692 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:58.199636 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4b5m\" (UniqueName: \"kubernetes.io/projected/e8251546-8656-4cdb-b307-69f62aba522c-kube-api-access-b4b5m\") pod \"limitador-operator-controller-manager-859d7b78bd-zjvkk\" (UID: \"e8251546-8656-4cdb-b307-69f62aba522c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zjvkk" Jul 06 20:17:58.209177 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:17:58.209132 2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-bqrxx\" is forbidden: User \"system:node:ip-10-0-137-178.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-178.ec2.internal' and this object" podUID="0426da4f-e8cb-46f6-a4cc-ea85903c6328" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bqrxx" Jul 06 20:17:58.237863 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:58.237819 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zjvkk"] Jul 06 20:17:58.301057 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:58.301018 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgmtn\" (UniqueName: \"kubernetes.io/projected/a405a81e-39e2-4461-83f4-f1bef15b5fd7-kube-api-access-zgmtn\") pod \"kuadrant-operator-controller-manager-8684c9ff49-724k9\" (UID: \"a405a81e-39e2-4461-83f4-f1bef15b5fd7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-724k9" Jul 06 20:17:58.301250 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:58.301067 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4b5m\" (UniqueName: \"kubernetes.io/projected/e8251546-8656-4cdb-b307-69f62aba522c-kube-api-access-b4b5m\") pod \"limitador-operator-controller-manager-859d7b78bd-zjvkk\" (UID: \"e8251546-8656-4cdb-b307-69f62aba522c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zjvkk" Jul 06 20:17:58.301250 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:58.301105 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a405a81e-39e2-4461-83f4-f1bef15b5fd7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-724k9\" (UID: \"a405a81e-39e2-4461-83f4-f1bef15b5fd7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-724k9" Jul 06 20:17:58.301642 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:58.301620 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a405a81e-39e2-4461-83f4-f1bef15b5fd7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-724k9\" (UID: \"a405a81e-39e2-4461-83f4-f1bef15b5fd7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-724k9" Jul 06 20:17:58.330755 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:58.330682 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgmtn\" (UniqueName: \"kubernetes.io/projected/a405a81e-39e2-4461-83f4-f1bef15b5fd7-kube-api-access-zgmtn\") pod \"kuadrant-operator-controller-manager-8684c9ff49-724k9\" (UID: \"a405a81e-39e2-4461-83f4-f1bef15b5fd7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-724k9" Jul 06 20:17:58.332445 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:58.332406 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4b5m\" (UniqueName: \"kubernetes.io/projected/e8251546-8656-4cdb-b307-69f62aba522c-kube-api-access-b4b5m\") pod \"limitador-operator-controller-manager-859d7b78bd-zjvkk\" (UID: \"e8251546-8656-4cdb-b307-69f62aba522c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zjvkk" Jul 06 20:17:58.405971 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:58.405940 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bqrxx" Jul 06 20:17:58.409422 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:58.409370 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ww8ms" Jul 06 20:17:58.502812 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:58.502774 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zlnbf\" (UniqueName: \"kubernetes.io/projected/0426da4f-e8cb-46f6-a4cc-ea85903c6328-kube-api-access-zlnbf\") pod \"0426da4f-e8cb-46f6-a4cc-ea85903c6328\" (UID: \"0426da4f-e8cb-46f6-a4cc-ea85903c6328\") " Jul 06 20:17:58.503042 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:58.502833 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8978ba4-094f-4189-ada0-c881870d8a2f-extensions-socket-volume\") pod \"c8978ba4-094f-4189-ada0-c881870d8a2f\" (UID: \"c8978ba4-094f-4189-ada0-c881870d8a2f\") " Jul 06 20:17:58.503042 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:58.502920 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pptg7\" (UniqueName: \"kubernetes.io/projected/c8978ba4-094f-4189-ada0-c881870d8a2f-kube-api-access-pptg7\") pod \"c8978ba4-094f-4189-ada0-c881870d8a2f\" (UID: \"c8978ba4-094f-4189-ada0-c881870d8a2f\") " Jul 06 20:17:58.503494 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:58.503457 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8978ba4-094f-4189-ada0-c881870d8a2f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "c8978ba4-094f-4189-ada0-c881870d8a2f" (UID: "c8978ba4-094f-4189-ada0-c881870d8a2f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:17:58.505549 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:58.505514 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0426da4f-e8cb-46f6-a4cc-ea85903c6328-kube-api-access-zlnbf" (OuterVolumeSpecName: "kube-api-access-zlnbf") pod "0426da4f-e8cb-46f6-a4cc-ea85903c6328" (UID: "0426da4f-e8cb-46f6-a4cc-ea85903c6328"). InnerVolumeSpecName "kube-api-access-zlnbf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:17:58.505687 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:58.505559 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8978ba4-094f-4189-ada0-c881870d8a2f-kube-api-access-pptg7" (OuterVolumeSpecName: "kube-api-access-pptg7") pod "c8978ba4-094f-4189-ada0-c881870d8a2f" (UID: "c8978ba4-094f-4189-ada0-c881870d8a2f"). InnerVolumeSpecName "kube-api-access-pptg7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:17:58.524100 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:58.524063 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-724k9" Jul 06 20:17:58.531298 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:58.531269 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zjvkk" Jul 06 20:17:58.604314 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:58.604219 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zlnbf\" (UniqueName: \"kubernetes.io/projected/0426da4f-e8cb-46f6-a4cc-ea85903c6328-kube-api-access-zlnbf\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:17:58.604314 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:58.604254 2551 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8978ba4-094f-4189-ada0-c881870d8a2f-extensions-socket-volume\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:17:58.604314 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:58.604272 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pptg7\" (UniqueName: \"kubernetes.io/projected/c8978ba4-094f-4189-ada0-c881870d8a2f-kube-api-access-pptg7\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:17:58.719514 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:58.719481 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-724k9"] Jul 06 20:17:58.720580 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:17:58.720548 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda405a81e_39e2_4461_83f4_f1bef15b5fd7.slice/crio-b89768c589cfe5566634ee1b32d049043e45047e439ab564894a0715901f00ca WatchSource:0}: Error finding container b89768c589cfe5566634ee1b32d049043e45047e439ab564894a0715901f00ca: Status 404 returned error can't find the container with id b89768c589cfe5566634ee1b32d049043e45047e439ab564894a0715901f00ca Jul 06 20:17:58.741296 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:58.741267 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zjvkk"] Jul 06 20:17:58.741576 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:17:58.741551 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8251546_8656_4cdb_b307_69f62aba522c.slice/crio-a559b2cd165654573a4048b1a6ab3193dfbeee48c2248f1130c21fbd6780cc80 WatchSource:0}: Error finding container a559b2cd165654573a4048b1a6ab3193dfbeee48c2248f1130c21fbd6780cc80: Status 404 returned error can't find the container with id a559b2cd165654573a4048b1a6ab3193dfbeee48c2248f1130c21fbd6780cc80 Jul 06 20:17:58.945139 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:58.945052 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-smk9d" Jul 06 20:17:59.012122 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:59.012079 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zjvkk" event={"ID":"e8251546-8656-4cdb-b307-69f62aba522c","Type":"ContainerStarted","Data":"0ce5c192c2659d0c539788f348ee93fbd5692661990656f83b95d43a812d7fd8"} Jul 06 20:17:59.012319 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:59.012142 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zjvkk" event={"ID":"e8251546-8656-4cdb-b307-69f62aba522c","Type":"ContainerStarted","Data":"a559b2cd165654573a4048b1a6ab3193dfbeee48c2248f1130c21fbd6780cc80"} Jul 06 20:17:59.012319 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:59.012172 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zjvkk" Jul 06 20:17:59.013762 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:59.013733 2551 generic.go:356] "Generic (PLEG): container finished" podID="c8978ba4-094f-4189-ada0-c881870d8a2f" containerID="52f4ec42be227a39bae42d46b8e4d3875da919b000b474ca88ab2fe05c4eaff0" exitCode=0 Jul 06 20:17:59.013935 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:59.013784 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ww8ms" Jul 06 20:17:59.013935 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:59.013837 2551 scope.go:117] "RemoveContainer" containerID="52f4ec42be227a39bae42d46b8e4d3875da919b000b474ca88ab2fe05c4eaff0" Jul 06 20:17:59.016264 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:59.016217 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-724k9" event={"ID":"a405a81e-39e2-4461-83f4-f1bef15b5fd7","Type":"ContainerStarted","Data":"737ff6fc4a8e48b69320c6dda39d7cb165c93e7705d402626b1cb868f20e4781"} Jul 06 20:17:59.016417 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:59.016290 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-724k9" Jul 06 20:17:59.016417 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:59.016319 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-724k9" event={"ID":"a405a81e-39e2-4461-83f4-f1bef15b5fd7","Type":"ContainerStarted","Data":"b89768c589cfe5566634ee1b32d049043e45047e439ab564894a0715901f00ca"} Jul 06 20:17:59.017881 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:59.017858 2551 generic.go:356] "Generic (PLEG): container finished" podID="0426da4f-e8cb-46f6-a4cc-ea85903c6328" containerID="e41db1f5dd944f310453803b548d0c2b9088b95f7547e550e3b4cd3ae62d57e8" exitCode=0 Jul 06 20:17:59.017995 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:59.017933 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bqrxx" Jul 06 20:17:59.026594 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:59.026574 2551 scope.go:117] "RemoveContainer" containerID="52f4ec42be227a39bae42d46b8e4d3875da919b000b474ca88ab2fe05c4eaff0" Jul 06 20:17:59.027032 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:17:59.027010 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52f4ec42be227a39bae42d46b8e4d3875da919b000b474ca88ab2fe05c4eaff0\": container with ID starting with 52f4ec42be227a39bae42d46b8e4d3875da919b000b474ca88ab2fe05c4eaff0 not found: ID does not exist" containerID="52f4ec42be227a39bae42d46b8e4d3875da919b000b474ca88ab2fe05c4eaff0" Jul 06 20:17:59.027139 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:59.027044 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52f4ec42be227a39bae42d46b8e4d3875da919b000b474ca88ab2fe05c4eaff0"} err="failed to get container status \"52f4ec42be227a39bae42d46b8e4d3875da919b000b474ca88ab2fe05c4eaff0\": rpc error: code = NotFound desc = could not find container \"52f4ec42be227a39bae42d46b8e4d3875da919b000b474ca88ab2fe05c4eaff0\": container with ID starting with 52f4ec42be227a39bae42d46b8e4d3875da919b000b474ca88ab2fe05c4eaff0 not found: ID does not exist" Jul 06 20:17:59.027139 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:59.027070 2551 scope.go:117] "RemoveContainer" containerID="e41db1f5dd944f310453803b548d0c2b9088b95f7547e550e3b4cd3ae62d57e8" Jul 06 20:17:59.036888 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:59.036867 2551 scope.go:117] "RemoveContainer" containerID="e41db1f5dd944f310453803b548d0c2b9088b95f7547e550e3b4cd3ae62d57e8" Jul 06 20:17:59.037314 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:17:59.037288 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e41db1f5dd944f310453803b548d0c2b9088b95f7547e550e3b4cd3ae62d57e8\": container with ID starting with e41db1f5dd944f310453803b548d0c2b9088b95f7547e550e3b4cd3ae62d57e8 not found: ID does not exist" containerID="e41db1f5dd944f310453803b548d0c2b9088b95f7547e550e3b4cd3ae62d57e8" Jul 06 20:17:59.037415 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:59.037323 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e41db1f5dd944f310453803b548d0c2b9088b95f7547e550e3b4cd3ae62d57e8"} err="failed to get container status \"e41db1f5dd944f310453803b548d0c2b9088b95f7547e550e3b4cd3ae62d57e8\": rpc error: code = NotFound desc = could not find container \"e41db1f5dd944f310453803b548d0c2b9088b95f7547e550e3b4cd3ae62d57e8\": container with ID starting with e41db1f5dd944f310453803b548d0c2b9088b95f7547e550e3b4cd3ae62d57e8 not found: ID does not exist" Jul 06 20:17:59.057208 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:59.057155 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zjvkk" podStartSLOduration=1.057135405 podStartE2EDuration="1.057135405s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:17:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:17:59.056697138 +0000 UTC m=+589.471896944" watchObservedRunningTime="2026-07-06 20:17:59.057135405 +0000 UTC m=+589.472335215" Jul 06 20:17:59.117502 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:59.117442 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-724k9" podStartSLOduration=1.117424842 podStartE2EDuration="1.117424842s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:17:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:17:59.115758315 +0000 UTC m=+589.530958157" watchObservedRunningTime="2026-07-06 20:17:59.117424842 +0000 UTC m=+589.532624647" Jul 06 20:17:59.568877 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:59.568836 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2xfsm"] Jul 06 20:17:59.574815 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:59.574798 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2xfsm" Jul 06 20:17:59.593003 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:59.592970 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2xfsm"] Jul 06 20:17:59.614663 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:59.614621 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cc8m\" (UniqueName: \"kubernetes.io/projected/426dcf4a-6ebc-4101-b15f-3d0f08d27bca-kube-api-access-5cc8m\") pod \"kuadrant-operator-controller-manager-86878c769f-2xfsm\" (UID: \"426dcf4a-6ebc-4101-b15f-3d0f08d27bca\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2xfsm" Jul 06 20:17:59.614841 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:59.614689 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/426dcf4a-6ebc-4101-b15f-3d0f08d27bca-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-2xfsm\" (UID: \"426dcf4a-6ebc-4101-b15f-3d0f08d27bca\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2xfsm" Jul 06 20:17:59.716151 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:59.716115 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/426dcf4a-6ebc-4101-b15f-3d0f08d27bca-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-2xfsm\" (UID: \"426dcf4a-6ebc-4101-b15f-3d0f08d27bca\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2xfsm" Jul 06 20:17:59.716342 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:59.716245 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cc8m\" (UniqueName: \"kubernetes.io/projected/426dcf4a-6ebc-4101-b15f-3d0f08d27bca-kube-api-access-5cc8m\") pod \"kuadrant-operator-controller-manager-86878c769f-2xfsm\" (UID: \"426dcf4a-6ebc-4101-b15f-3d0f08d27bca\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2xfsm" Jul 06 20:17:59.716601 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:59.716575 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/426dcf4a-6ebc-4101-b15f-3d0f08d27bca-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-2xfsm\" (UID: \"426dcf4a-6ebc-4101-b15f-3d0f08d27bca\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2xfsm" Jul 06 20:17:59.744036 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:59.744001 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cc8m\" (UniqueName: \"kubernetes.io/projected/426dcf4a-6ebc-4101-b15f-3d0f08d27bca-kube-api-access-5cc8m\") pod \"kuadrant-operator-controller-manager-86878c769f-2xfsm\" (UID: \"426dcf4a-6ebc-4101-b15f-3d0f08d27bca\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2xfsm" Jul 06 20:17:59.885523 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:17:59.885430 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2xfsm" Jul 06 20:18:00.054626 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:00.054598 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2xfsm"] Jul 06 20:18:00.056087 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:18:00.056063 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod426dcf4a_6ebc_4101_b15f_3d0f08d27bca.slice/crio-8360896a00ce3a3acf2a3ddf41291e0bf0dcbb2dee3c3c225140003e912ea7cc WatchSource:0}: Error finding container 8360896a00ce3a3acf2a3ddf41291e0bf0dcbb2dee3c3c225140003e912ea7cc: Status 404 returned error can't find the container with id 8360896a00ce3a3acf2a3ddf41291e0bf0dcbb2dee3c3c225140003e912ea7cc Jul 06 20:18:00.165087 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:00.165058 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0426da4f-e8cb-46f6-a4cc-ea85903c6328" path="/var/lib/kubelet/pods/0426da4f-e8cb-46f6-a4cc-ea85903c6328/volumes" Jul 06 20:18:00.165445 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:00.165431 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8978ba4-094f-4189-ada0-c881870d8a2f" path="/var/lib/kubelet/pods/c8978ba4-094f-4189-ada0-c881870d8a2f/volumes" Jul 06 20:18:01.038856 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:01.038821 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2xfsm" event={"ID":"426dcf4a-6ebc-4101-b15f-3d0f08d27bca","Type":"ContainerStarted","Data":"6069eb0dd2a6cbc9fe6be0ec4a1c29b9d1fb5deb9fb67b1977c2022c3ba09aee"} Jul 06 20:18:01.039067 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:01.038877 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2xfsm" event={"ID":"426dcf4a-6ebc-4101-b15f-3d0f08d27bca","Type":"ContainerStarted","Data":"8360896a00ce3a3acf2a3ddf41291e0bf0dcbb2dee3c3c225140003e912ea7cc"} Jul 06 20:18:01.039067 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:01.039046 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2xfsm" Jul 06 20:18:01.096016 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:01.095955 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2xfsm" podStartSLOduration=2.095938115 podStartE2EDuration="2.095938115s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:17:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:18:01.09508895 +0000 UTC m=+591.510288756" watchObservedRunningTime="2026-07-06 20:18:01.095938115 +0000 UTC m=+591.511137920" Jul 06 20:18:10.032176 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:10.032143 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zjvkk" Jul 06 20:18:10.032713 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:10.032237 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-724k9" Jul 06 20:18:12.050838 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:12.050808 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2xfsm" Jul 06 20:18:12.152074 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:12.152041 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-724k9"] Jul 06 20:18:12.152332 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:12.152272 2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-724k9" podUID="a405a81e-39e2-4461-83f4-f1bef15b5fd7" containerName="manager" containerID="cri-o://737ff6fc4a8e48b69320c6dda39d7cb165c93e7705d402626b1cb868f20e4781" gracePeriod=10 Jul 06 20:18:15.961118 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:18:15.961082 2551 log.go:32] "ImageFsInfo from image service failed" err="rpc error: code = Unknown desc = get image fs info unable to get usage for /var/lib/containers/storage/overlay-images: get disk usage for path /var/lib/containers/storage/overlay-images: lstat /var/lib/containers/storage/overlay-images/.tmp-images.json1126638580: no such file or directory" Jul 06 20:18:15.961489 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:18:15.961130 2551 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="missing image stats: " Jul 06 20:18:16.089223 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:16.089191 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-724k9" Jul 06 20:18:16.150832 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:16.150795 2551 generic.go:356] "Generic (PLEG): container finished" podID="a405a81e-39e2-4461-83f4-f1bef15b5fd7" containerID="737ff6fc4a8e48b69320c6dda39d7cb165c93e7705d402626b1cb868f20e4781" exitCode=0 Jul 06 20:18:16.151042 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:16.150847 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-724k9" Jul 06 20:18:16.151042 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:16.150880 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-724k9" event={"ID":"a405a81e-39e2-4461-83f4-f1bef15b5fd7","Type":"ContainerDied","Data":"737ff6fc4a8e48b69320c6dda39d7cb165c93e7705d402626b1cb868f20e4781"} Jul 06 20:18:16.151042 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:16.150960 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-724k9" event={"ID":"a405a81e-39e2-4461-83f4-f1bef15b5fd7","Type":"ContainerDied","Data":"b89768c589cfe5566634ee1b32d049043e45047e439ab564894a0715901f00ca"} Jul 06 20:18:16.151042 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:16.150985 2551 scope.go:117] "RemoveContainer" containerID="737ff6fc4a8e48b69320c6dda39d7cb165c93e7705d402626b1cb868f20e4781" Jul 06 20:18:16.152698 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:16.152671 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gz2pn" event={"ID":"09faf81c-561c-4eaf-86d1-a6e5856291c8","Type":"ContainerStarted","Data":"3ba9556d45328b6d875fc94640d8e8069e2fd163a9a80a53e7a457af84ddd272"} Jul 06 20:18:16.162369 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:16.162344 2551 scope.go:117] "RemoveContainer" containerID="737ff6fc4a8e48b69320c6dda39d7cb165c93e7705d402626b1cb868f20e4781" Jul 06 20:18:16.162769 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:18:16.162737 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"737ff6fc4a8e48b69320c6dda39d7cb165c93e7705d402626b1cb868f20e4781\": container with ID starting with 737ff6fc4a8e48b69320c6dda39d7cb165c93e7705d402626b1cb868f20e4781 not found: ID does not exist" containerID="737ff6fc4a8e48b69320c6dda39d7cb165c93e7705d402626b1cb868f20e4781" Jul 06 20:18:16.162907 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:16.162775 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"737ff6fc4a8e48b69320c6dda39d7cb165c93e7705d402626b1cb868f20e4781"} err="failed to get container status \"737ff6fc4a8e48b69320c6dda39d7cb165c93e7705d402626b1cb868f20e4781\": rpc error: code = NotFound desc = could not find container \"737ff6fc4a8e48b69320c6dda39d7cb165c93e7705d402626b1cb868f20e4781\": container with ID starting with 737ff6fc4a8e48b69320c6dda39d7cb165c93e7705d402626b1cb868f20e4781 not found: ID does not exist" Jul 06 20:18:16.183759 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:16.183728 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a405a81e-39e2-4461-83f4-f1bef15b5fd7-extensions-socket-volume\") pod \"a405a81e-39e2-4461-83f4-f1bef15b5fd7\" (UID: \"a405a81e-39e2-4461-83f4-f1bef15b5fd7\") " Jul 06 20:18:16.183949 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:16.183839 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zgmtn\" (UniqueName: \"kubernetes.io/projected/a405a81e-39e2-4461-83f4-f1bef15b5fd7-kube-api-access-zgmtn\") pod \"a405a81e-39e2-4461-83f4-f1bef15b5fd7\" (UID: \"a405a81e-39e2-4461-83f4-f1bef15b5fd7\") " Jul 06 20:18:16.184120 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:16.184097 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a405a81e-39e2-4461-83f4-f1bef15b5fd7-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a405a81e-39e2-4461-83f4-f1bef15b5fd7" (UID: "a405a81e-39e2-4461-83f4-f1bef15b5fd7"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:18:16.185987 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:16.185962 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a405a81e-39e2-4461-83f4-f1bef15b5fd7-kube-api-access-zgmtn" (OuterVolumeSpecName: "kube-api-access-zgmtn") pod "a405a81e-39e2-4461-83f4-f1bef15b5fd7" (UID: "a405a81e-39e2-4461-83f4-f1bef15b5fd7"). InnerVolumeSpecName "kube-api-access-zgmtn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:18:16.285675 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:16.285634 2551 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a405a81e-39e2-4461-83f4-f1bef15b5fd7-extensions-socket-volume\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:18:16.285943 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:16.285925 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zgmtn\" (UniqueName: \"kubernetes.io/projected/a405a81e-39e2-4461-83f4-f1bef15b5fd7-kube-api-access-zgmtn\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:18:16.477238 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:16.477175 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gz2pn" podStartSLOduration=2.257949622 podStartE2EDuration="29.477156525s" totalImagesPullingTime="27.219206903s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:17:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:18:16.183705334 +0000 UTC m=+606.598905139" watchObservedRunningTime="2026-07-06 20:18:16.477156525 +0000 UTC m=+606.892356330" Jul 06 20:18:16.479802 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:16.479780 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-724k9"] Jul 06 20:18:16.484041 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:16.484015 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-724k9"] Jul 06 20:18:18.168986 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:18.168953 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a405a81e-39e2-4461-83f4-f1bef15b5fd7" path="/var/lib/kubelet/pods/a405a81e-39e2-4461-83f4-f1bef15b5fd7/volumes" Jul 06 20:18:29.005170 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:29.005137 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-qj8pt"] Jul 06 20:18:29.005573 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:29.005560 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a405a81e-39e2-4461-83f4-f1bef15b5fd7" containerName="manager" Jul 06 20:18:29.005618 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:29.005575 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="a405a81e-39e2-4461-83f4-f1bef15b5fd7" containerName="manager" Jul 06 20:18:29.005668 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:29.005659 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="a405a81e-39e2-4461-83f4-f1bef15b5fd7" containerName="manager" Jul 06 20:18:29.018276 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:29.018247 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-qj8pt" Jul 06 20:18:29.021599 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:29.021576 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-4gxtx\"" Jul 06 20:18:29.113738 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:29.113705 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f45bd611-bbaa-4f58-a40a-528378e433bb-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-qj8pt\" (UID: \"f45bd611-bbaa-4f58-a40a-528378e433bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-qj8pt" Jul 06 20:18:29.113738 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:29.113738 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f45bd611-bbaa-4f58-a40a-528378e433bb-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-qj8pt\" (UID: \"f45bd611-bbaa-4f58-a40a-528378e433bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-qj8pt" Jul 06 20:18:29.114018 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:29.113761 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f45bd611-bbaa-4f58-a40a-528378e433bb-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-qj8pt\" (UID: \"f45bd611-bbaa-4f58-a40a-528378e433bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-qj8pt" Jul 06 20:18:29.114018 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:29.113884 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f45bd611-bbaa-4f58-a40a-528378e433bb-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-qj8pt\" (UID: \"f45bd611-bbaa-4f58-a40a-528378e433bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-qj8pt" Jul 06 20:18:29.114018 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:29.114004 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f45bd611-bbaa-4f58-a40a-528378e433bb-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-qj8pt\" (UID: \"f45bd611-bbaa-4f58-a40a-528378e433bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-qj8pt" Jul 06 20:18:29.114189 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:29.114054 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f45bd611-bbaa-4f58-a40a-528378e433bb-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-qj8pt\" (UID: \"f45bd611-bbaa-4f58-a40a-528378e433bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-qj8pt" Jul 06 20:18:29.114189 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:29.114118 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f45bd611-bbaa-4f58-a40a-528378e433bb-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-qj8pt\" (UID: \"f45bd611-bbaa-4f58-a40a-528378e433bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-qj8pt" Jul 06 20:18:29.114189 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:29.114145 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f45bd611-bbaa-4f58-a40a-528378e433bb-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-qj8pt\" (UID: \"f45bd611-bbaa-4f58-a40a-528378e433bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-qj8pt" Jul 06 20:18:29.114288 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:29.114212 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mw8x7\" (UniqueName: \"kubernetes.io/projected/f45bd611-bbaa-4f58-a40a-528378e433bb-kube-api-access-mw8x7\") pod \"maas-default-gateway-openshift-default-5896d4dd49-qj8pt\" (UID: \"f45bd611-bbaa-4f58-a40a-528378e433bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-qj8pt" Jul 06 20:18:29.135310 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:29.135268 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-qj8pt"] Jul 06 20:18:29.215016 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:29.214978 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f45bd611-bbaa-4f58-a40a-528378e433bb-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-qj8pt\" (UID: \"f45bd611-bbaa-4f58-a40a-528378e433bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-qj8pt" Jul 06 20:18:29.215213 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:29.215029 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f45bd611-bbaa-4f58-a40a-528378e433bb-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-qj8pt\" (UID: \"f45bd611-bbaa-4f58-a40a-528378e433bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-qj8pt" Jul 06 20:18:29.215213 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:29.215055 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f45bd611-bbaa-4f58-a40a-528378e433bb-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-qj8pt\" (UID: \"f45bd611-bbaa-4f58-a40a-528378e433bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-qj8pt" Jul 06 20:18:29.215213 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:29.215083 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f45bd611-bbaa-4f58-a40a-528378e433bb-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-qj8pt\" (UID: \"f45bd611-bbaa-4f58-a40a-528378e433bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-qj8pt" Jul 06 20:18:29.215213 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:29.215102 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f45bd611-bbaa-4f58-a40a-528378e433bb-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-qj8pt\" (UID: \"f45bd611-bbaa-4f58-a40a-528378e433bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-qj8pt" Jul 06 20:18:29.215213 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:29.215134 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mw8x7\" (UniqueName: \"kubernetes.io/projected/f45bd611-bbaa-4f58-a40a-528378e433bb-kube-api-access-mw8x7\") pod \"maas-default-gateway-openshift-default-5896d4dd49-qj8pt\" (UID: \"f45bd611-bbaa-4f58-a40a-528378e433bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-qj8pt" Jul 06 20:18:29.215213 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:29.215172 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f45bd611-bbaa-4f58-a40a-528378e433bb-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-qj8pt\" (UID: \"f45bd611-bbaa-4f58-a40a-528378e433bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-qj8pt" Jul 06 20:18:29.215544 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:29.215431 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f45bd611-bbaa-4f58-a40a-528378e433bb-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-qj8pt\" (UID: \"f45bd611-bbaa-4f58-a40a-528378e433bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-qj8pt" Jul 06 20:18:29.215544 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:29.215460 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f45bd611-bbaa-4f58-a40a-528378e433bb-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-qj8pt\" (UID: \"f45bd611-bbaa-4f58-a40a-528378e433bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-qj8pt" Jul 06 20:18:29.215544 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:29.215477 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f45bd611-bbaa-4f58-a40a-528378e433bb-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-qj8pt\" (UID: \"f45bd611-bbaa-4f58-a40a-528378e433bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-qj8pt" Jul 06 20:18:29.215708 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:29.215549 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f45bd611-bbaa-4f58-a40a-528378e433bb-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-qj8pt\" (UID: \"f45bd611-bbaa-4f58-a40a-528378e433bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-qj8pt" Jul 06 20:18:29.215708 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:29.215589 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f45bd611-bbaa-4f58-a40a-528378e433bb-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-qj8pt\" (UID: \"f45bd611-bbaa-4f58-a40a-528378e433bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-qj8pt" Jul 06 20:18:29.215823 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:29.215767 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f45bd611-bbaa-4f58-a40a-528378e433bb-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-qj8pt\" (UID: \"f45bd611-bbaa-4f58-a40a-528378e433bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-qj8pt" Jul 06 20:18:29.215881 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:29.215861 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f45bd611-bbaa-4f58-a40a-528378e433bb-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-qj8pt\" (UID: \"f45bd611-bbaa-4f58-a40a-528378e433bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-qj8pt" Jul 06 20:18:29.217703 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:29.217685 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f45bd611-bbaa-4f58-a40a-528378e433bb-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-qj8pt\" (UID: \"f45bd611-bbaa-4f58-a40a-528378e433bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-qj8pt" Jul 06 20:18:29.217771 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:29.217703 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f45bd611-bbaa-4f58-a40a-528378e433bb-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-qj8pt\" (UID: \"f45bd611-bbaa-4f58-a40a-528378e433bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-qj8pt" Jul 06 20:18:29.231787 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:29.231756 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mw8x7\" (UniqueName: \"kubernetes.io/projected/f45bd611-bbaa-4f58-a40a-528378e433bb-kube-api-access-mw8x7\") pod \"maas-default-gateway-openshift-default-5896d4dd49-qj8pt\" (UID: \"f45bd611-bbaa-4f58-a40a-528378e433bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-qj8pt" Jul 06 20:18:29.232557 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:29.232537 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f45bd611-bbaa-4f58-a40a-528378e433bb-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-qj8pt\" (UID: \"f45bd611-bbaa-4f58-a40a-528378e433bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-qj8pt" Jul 06 20:18:29.328855 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:29.328756 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-qj8pt" Jul 06 20:18:29.515227 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:29.515201 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-qj8pt"] Jul 06 20:18:29.517517 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:18:29.517492 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf45bd611_bbaa_4f58_a40a_528378e433bb.slice/crio-c9c69da17dc3ed81f8a64e1da83e6798cc28abb14971eb87fcf18938e88f2eb7 WatchSource:0}: Error finding container c9c69da17dc3ed81f8a64e1da83e6798cc28abb14971eb87fcf18938e88f2eb7: Status 404 returned error can't find the container with id c9c69da17dc3ed81f8a64e1da83e6798cc28abb14971eb87fcf18938e88f2eb7 Jul 06 20:18:29.519745 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:29.519708 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:18:29.519847 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:29.519779 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:18:29.519847 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:29.519813 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:18:30.235226 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:30.235188 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-qj8pt" event={"ID":"f45bd611-bbaa-4f58-a40a-528378e433bb","Type":"ContainerStarted","Data":"bc2cb4b4c37996f49c8062f55c4bdc94d065a58e67b86a5a3f948d49716caabf"} Jul 06 20:18:30.235628 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:30.235252 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-qj8pt" event={"ID":"f45bd611-bbaa-4f58-a40a-528378e433bb","Type":"ContainerStarted","Data":"c9c69da17dc3ed81f8a64e1da83e6798cc28abb14971eb87fcf18938e88f2eb7"} Jul 06 20:18:30.296581 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:30.296530 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-qj8pt" podStartSLOduration=2.296512796 podStartE2EDuration="2.296512796s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:18:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:18:30.2943904 +0000 UTC m=+620.709590229" watchObservedRunningTime="2026-07-06 20:18:30.296512796 +0000 UTC m=+620.711712611" Jul 06 20:18:30.329854 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:30.329821 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-qj8pt" Jul 06 20:18:30.334949 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:30.334914 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-qj8pt" Jul 06 20:18:31.240764 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:31.240688 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-qj8pt" Jul 06 20:18:31.241849 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:31.241830 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-qj8pt" Jul 06 20:18:32.797605 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:32.797570 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-f8w2p"] Jul 06 20:18:32.808769 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:32.808744 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-f8w2p" Jul 06 20:18:32.822537 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:32.822514 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-dsr4r\"" Jul 06 20:18:32.858711 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:32.858672 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-f8w2p"] Jul 06 20:18:32.952017 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:32.951982 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgp2k\" (UniqueName: \"kubernetes.io/projected/4e06d7c8-04f3-43b4-8074-c51a723dffa8-kube-api-access-sgp2k\") pod \"authorino-58cd955675-f8w2p\" (UID: \"4e06d7c8-04f3-43b4-8074-c51a723dffa8\") " pod="kuadrant-system/authorino-58cd955675-f8w2p" Jul 06 20:18:33.052824 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:33.052722 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgp2k\" (UniqueName: \"kubernetes.io/projected/4e06d7c8-04f3-43b4-8074-c51a723dffa8-kube-api-access-sgp2k\") pod \"authorino-58cd955675-f8w2p\" (UID: \"4e06d7c8-04f3-43b4-8074-c51a723dffa8\") " pod="kuadrant-system/authorino-58cd955675-f8w2p" Jul 06 20:18:33.067175 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:33.067139 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgp2k\" (UniqueName: \"kubernetes.io/projected/4e06d7c8-04f3-43b4-8074-c51a723dffa8-kube-api-access-sgp2k\") pod \"authorino-58cd955675-f8w2p\" (UID: \"4e06d7c8-04f3-43b4-8074-c51a723dffa8\") " pod="kuadrant-system/authorino-58cd955675-f8w2p" Jul 06 20:18:33.118752 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:33.118716 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-f8w2p" Jul 06 20:18:33.200364 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:33.200314 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-db2wx"] Jul 06 20:18:33.208398 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:33.208368 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-db2wx" Jul 06 20:18:33.231983 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:33.231953 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-db2wx"] Jul 06 20:18:33.296292 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:33.296264 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-f8w2p"] Jul 06 20:18:33.296795 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:18:33.296771 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4e06d7c8_04f3_43b4_8074_c51a723dffa8.slice/crio-18b83bd427c1af3564f1e700de520dc090fe7695e2d42da151bf783a927d94c2 WatchSource:0}: Error finding container 18b83bd427c1af3564f1e700de520dc090fe7695e2d42da151bf783a927d94c2: Status 404 returned error can't find the container with id 18b83bd427c1af3564f1e700de520dc090fe7695e2d42da151bf783a927d94c2 Jul 06 20:18:33.298393 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:33.298372 2551 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 20:18:33.354837 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:33.354732 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvrmw\" (UniqueName: \"kubernetes.io/projected/d8008368-eaed-4886-adfa-703f3d124f7d-kube-api-access-zvrmw\") pod \"authorino-75d556bd77-db2wx\" (UID: \"d8008368-eaed-4886-adfa-703f3d124f7d\") " pod="kuadrant-system/authorino-75d556bd77-db2wx" Jul 06 20:18:33.455813 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:33.455777 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvrmw\" (UniqueName: \"kubernetes.io/projected/d8008368-eaed-4886-adfa-703f3d124f7d-kube-api-access-zvrmw\") pod \"authorino-75d556bd77-db2wx\" (UID: \"d8008368-eaed-4886-adfa-703f3d124f7d\") " pod="kuadrant-system/authorino-75d556bd77-db2wx" Jul 06 20:18:33.480178 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:33.480151 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvrmw\" (UniqueName: \"kubernetes.io/projected/d8008368-eaed-4886-adfa-703f3d124f7d-kube-api-access-zvrmw\") pod \"authorino-75d556bd77-db2wx\" (UID: \"d8008368-eaed-4886-adfa-703f3d124f7d\") " pod="kuadrant-system/authorino-75d556bd77-db2wx" Jul 06 20:18:33.519227 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:33.519189 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-db2wx" Jul 06 20:18:33.681500 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:33.681470 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-db2wx"] Jul 06 20:18:33.683730 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:18:33.683701 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd8008368_eaed_4886_adfa_703f3d124f7d.slice/crio-6800acc507ce8d3046173c634a23b3f1cbec3a2716cc93f6dfa48b8c26a25e8a WatchSource:0}: Error finding container 6800acc507ce8d3046173c634a23b3f1cbec3a2716cc93f6dfa48b8c26a25e8a: Status 404 returned error can't find the container with id 6800acc507ce8d3046173c634a23b3f1cbec3a2716cc93f6dfa48b8c26a25e8a Jul 06 20:18:34.263623 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:34.263578 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-db2wx" event={"ID":"d8008368-eaed-4886-adfa-703f3d124f7d","Type":"ContainerStarted","Data":"6800acc507ce8d3046173c634a23b3f1cbec3a2716cc93f6dfa48b8c26a25e8a"} Jul 06 20:18:34.265173 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:34.265132 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-f8w2p" event={"ID":"4e06d7c8-04f3-43b4-8074-c51a723dffa8","Type":"ContainerStarted","Data":"18b83bd427c1af3564f1e700de520dc090fe7695e2d42da151bf783a927d94c2"} Jul 06 20:18:39.298433 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:39.298386 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-f8w2p" event={"ID":"4e06d7c8-04f3-43b4-8074-c51a723dffa8","Type":"ContainerStarted","Data":"5ddef763c2dafd2d1fe4f5dc96f1085c3a9f1dc038eabb6600aebc58a8d87546"} Jul 06 20:18:39.299909 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:39.299868 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-db2wx" event={"ID":"d8008368-eaed-4886-adfa-703f3d124f7d","Type":"ContainerStarted","Data":"c6c2a51230052552d6e36d04d0e2e298fe8520ca18eab9351c248a9117e87607"} Jul 06 20:18:39.330765 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:39.330704 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-f8w2p" podStartSLOduration=1.952583795 podStartE2EDuration="7.33068761s" totalImagesPullingTime="5.378103815s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:18:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:18:39.329440418 +0000 UTC m=+629.744640224" watchObservedRunningTime="2026-07-06 20:18:39.33068761 +0000 UTC m=+629.745887415" Jul 06 20:18:39.364083 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:39.364019 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-db2wx" podStartSLOduration=1.385576779 podStartE2EDuration="6.363998264s" totalImagesPullingTime="4.978421485s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:18:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:18:39.363864136 +0000 UTC m=+629.779063941" watchObservedRunningTime="2026-07-06 20:18:39.363998264 +0000 UTC m=+629.779198071" Jul 06 20:18:39.451709 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:39.451673 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-f8w2p"] Jul 06 20:18:41.312269 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:41.312220 2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-f8w2p" podUID="4e06d7c8-04f3-43b4-8074-c51a723dffa8" containerName="authorino" containerID="cri-o://5ddef763c2dafd2d1fe4f5dc96f1085c3a9f1dc038eabb6600aebc58a8d87546" gracePeriod=30 Jul 06 20:18:42.126266 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:42.126233 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-f8w2p" Jul 06 20:18:42.135241 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:42.135207 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sgp2k\" (UniqueName: \"kubernetes.io/projected/4e06d7c8-04f3-43b4-8074-c51a723dffa8-kube-api-access-sgp2k\") pod \"4e06d7c8-04f3-43b4-8074-c51a723dffa8\" (UID: \"4e06d7c8-04f3-43b4-8074-c51a723dffa8\") " Jul 06 20:18:42.137244 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:42.137217 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e06d7c8-04f3-43b4-8074-c51a723dffa8-kube-api-access-sgp2k" (OuterVolumeSpecName: "kube-api-access-sgp2k") pod "4e06d7c8-04f3-43b4-8074-c51a723dffa8" (UID: "4e06d7c8-04f3-43b4-8074-c51a723dffa8"). InnerVolumeSpecName "kube-api-access-sgp2k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:18:42.236322 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:42.236285 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sgp2k\" (UniqueName: \"kubernetes.io/projected/4e06d7c8-04f3-43b4-8074-c51a723dffa8-kube-api-access-sgp2k\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:18:42.319484 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:42.319449 2551 generic.go:356] "Generic (PLEG): container finished" podID="4e06d7c8-04f3-43b4-8074-c51a723dffa8" containerID="5ddef763c2dafd2d1fe4f5dc96f1085c3a9f1dc038eabb6600aebc58a8d87546" exitCode=0 Jul 06 20:18:42.319952 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:42.319491 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-f8w2p" Jul 06 20:18:42.319952 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:42.319541 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-f8w2p" event={"ID":"4e06d7c8-04f3-43b4-8074-c51a723dffa8","Type":"ContainerDied","Data":"5ddef763c2dafd2d1fe4f5dc96f1085c3a9f1dc038eabb6600aebc58a8d87546"} Jul 06 20:18:42.319952 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:42.319593 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-f8w2p" event={"ID":"4e06d7c8-04f3-43b4-8074-c51a723dffa8","Type":"ContainerDied","Data":"18b83bd427c1af3564f1e700de520dc090fe7695e2d42da151bf783a927d94c2"} Jul 06 20:18:42.319952 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:42.319608 2551 scope.go:117] "RemoveContainer" containerID="5ddef763c2dafd2d1fe4f5dc96f1085c3a9f1dc038eabb6600aebc58a8d87546" Jul 06 20:18:42.329152 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:42.329132 2551 scope.go:117] "RemoveContainer" containerID="5ddef763c2dafd2d1fe4f5dc96f1085c3a9f1dc038eabb6600aebc58a8d87546" Jul 06 20:18:42.329500 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:18:42.329481 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ddef763c2dafd2d1fe4f5dc96f1085c3a9f1dc038eabb6600aebc58a8d87546\": container with ID starting with 5ddef763c2dafd2d1fe4f5dc96f1085c3a9f1dc038eabb6600aebc58a8d87546 not found: ID does not exist" containerID="5ddef763c2dafd2d1fe4f5dc96f1085c3a9f1dc038eabb6600aebc58a8d87546" Jul 06 20:18:42.329571 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:42.329526 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ddef763c2dafd2d1fe4f5dc96f1085c3a9f1dc038eabb6600aebc58a8d87546"} err="failed to get container status \"5ddef763c2dafd2d1fe4f5dc96f1085c3a9f1dc038eabb6600aebc58a8d87546\": rpc error: code = NotFound desc = could not find container \"5ddef763c2dafd2d1fe4f5dc96f1085c3a9f1dc038eabb6600aebc58a8d87546\": container with ID starting with 5ddef763c2dafd2d1fe4f5dc96f1085c3a9f1dc038eabb6600aebc58a8d87546 not found: ID does not exist" Jul 06 20:18:42.346572 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:42.346541 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-f8w2p"] Jul 06 20:18:42.354487 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:42.354459 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-f8w2p"] Jul 06 20:18:44.165410 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:18:44.165362 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e06d7c8-04f3-43b4-8074-c51a723dffa8" path="/var/lib/kubelet/pods/4e06d7c8-04f3-43b4-8074-c51a723dffa8/volumes" Jul 06 20:19:07.750232 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:07.750193 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8dk5jf"] Jul 06 20:19:07.750708 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:07.750617 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e06d7c8-04f3-43b4-8074-c51a723dffa8" containerName="authorino" Jul 06 20:19:07.750708 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:07.750631 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e06d7c8-04f3-43b4-8074-c51a723dffa8" containerName="authorino" Jul 06 20:19:07.750782 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:07.750712 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e06d7c8-04f3-43b4-8074-c51a723dffa8" containerName="authorino" Jul 06 20:19:07.755321 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:07.755304 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8dk5jf" Jul 06 20:19:07.765506 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:07.765486 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 06 20:19:07.765680 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:07.765662 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-wt4k5\"" Jul 06 20:19:07.770077 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:07.770057 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 06 20:19:07.793623 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:07.793592 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8dk5jf"] Jul 06 20:19:07.890712 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:07.890672 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0c4fca91-dafc-4beb-b657-016f3aa4ac32-util\") pod \"2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8dk5jf\" (UID: \"0c4fca91-dafc-4beb-b657-016f3aa4ac32\") " pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8dk5jf" Jul 06 20:19:07.890712 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:07.890717 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjb27\" (UniqueName: \"kubernetes.io/projected/0c4fca91-dafc-4beb-b657-016f3aa4ac32-kube-api-access-kjb27\") pod \"2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8dk5jf\" (UID: \"0c4fca91-dafc-4beb-b657-016f3aa4ac32\") " pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8dk5jf" Jul 06 20:19:07.890986 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:07.890767 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0c4fca91-dafc-4beb-b657-016f3aa4ac32-bundle\") pod \"2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8dk5jf\" (UID: \"0c4fca91-dafc-4beb-b657-016f3aa4ac32\") " pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8dk5jf" Jul 06 20:19:07.992282 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:07.992241 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0c4fca91-dafc-4beb-b657-016f3aa4ac32-util\") pod \"2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8dk5jf\" (UID: \"0c4fca91-dafc-4beb-b657-016f3aa4ac32\") " pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8dk5jf" Jul 06 20:19:07.992451 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:07.992299 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjb27\" (UniqueName: \"kubernetes.io/projected/0c4fca91-dafc-4beb-b657-016f3aa4ac32-kube-api-access-kjb27\") pod \"2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8dk5jf\" (UID: \"0c4fca91-dafc-4beb-b657-016f3aa4ac32\") " pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8dk5jf" Jul 06 20:19:07.992451 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:07.992324 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0c4fca91-dafc-4beb-b657-016f3aa4ac32-bundle\") pod \"2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8dk5jf\" (UID: \"0c4fca91-dafc-4beb-b657-016f3aa4ac32\") " pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8dk5jf" Jul 06 20:19:07.992637 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:07.992614 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0c4fca91-dafc-4beb-b657-016f3aa4ac32-util\") pod \"2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8dk5jf\" (UID: \"0c4fca91-dafc-4beb-b657-016f3aa4ac32\") " pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8dk5jf" Jul 06 20:19:07.992708 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:07.992688 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0c4fca91-dafc-4beb-b657-016f3aa4ac32-bundle\") pod \"2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8dk5jf\" (UID: \"0c4fca91-dafc-4beb-b657-016f3aa4ac32\") " pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8dk5jf" Jul 06 20:19:08.002924 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:08.002875 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjb27\" (UniqueName: \"kubernetes.io/projected/0c4fca91-dafc-4beb-b657-016f3aa4ac32-kube-api-access-kjb27\") pod \"2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8dk5jf\" (UID: \"0c4fca91-dafc-4beb-b657-016f3aa4ac32\") " pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8dk5jf" Jul 06 20:19:08.064508 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:08.064474 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8dk5jf" Jul 06 20:19:08.235939 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:08.235918 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8dk5jf"] Jul 06 20:19:08.238378 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:19:08.238353 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c4fca91_dafc_4beb_b657_016f3aa4ac32.slice/crio-bac2a9ae9e103efd876b7b5426e95aa5986a2a71458dee6337410a08fe2889f0 WatchSource:0}: Error finding container bac2a9ae9e103efd876b7b5426e95aa5986a2a71458dee6337410a08fe2889f0: Status 404 returned error can't find the container with id bac2a9ae9e103efd876b7b5426e95aa5986a2a71458dee6337410a08fe2889f0 Jul 06 20:19:08.480395 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:08.480357 2551 generic.go:356] "Generic (PLEG): container finished" podID="0c4fca91-dafc-4beb-b657-016f3aa4ac32" containerID="a6aca7697cb1c29e957c1b496b0abce608bc6c516ca919188c8d958e440d0715" exitCode=0 Jul 06 20:19:08.480588 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:08.480444 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8dk5jf" event={"ID":"0c4fca91-dafc-4beb-b657-016f3aa4ac32","Type":"ContainerDied","Data":"a6aca7697cb1c29e957c1b496b0abce608bc6c516ca919188c8d958e440d0715"} Jul 06 20:19:08.480588 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:08.480495 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8dk5jf" event={"ID":"0c4fca91-dafc-4beb-b657-016f3aa4ac32","Type":"ContainerStarted","Data":"bac2a9ae9e103efd876b7b5426e95aa5986a2a71458dee6337410a08fe2889f0"} Jul 06 20:19:09.489785 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:09.489690 2551 generic.go:356] "Generic (PLEG): container finished" podID="0c4fca91-dafc-4beb-b657-016f3aa4ac32" containerID="e57ed5489671e4901be7f7bb27a22cf3af205121f3f4861971cda92314ea1b28" exitCode=0 Jul 06 20:19:09.490172 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:09.489782 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8dk5jf" event={"ID":"0c4fca91-dafc-4beb-b657-016f3aa4ac32","Type":"ContainerDied","Data":"e57ed5489671e4901be7f7bb27a22cf3af205121f3f4861971cda92314ea1b28"} Jul 06 20:19:10.497744 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:10.497710 2551 generic.go:356] "Generic (PLEG): container finished" podID="0c4fca91-dafc-4beb-b657-016f3aa4ac32" containerID="7e9c8c112f27f68a636a42f110587d02270195dec94cb1c570221660e3202dac" exitCode=0 Jul 06 20:19:10.498226 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:10.497764 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8dk5jf" event={"ID":"0c4fca91-dafc-4beb-b657-016f3aa4ac32","Type":"ContainerDied","Data":"7e9c8c112f27f68a636a42f110587d02270195dec94cb1c570221660e3202dac"} Jul 06 20:19:11.662972 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:11.662944 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8dk5jf" Jul 06 20:19:11.835227 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:11.835129 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0c4fca91-dafc-4beb-b657-016f3aa4ac32-bundle\") pod \"0c4fca91-dafc-4beb-b657-016f3aa4ac32\" (UID: \"0c4fca91-dafc-4beb-b657-016f3aa4ac32\") " Jul 06 20:19:11.835396 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:11.835284 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0c4fca91-dafc-4beb-b657-016f3aa4ac32-util\") pod \"0c4fca91-dafc-4beb-b657-016f3aa4ac32\" (UID: \"0c4fca91-dafc-4beb-b657-016f3aa4ac32\") " Jul 06 20:19:11.835396 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:11.835312 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kjb27\" (UniqueName: \"kubernetes.io/projected/0c4fca91-dafc-4beb-b657-016f3aa4ac32-kube-api-access-kjb27\") pod \"0c4fca91-dafc-4beb-b657-016f3aa4ac32\" (UID: \"0c4fca91-dafc-4beb-b657-016f3aa4ac32\") " Jul 06 20:19:11.835793 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:11.835764 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c4fca91-dafc-4beb-b657-016f3aa4ac32-bundle" (OuterVolumeSpecName: "bundle") pod "0c4fca91-dafc-4beb-b657-016f3aa4ac32" (UID: "0c4fca91-dafc-4beb-b657-016f3aa4ac32"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:19:11.837611 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:11.837581 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c4fca91-dafc-4beb-b657-016f3aa4ac32-kube-api-access-kjb27" (OuterVolumeSpecName: "kube-api-access-kjb27") pod "0c4fca91-dafc-4beb-b657-016f3aa4ac32" (UID: "0c4fca91-dafc-4beb-b657-016f3aa4ac32"). InnerVolumeSpecName "kube-api-access-kjb27". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:19:11.841547 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:11.841523 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c4fca91-dafc-4beb-b657-016f3aa4ac32-util" (OuterVolumeSpecName: "util") pod "0c4fca91-dafc-4beb-b657-016f3aa4ac32" (UID: "0c4fca91-dafc-4beb-b657-016f3aa4ac32"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:19:11.936794 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:11.936750 2551 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0c4fca91-dafc-4beb-b657-016f3aa4ac32-util\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:19:11.936794 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:11.936789 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kjb27\" (UniqueName: \"kubernetes.io/projected/0c4fca91-dafc-4beb-b657-016f3aa4ac32-kube-api-access-kjb27\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:19:11.936794 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:11.936800 2551 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0c4fca91-dafc-4beb-b657-016f3aa4ac32-bundle\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:19:12.517299 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:12.517272 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8dk5jf" Jul 06 20:19:12.517299 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:12.517296 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8dk5jf" event={"ID":"0c4fca91-dafc-4beb-b657-016f3aa4ac32","Type":"ContainerDied","Data":"bac2a9ae9e103efd876b7b5426e95aa5986a2a71458dee6337410a08fe2889f0"} Jul 06 20:19:12.517527 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:12.517339 2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bac2a9ae9e103efd876b7b5426e95aa5986a2a71458dee6337410a08fe2889f0" Jul 06 20:19:33.190972 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:33.190936 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 06 20:19:33.191616 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:33.191593 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c4fca91-dafc-4beb-b657-016f3aa4ac32" containerName="util" Jul 06 20:19:33.191708 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:33.191621 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c4fca91-dafc-4beb-b657-016f3aa4ac32" containerName="util" Jul 06 20:19:33.191708 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:33.191639 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c4fca91-dafc-4beb-b657-016f3aa4ac32" containerName="pull" Jul 06 20:19:33.191708 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:33.191646 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c4fca91-dafc-4beb-b657-016f3aa4ac32" containerName="pull" Jul 06 20:19:33.191708 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:33.191658 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c4fca91-dafc-4beb-b657-016f3aa4ac32" containerName="extract" Jul 06 20:19:33.191708 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:33.191669 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c4fca91-dafc-4beb-b657-016f3aa4ac32" containerName="extract" Jul 06 20:19:33.191989 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:33.191781 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c4fca91-dafc-4beb-b657-016f3aa4ac32" containerName="extract" Jul 06 20:19:33.196723 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:33.196700 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jul 06 20:19:33.199806 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:33.199779 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jul 06 20:19:33.199963 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:33.199857 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-znqkw\"" Jul 06 20:19:33.200686 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:33.200672 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jul 06 20:19:33.200728 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:33.200717 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jul 06 20:19:33.229038 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:33.229008 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 06 20:19:33.248335 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:33.248292 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bphlt\" (UniqueName: \"kubernetes.io/projected/25fe3dc2-308a-4b09-ba2e-5e847e836d10-kube-api-access-bphlt\") pod \"maas-keycloak-0\" (UID: \"25fe3dc2-308a-4b09-ba2e-5e847e836d10\") " pod="keycloak-system/maas-keycloak-0" Jul 06 20:19:33.349611 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:33.349575 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bphlt\" (UniqueName: \"kubernetes.io/projected/25fe3dc2-308a-4b09-ba2e-5e847e836d10-kube-api-access-bphlt\") pod \"maas-keycloak-0\" (UID: \"25fe3dc2-308a-4b09-ba2e-5e847e836d10\") " pod="keycloak-system/maas-keycloak-0" Jul 06 20:19:33.365108 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:33.365076 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bphlt\" (UniqueName: \"kubernetes.io/projected/25fe3dc2-308a-4b09-ba2e-5e847e836d10-kube-api-access-bphlt\") pod \"maas-keycloak-0\" (UID: \"25fe3dc2-308a-4b09-ba2e-5e847e836d10\") " pod="keycloak-system/maas-keycloak-0" Jul 06 20:19:33.507416 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:33.507383 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jul 06 20:19:33.679024 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:33.678996 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 06 20:19:33.679826 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:19:33.679791 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25fe3dc2_308a_4b09_ba2e_5e847e836d10.slice/crio-794e609f341c00ed09b9adfeeeeed465a92cc148faed00d675fcad7610847bbc WatchSource:0}: Error finding container 794e609f341c00ed09b9adfeeeeed465a92cc148faed00d675fcad7610847bbc: Status 404 returned error can't find the container with id 794e609f341c00ed09b9adfeeeeed465a92cc148faed00d675fcad7610847bbc Jul 06 20:19:34.647339 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:34.647305 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"25fe3dc2-308a-4b09-ba2e-5e847e836d10","Type":"ContainerStarted","Data":"794e609f341c00ed09b9adfeeeeed465a92cc148faed00d675fcad7610847bbc"} Jul 06 20:19:40.692152 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:40.692047 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"25fe3dc2-308a-4b09-ba2e-5e847e836d10","Type":"ContainerStarted","Data":"d8883b483786c83cbfbde42e172ecb1c3de7d2f5807dcc72a9e9909ffcaa44db"} Jul 06 20:19:40.721419 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:40.721347 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=0.985681666 podStartE2EDuration="7.721321519s" totalImagesPullingTime="6.735639853s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:19:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:19:40.718379907 +0000 UTC m=+691.133579733" watchObservedRunningTime="2026-07-06 20:19:40.721321519 +0000 UTC m=+691.136521327" Jul 06 20:19:41.507799 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:41.507750 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jul 06 20:19:41.509454 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:41.509409 2551 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="25fe3dc2-308a-4b09-ba2e-5e847e836d10" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.55:9000/health/started\": dial tcp 10.133.0.55:9000: connect: connection refused" Jul 06 20:19:42.508162 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:42.508104 2551 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="25fe3dc2-308a-4b09-ba2e-5e847e836d10" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.55:9000/health/started\": dial tcp 10.133.0.55:9000: connect: connection refused" Jul 06 20:19:43.507591 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:43.507550 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jul 06 20:19:43.507965 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:43.507929 2551 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="25fe3dc2-308a-4b09-ba2e-5e847e836d10" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.55:9000/health/started\": dial tcp 10.133.0.55:9000: connect: connection refused" Jul 06 20:19:44.508442 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:44.508390 2551 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="25fe3dc2-308a-4b09-ba2e-5e847e836d10" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.55:9000/health/started\": dial tcp 10.133.0.55:9000: connect: connection refused" Jul 06 20:19:45.509025 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:45.508968 2551 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="25fe3dc2-308a-4b09-ba2e-5e847e836d10" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.55:9000/health/started\": dial tcp 10.133.0.55:9000: connect: connection refused" Jul 06 20:19:46.508196 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:46.508147 2551 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="25fe3dc2-308a-4b09-ba2e-5e847e836d10" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.55:9000/health/started\": dial tcp 10.133.0.55:9000: connect: connection refused" Jul 06 20:19:47.508929 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:47.508846 2551 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="25fe3dc2-308a-4b09-ba2e-5e847e836d10" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.55:9000/health/started\": dial tcp 10.133.0.55:9000: connect: connection refused" Jul 06 20:19:48.507947 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:48.507882 2551 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="25fe3dc2-308a-4b09-ba2e-5e847e836d10" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.55:9000/health/started\": dial tcp 10.133.0.55:9000: connect: connection refused" Jul 06 20:19:49.508103 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:49.508049 2551 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="25fe3dc2-308a-4b09-ba2e-5e847e836d10" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.55:9000/health/started\": dial tcp 10.133.0.55:9000: connect: connection refused" Jul 06 20:19:50.508281 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:50.508236 2551 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="25fe3dc2-308a-4b09-ba2e-5e847e836d10" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.55:9000/health/started\": dial tcp 10.133.0.55:9000: connect: connection refused" Jul 06 20:19:51.508255 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:51.508199 2551 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="25fe3dc2-308a-4b09-ba2e-5e847e836d10" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.55:9000/health/started\": dial tcp 10.133.0.55:9000: connect: connection refused" Jul 06 20:19:52.508543 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:52.508491 2551 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="25fe3dc2-308a-4b09-ba2e-5e847e836d10" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.55:9000/health/started\": dial tcp 10.133.0.55:9000: connect: connection refused" Jul 06 20:19:53.508492 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:53.508438 2551 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="25fe3dc2-308a-4b09-ba2e-5e847e836d10" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.55:9000/health/started\": dial tcp 10.133.0.55:9000: connect: connection refused" Jul 06 20:19:54.638335 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:54.638289 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jul 06 20:19:54.656581 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:19:54.656536 2551 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="25fe3dc2-308a-4b09-ba2e-5e847e836d10" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 20:20:04.645505 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:04.645420 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jul 06 20:20:05.732196 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:05.732156 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7p2c6"] Jul 06 20:20:05.734969 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:05.734949 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-7p2c6" Jul 06 20:20:05.743338 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:05.743310 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7p2c6"] Jul 06 20:20:05.798023 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:05.797978 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxm59\" (UniqueName: \"kubernetes.io/projected/2b0f3169-fd02-4795-8bea-b2a0eeb0d517-kube-api-access-nxm59\") pod \"authorino-5d4b67c446-7p2c6\" (UID: \"2b0f3169-fd02-4795-8bea-b2a0eeb0d517\") " pod="kuadrant-system/authorino-5d4b67c446-7p2c6" Jul 06 20:20:05.899023 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:05.898989 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxm59\" (UniqueName: \"kubernetes.io/projected/2b0f3169-fd02-4795-8bea-b2a0eeb0d517-kube-api-access-nxm59\") pod \"authorino-5d4b67c446-7p2c6\" (UID: \"2b0f3169-fd02-4795-8bea-b2a0eeb0d517\") " pod="kuadrant-system/authorino-5d4b67c446-7p2c6" Jul 06 20:20:05.910821 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:05.910788 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxm59\" (UniqueName: \"kubernetes.io/projected/2b0f3169-fd02-4795-8bea-b2a0eeb0d517-kube-api-access-nxm59\") pod \"authorino-5d4b67c446-7p2c6\" (UID: \"2b0f3169-fd02-4795-8bea-b2a0eeb0d517\") " pod="kuadrant-system/authorino-5d4b67c446-7p2c6" Jul 06 20:20:05.971765 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:05.971727 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7p2c6"] Jul 06 20:20:05.972084 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:05.972070 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-7p2c6" Jul 06 20:20:06.001178 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:06.001151 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-688647f94f-hqlgx"] Jul 06 20:20:06.004188 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:06.004169 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-688647f94f-hqlgx" Jul 06 20:20:06.019936 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:06.019876 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-688647f94f-hqlgx"] Jul 06 20:20:06.101967 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:06.101885 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrll8\" (UniqueName: \"kubernetes.io/projected/5254e0b9-eec0-49c1-b88d-9de362b453b2-kube-api-access-lrll8\") pod \"authorino-688647f94f-hqlgx\" (UID: \"5254e0b9-eec0-49c1-b88d-9de362b453b2\") " pod="kuadrant-system/authorino-688647f94f-hqlgx" Jul 06 20:20:06.123228 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:06.123203 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7p2c6"] Jul 06 20:20:06.125591 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:20:06.125554 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2b0f3169_fd02_4795_8bea_b2a0eeb0d517.slice/crio-64ac7075dbcc1d994d9f082333d4afd8da34ae06f45a3d08e2f4b05df5623232 WatchSource:0}: Error finding container 64ac7075dbcc1d994d9f082333d4afd8da34ae06f45a3d08e2f4b05df5623232: Status 404 returned error can't find the container with id 64ac7075dbcc1d994d9f082333d4afd8da34ae06f45a3d08e2f4b05df5623232 Jul 06 20:20:06.203179 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:06.203144 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrll8\" (UniqueName: \"kubernetes.io/projected/5254e0b9-eec0-49c1-b88d-9de362b453b2-kube-api-access-lrll8\") pod \"authorino-688647f94f-hqlgx\" (UID: \"5254e0b9-eec0-49c1-b88d-9de362b453b2\") " pod="kuadrant-system/authorino-688647f94f-hqlgx" Jul 06 20:20:06.213003 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:06.212978 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrll8\" (UniqueName: \"kubernetes.io/projected/5254e0b9-eec0-49c1-b88d-9de362b453b2-kube-api-access-lrll8\") pod \"authorino-688647f94f-hqlgx\" (UID: \"5254e0b9-eec0-49c1-b88d-9de362b453b2\") " pod="kuadrant-system/authorino-688647f94f-hqlgx" Jul 06 20:20:06.315534 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:06.315494 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-688647f94f-hqlgx" Jul 06 20:20:06.454669 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:06.454635 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-688647f94f-hqlgx"] Jul 06 20:20:06.456159 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:20:06.456128 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5254e0b9_eec0_49c1_b88d_9de362b453b2.slice/crio-6d4048e7e700db3c8faff36843aca7bbdb8d5c412734caec2c8fd65fae611f6d WatchSource:0}: Error finding container 6d4048e7e700db3c8faff36843aca7bbdb8d5c412734caec2c8fd65fae611f6d: Status 404 returned error can't find the container with id 6d4048e7e700db3c8faff36843aca7bbdb8d5c412734caec2c8fd65fae611f6d Jul 06 20:20:06.493164 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:06.493125 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-688647f94f-hqlgx"] Jul 06 20:20:06.526655 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:06.526617 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-74bcb76b9-8gn8q"] Jul 06 20:20:06.530048 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:06.530027 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-74bcb76b9-8gn8q" Jul 06 20:20:06.532458 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:06.532440 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 06 20:20:06.541666 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:06.541639 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-74bcb76b9-8gn8q"] Jul 06 20:20:06.607150 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:06.607104 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ffe3859d-c527-4c64-b18a-30469b83889a-tls-cert\") pod \"authorino-74bcb76b9-8gn8q\" (UID: \"ffe3859d-c527-4c64-b18a-30469b83889a\") " pod="kuadrant-system/authorino-74bcb76b9-8gn8q" Jul 06 20:20:06.607280 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:06.607184 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgq42\" (UniqueName: \"kubernetes.io/projected/ffe3859d-c527-4c64-b18a-30469b83889a-kube-api-access-cgq42\") pod \"authorino-74bcb76b9-8gn8q\" (UID: \"ffe3859d-c527-4c64-b18a-30469b83889a\") " pod="kuadrant-system/authorino-74bcb76b9-8gn8q" Jul 06 20:20:06.708480 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:06.708427 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ffe3859d-c527-4c64-b18a-30469b83889a-tls-cert\") pod \"authorino-74bcb76b9-8gn8q\" (UID: \"ffe3859d-c527-4c64-b18a-30469b83889a\") " pod="kuadrant-system/authorino-74bcb76b9-8gn8q" Jul 06 20:20:06.708699 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:06.708500 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgq42\" (UniqueName: \"kubernetes.io/projected/ffe3859d-c527-4c64-b18a-30469b83889a-kube-api-access-cgq42\") pod \"authorino-74bcb76b9-8gn8q\" (UID: \"ffe3859d-c527-4c64-b18a-30469b83889a\") " pod="kuadrant-system/authorino-74bcb76b9-8gn8q" Jul 06 20:20:06.710931 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:06.710907 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ffe3859d-c527-4c64-b18a-30469b83889a-tls-cert\") pod \"authorino-74bcb76b9-8gn8q\" (UID: \"ffe3859d-c527-4c64-b18a-30469b83889a\") " pod="kuadrant-system/authorino-74bcb76b9-8gn8q" Jul 06 20:20:06.719200 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:06.719170 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgq42\" (UniqueName: \"kubernetes.io/projected/ffe3859d-c527-4c64-b18a-30469b83889a-kube-api-access-cgq42\") pod \"authorino-74bcb76b9-8gn8q\" (UID: \"ffe3859d-c527-4c64-b18a-30469b83889a\") " pod="kuadrant-system/authorino-74bcb76b9-8gn8q" Jul 06 20:20:06.841632 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:06.841531 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-74bcb76b9-8gn8q" Jul 06 20:20:06.914997 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:06.914955 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-688647f94f-hqlgx" event={"ID":"5254e0b9-eec0-49c1-b88d-9de362b453b2","Type":"ContainerStarted","Data":"6d4048e7e700db3c8faff36843aca7bbdb8d5c412734caec2c8fd65fae611f6d"} Jul 06 20:20:06.917085 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:06.917055 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-7p2c6" event={"ID":"2b0f3169-fd02-4795-8bea-b2a0eeb0d517","Type":"ContainerStarted","Data":"587f66f7ae039d7434d37c33d8d93cff246004eeb8c0008f20ed156e2bc96cb3"} Jul 06 20:20:06.917212 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:06.917093 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-7p2c6" event={"ID":"2b0f3169-fd02-4795-8bea-b2a0eeb0d517","Type":"ContainerStarted","Data":"64ac7075dbcc1d994d9f082333d4afd8da34ae06f45a3d08e2f4b05df5623232"} Jul 06 20:20:06.917212 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:06.917125 2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-7p2c6" podUID="2b0f3169-fd02-4795-8bea-b2a0eeb0d517" containerName="authorino" containerID="cri-o://587f66f7ae039d7434d37c33d8d93cff246004eeb8c0008f20ed156e2bc96cb3" gracePeriod=30 Jul 06 20:20:06.941976 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:06.941531 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-7p2c6" podStartSLOduration=1.499764361 podStartE2EDuration="1.941507653s" totalImagesPullingTime="441.743292ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:20:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:20:06.934556615 +0000 UTC m=+717.349756422" watchObservedRunningTime="2026-07-06 20:20:06.941507653 +0000 UTC m=+717.356707459" Jul 06 20:20:07.007275 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:07.007244 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-74bcb76b9-8gn8q"] Jul 06 20:20:07.008963 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:20:07.008932 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podffe3859d_c527_4c64_b18a_30469b83889a.slice/crio-3d4e0b745ea60fe6cabcec0c2446e16fdb80be66a0c3a15dc7b53918a37fd5a0 WatchSource:0}: Error finding container 3d4e0b745ea60fe6cabcec0c2446e16fdb80be66a0c3a15dc7b53918a37fd5a0: Status 404 returned error can't find the container with id 3d4e0b745ea60fe6cabcec0c2446e16fdb80be66a0c3a15dc7b53918a37fd5a0 Jul 06 20:20:07.177787 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:07.177757 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-7p2c6" Jul 06 20:20:07.213813 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:07.213775 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nxm59\" (UniqueName: \"kubernetes.io/projected/2b0f3169-fd02-4795-8bea-b2a0eeb0d517-kube-api-access-nxm59\") pod \"2b0f3169-fd02-4795-8bea-b2a0eeb0d517\" (UID: \"2b0f3169-fd02-4795-8bea-b2a0eeb0d517\") " Jul 06 20:20:07.216046 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:07.216011 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b0f3169-fd02-4795-8bea-b2a0eeb0d517-kube-api-access-nxm59" (OuterVolumeSpecName: "kube-api-access-nxm59") pod "2b0f3169-fd02-4795-8bea-b2a0eeb0d517" (UID: "2b0f3169-fd02-4795-8bea-b2a0eeb0d517"). InnerVolumeSpecName "kube-api-access-nxm59". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:20:07.314695 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:07.314662 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nxm59\" (UniqueName: \"kubernetes.io/projected/2b0f3169-fd02-4795-8bea-b2a0eeb0d517-kube-api-access-nxm59\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:20:07.924742 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:07.924711 2551 generic.go:356] "Generic (PLEG): container finished" podID="2b0f3169-fd02-4795-8bea-b2a0eeb0d517" containerID="587f66f7ae039d7434d37c33d8d93cff246004eeb8c0008f20ed156e2bc96cb3" exitCode=0 Jul 06 20:20:07.925250 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:07.924766 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-7p2c6" Jul 06 20:20:07.925250 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:07.924799 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-7p2c6" event={"ID":"2b0f3169-fd02-4795-8bea-b2a0eeb0d517","Type":"ContainerDied","Data":"587f66f7ae039d7434d37c33d8d93cff246004eeb8c0008f20ed156e2bc96cb3"} Jul 06 20:20:07.925250 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:07.924841 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-7p2c6" event={"ID":"2b0f3169-fd02-4795-8bea-b2a0eeb0d517","Type":"ContainerDied","Data":"64ac7075dbcc1d994d9f082333d4afd8da34ae06f45a3d08e2f4b05df5623232"} Jul 06 20:20:07.925250 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:07.924864 2551 scope.go:117] "RemoveContainer" containerID="587f66f7ae039d7434d37c33d8d93cff246004eeb8c0008f20ed156e2bc96cb3" Jul 06 20:20:07.926569 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:07.926540 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-688647f94f-hqlgx" event={"ID":"5254e0b9-eec0-49c1-b88d-9de362b453b2","Type":"ContainerStarted","Data":"8136228bd761fc7b6237c2160df68d24bef2d68e43a3002abca1e7683db49363"} Jul 06 20:20:07.926692 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:07.926583 2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-688647f94f-hqlgx" podUID="5254e0b9-eec0-49c1-b88d-9de362b453b2" containerName="authorino" containerID="cri-o://8136228bd761fc7b6237c2160df68d24bef2d68e43a3002abca1e7683db49363" gracePeriod=30 Jul 06 20:20:07.928455 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:07.928428 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-74bcb76b9-8gn8q" event={"ID":"ffe3859d-c527-4c64-b18a-30469b83889a","Type":"ContainerStarted","Data":"be51ddec2160e1492c6508cc9dd9038865d84a830fd00334c8fda356df6f8ce6"} Jul 06 20:20:07.928565 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:07.928466 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-74bcb76b9-8gn8q" event={"ID":"ffe3859d-c527-4c64-b18a-30469b83889a","Type":"ContainerStarted","Data":"3d4e0b745ea60fe6cabcec0c2446e16fdb80be66a0c3a15dc7b53918a37fd5a0"} Jul 06 20:20:07.935952 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:07.935930 2551 scope.go:117] "RemoveContainer" containerID="587f66f7ae039d7434d37c33d8d93cff246004eeb8c0008f20ed156e2bc96cb3" Jul 06 20:20:07.936262 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:20:07.936240 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"587f66f7ae039d7434d37c33d8d93cff246004eeb8c0008f20ed156e2bc96cb3\": container with ID starting with 587f66f7ae039d7434d37c33d8d93cff246004eeb8c0008f20ed156e2bc96cb3 not found: ID does not exist" containerID="587f66f7ae039d7434d37c33d8d93cff246004eeb8c0008f20ed156e2bc96cb3" Jul 06 20:20:07.936332 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:07.936276 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"587f66f7ae039d7434d37c33d8d93cff246004eeb8c0008f20ed156e2bc96cb3"} err="failed to get container status \"587f66f7ae039d7434d37c33d8d93cff246004eeb8c0008f20ed156e2bc96cb3\": rpc error: code = NotFound desc = could not find container \"587f66f7ae039d7434d37c33d8d93cff246004eeb8c0008f20ed156e2bc96cb3\": container with ID starting with 587f66f7ae039d7434d37c33d8d93cff246004eeb8c0008f20ed156e2bc96cb3 not found: ID does not exist" Jul 06 20:20:07.945180 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:07.945124 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-688647f94f-hqlgx" podStartSLOduration=1.892772578 podStartE2EDuration="2.945104777s" totalImagesPullingTime="1.052332199s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:20:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:20:07.94398151 +0000 UTC m=+718.359181316" watchObservedRunningTime="2026-07-06 20:20:07.945104777 +0000 UTC m=+718.360304583" Jul 06 20:20:07.960801 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:07.960744 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-74bcb76b9-8gn8q" podStartSLOduration=1.565764513 podStartE2EDuration="1.960727011s" totalImagesPullingTime="394.962498ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:20:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:20:07.95846734 +0000 UTC m=+718.373667145" watchObservedRunningTime="2026-07-06 20:20:07.960727011 +0000 UTC m=+718.375926878" Jul 06 20:20:07.976779 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:07.976737 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7p2c6"] Jul 06 20:20:07.981502 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:07.981466 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7p2c6"] Jul 06 20:20:07.985643 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:07.985615 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-db2wx"] Jul 06 20:20:07.985824 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:07.985802 2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-db2wx" podUID="d8008368-eaed-4886-adfa-703f3d124f7d" containerName="authorino" containerID="cri-o://c6c2a51230052552d6e36d04d0e2e298fe8520ca18eab9351c248a9117e87607" gracePeriod=30 Jul 06 20:20:08.170198 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:08.170109 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b0f3169-fd02-4795-8bea-b2a0eeb0d517" path="/var/lib/kubelet/pods/2b0f3169-fd02-4795-8bea-b2a0eeb0d517/volumes" Jul 06 20:20:08.269075 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:08.269048 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-688647f94f-hqlgx" Jul 06 20:20:08.271671 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:08.271649 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-db2wx" Jul 06 20:20:08.325761 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:08.325727 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lrll8\" (UniqueName: \"kubernetes.io/projected/5254e0b9-eec0-49c1-b88d-9de362b453b2-kube-api-access-lrll8\") pod \"5254e0b9-eec0-49c1-b88d-9de362b453b2\" (UID: \"5254e0b9-eec0-49c1-b88d-9de362b453b2\") " Jul 06 20:20:08.325977 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:08.325855 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zvrmw\" (UniqueName: \"kubernetes.io/projected/d8008368-eaed-4886-adfa-703f3d124f7d-kube-api-access-zvrmw\") pod \"d8008368-eaed-4886-adfa-703f3d124f7d\" (UID: \"d8008368-eaed-4886-adfa-703f3d124f7d\") " Jul 06 20:20:08.327847 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:08.327819 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8008368-eaed-4886-adfa-703f3d124f7d-kube-api-access-zvrmw" (OuterVolumeSpecName: "kube-api-access-zvrmw") pod "d8008368-eaed-4886-adfa-703f3d124f7d" (UID: "d8008368-eaed-4886-adfa-703f3d124f7d"). InnerVolumeSpecName "kube-api-access-zvrmw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:20:08.327989 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:08.327866 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5254e0b9-eec0-49c1-b88d-9de362b453b2-kube-api-access-lrll8" (OuterVolumeSpecName: "kube-api-access-lrll8") pod "5254e0b9-eec0-49c1-b88d-9de362b453b2" (UID: "5254e0b9-eec0-49c1-b88d-9de362b453b2"). InnerVolumeSpecName "kube-api-access-lrll8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:20:08.427069 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:08.426983 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lrll8\" (UniqueName: \"kubernetes.io/projected/5254e0b9-eec0-49c1-b88d-9de362b453b2-kube-api-access-lrll8\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:20:08.427069 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:08.427011 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zvrmw\" (UniqueName: \"kubernetes.io/projected/d8008368-eaed-4886-adfa-703f3d124f7d-kube-api-access-zvrmw\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:20:08.936243 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:08.936209 2551 generic.go:356] "Generic (PLEG): container finished" podID="5254e0b9-eec0-49c1-b88d-9de362b453b2" containerID="8136228bd761fc7b6237c2160df68d24bef2d68e43a3002abca1e7683db49363" exitCode=0 Jul 06 20:20:08.936728 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:08.936257 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-688647f94f-hqlgx" Jul 06 20:20:08.936728 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:08.936297 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-688647f94f-hqlgx" event={"ID":"5254e0b9-eec0-49c1-b88d-9de362b453b2","Type":"ContainerDied","Data":"8136228bd761fc7b6237c2160df68d24bef2d68e43a3002abca1e7683db49363"} Jul 06 20:20:08.936728 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:08.936346 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-688647f94f-hqlgx" event={"ID":"5254e0b9-eec0-49c1-b88d-9de362b453b2","Type":"ContainerDied","Data":"6d4048e7e700db3c8faff36843aca7bbdb8d5c412734caec2c8fd65fae611f6d"} Jul 06 20:20:08.936728 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:08.936373 2551 scope.go:117] "RemoveContainer" containerID="8136228bd761fc7b6237c2160df68d24bef2d68e43a3002abca1e7683db49363" Jul 06 20:20:08.937587 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:08.937562 2551 generic.go:356] "Generic (PLEG): container finished" podID="d8008368-eaed-4886-adfa-703f3d124f7d" containerID="c6c2a51230052552d6e36d04d0e2e298fe8520ca18eab9351c248a9117e87607" exitCode=0 Jul 06 20:20:08.937728 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:08.937601 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-db2wx" Jul 06 20:20:08.937728 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:08.937651 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-db2wx" event={"ID":"d8008368-eaed-4886-adfa-703f3d124f7d","Type":"ContainerDied","Data":"c6c2a51230052552d6e36d04d0e2e298fe8520ca18eab9351c248a9117e87607"} Jul 06 20:20:08.937728 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:08.937689 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-db2wx" event={"ID":"d8008368-eaed-4886-adfa-703f3d124f7d","Type":"ContainerDied","Data":"6800acc507ce8d3046173c634a23b3f1cbec3a2716cc93f6dfa48b8c26a25e8a"} Jul 06 20:20:08.946020 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:08.946003 2551 scope.go:117] "RemoveContainer" containerID="8136228bd761fc7b6237c2160df68d24bef2d68e43a3002abca1e7683db49363" Jul 06 20:20:08.946289 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:20:08.946270 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8136228bd761fc7b6237c2160df68d24bef2d68e43a3002abca1e7683db49363\": container with ID starting with 8136228bd761fc7b6237c2160df68d24bef2d68e43a3002abca1e7683db49363 not found: ID does not exist" containerID="8136228bd761fc7b6237c2160df68d24bef2d68e43a3002abca1e7683db49363" Jul 06 20:20:08.946365 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:08.946295 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8136228bd761fc7b6237c2160df68d24bef2d68e43a3002abca1e7683db49363"} err="failed to get container status \"8136228bd761fc7b6237c2160df68d24bef2d68e43a3002abca1e7683db49363\": rpc error: code = NotFound desc = could not find container \"8136228bd761fc7b6237c2160df68d24bef2d68e43a3002abca1e7683db49363\": container with ID starting with 8136228bd761fc7b6237c2160df68d24bef2d68e43a3002abca1e7683db49363 not found: ID does not exist" Jul 06 20:20:08.946365 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:08.946315 2551 scope.go:117] "RemoveContainer" containerID="c6c2a51230052552d6e36d04d0e2e298fe8520ca18eab9351c248a9117e87607" Jul 06 20:20:08.955009 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:08.954981 2551 scope.go:117] "RemoveContainer" containerID="c6c2a51230052552d6e36d04d0e2e298fe8520ca18eab9351c248a9117e87607" Jul 06 20:20:08.955253 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:20:08.955230 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6c2a51230052552d6e36d04d0e2e298fe8520ca18eab9351c248a9117e87607\": container with ID starting with c6c2a51230052552d6e36d04d0e2e298fe8520ca18eab9351c248a9117e87607 not found: ID does not exist" containerID="c6c2a51230052552d6e36d04d0e2e298fe8520ca18eab9351c248a9117e87607" Jul 06 20:20:08.955305 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:08.955261 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6c2a51230052552d6e36d04d0e2e298fe8520ca18eab9351c248a9117e87607"} err="failed to get container status \"c6c2a51230052552d6e36d04d0e2e298fe8520ca18eab9351c248a9117e87607\": rpc error: code = NotFound desc = could not find container \"c6c2a51230052552d6e36d04d0e2e298fe8520ca18eab9351c248a9117e87607\": container with ID starting with c6c2a51230052552d6e36d04d0e2e298fe8520ca18eab9351c248a9117e87607 not found: ID does not exist" Jul 06 20:20:08.962562 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:08.962536 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-688647f94f-hqlgx"] Jul 06 20:20:08.965301 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:08.965280 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-688647f94f-hqlgx"] Jul 06 20:20:08.975502 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:08.975473 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-db2wx"] Jul 06 20:20:08.977977 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:08.977957 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-db2wx"] Jul 06 20:20:10.167034 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:10.166993 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5254e0b9-eec0-49c1-b88d-9de362b453b2" path="/var/lib/kubelet/pods/5254e0b9-eec0-49c1-b88d-9de362b453b2/volumes" Jul 06 20:20:10.167528 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:10.167504 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8008368-eaed-4886-adfa-703f3d124f7d" path="/var/lib/kubelet/pods/d8008368-eaed-4886-adfa-703f3d124f7d/volumes" Jul 06 20:20:12.637715 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:12.637671 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-57898744f7-h5mp2"] Jul 06 20:20:12.638147 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:12.638113 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b0f3169-fd02-4795-8bea-b2a0eeb0d517" containerName="authorino" Jul 06 20:20:12.638147 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:12.638125 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b0f3169-fd02-4795-8bea-b2a0eeb0d517" containerName="authorino" Jul 06 20:20:12.638147 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:12.638148 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5254e0b9-eec0-49c1-b88d-9de362b453b2" containerName="authorino" Jul 06 20:20:12.638289 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:12.638153 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="5254e0b9-eec0-49c1-b88d-9de362b453b2" containerName="authorino" Jul 06 20:20:12.638289 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:12.638167 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8008368-eaed-4886-adfa-703f3d124f7d" containerName="authorino" Jul 06 20:20:12.638289 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:12.638173 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8008368-eaed-4886-adfa-703f3d124f7d" containerName="authorino" Jul 06 20:20:12.638289 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:12.638252 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8008368-eaed-4886-adfa-703f3d124f7d" containerName="authorino" Jul 06 20:20:12.638289 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:12.638259 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="5254e0b9-eec0-49c1-b88d-9de362b453b2" containerName="authorino" Jul 06 20:20:12.638289 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:12.638270 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b0f3169-fd02-4795-8bea-b2a0eeb0d517" containerName="authorino" Jul 06 20:20:12.642586 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:12.642566 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57898744f7-h5mp2" Jul 06 20:20:12.645714 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:12.645692 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 06 20:20:12.645714 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:12.645707 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-w5hb5\"" Jul 06 20:20:12.645869 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:12.645798 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 06 20:20:12.651419 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:12.651392 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-57898744f7-h5mp2"] Jul 06 20:20:12.767794 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:12.767752 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sdz8\" (UniqueName: \"kubernetes.io/projected/ef23b8f9-4342-4eea-8b9e-0155849250d0-kube-api-access-6sdz8\") pod \"maas-controller-57898744f7-h5mp2\" (UID: \"ef23b8f9-4342-4eea-8b9e-0155849250d0\") " pod="opendatahub/maas-controller-57898744f7-h5mp2" Jul 06 20:20:12.768012 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:12.767858 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ef23b8f9-4342-4eea-8b9e-0155849250d0-webhook-cert\") pod \"maas-controller-57898744f7-h5mp2\" (UID: \"ef23b8f9-4342-4eea-8b9e-0155849250d0\") " pod="opendatahub/maas-controller-57898744f7-h5mp2" Jul 06 20:20:12.827647 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:12.827609 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-66d86b4f9b-vtsjr"] Jul 06 20:20:12.831155 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:12.831138 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66d86b4f9b-vtsjr" Jul 06 20:20:12.843825 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:12.843798 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-66d86b4f9b-vtsjr"] Jul 06 20:20:12.869231 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:12.869195 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sx5js\" (UniqueName: \"kubernetes.io/projected/90746b84-4f6e-4cd5-b9e6-a6f10a326832-kube-api-access-sx5js\") pod \"maas-controller-66d86b4f9b-vtsjr\" (UID: \"90746b84-4f6e-4cd5-b9e6-a6f10a326832\") " pod="opendatahub/maas-controller-66d86b4f9b-vtsjr" Jul 06 20:20:12.869630 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:12.869245 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sdz8\" (UniqueName: \"kubernetes.io/projected/ef23b8f9-4342-4eea-8b9e-0155849250d0-kube-api-access-6sdz8\") pod \"maas-controller-57898744f7-h5mp2\" (UID: \"ef23b8f9-4342-4eea-8b9e-0155849250d0\") " pod="opendatahub/maas-controller-57898744f7-h5mp2" Jul 06 20:20:12.869630 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:12.869338 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/90746b84-4f6e-4cd5-b9e6-a6f10a326832-webhook-cert\") pod \"maas-controller-66d86b4f9b-vtsjr\" (UID: \"90746b84-4f6e-4cd5-b9e6-a6f10a326832\") " pod="opendatahub/maas-controller-66d86b4f9b-vtsjr" Jul 06 20:20:12.869630 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:12.869374 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ef23b8f9-4342-4eea-8b9e-0155849250d0-webhook-cert\") pod \"maas-controller-57898744f7-h5mp2\" (UID: \"ef23b8f9-4342-4eea-8b9e-0155849250d0\") " pod="opendatahub/maas-controller-57898744f7-h5mp2" Jul 06 20:20:12.869630 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:20:12.869482 2551 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 06 20:20:12.869630 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:20:12.869533 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef23b8f9-4342-4eea-8b9e-0155849250d0-webhook-cert podName:ef23b8f9-4342-4eea-8b9e-0155849250d0 nodeName:}" failed. No retries permitted until 2026-07-06 20:20:13.369516105 +0000 UTC m=+723.784715889 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/ef23b8f9-4342-4eea-8b9e-0155849250d0-webhook-cert") pod "maas-controller-57898744f7-h5mp2" (UID: "ef23b8f9-4342-4eea-8b9e-0155849250d0") : secret "maas-controller-webhook-cert" not found Jul 06 20:20:12.880709 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:12.880683 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sdz8\" (UniqueName: \"kubernetes.io/projected/ef23b8f9-4342-4eea-8b9e-0155849250d0-kube-api-access-6sdz8\") pod \"maas-controller-57898744f7-h5mp2\" (UID: \"ef23b8f9-4342-4eea-8b9e-0155849250d0\") " pod="opendatahub/maas-controller-57898744f7-h5mp2" Jul 06 20:20:12.970717 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:12.970621 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sx5js\" (UniqueName: \"kubernetes.io/projected/90746b84-4f6e-4cd5-b9e6-a6f10a326832-kube-api-access-sx5js\") pod \"maas-controller-66d86b4f9b-vtsjr\" (UID: \"90746b84-4f6e-4cd5-b9e6-a6f10a326832\") " pod="opendatahub/maas-controller-66d86b4f9b-vtsjr" Jul 06 20:20:12.970932 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:12.970756 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/90746b84-4f6e-4cd5-b9e6-a6f10a326832-webhook-cert\") pod \"maas-controller-66d86b4f9b-vtsjr\" (UID: \"90746b84-4f6e-4cd5-b9e6-a6f10a326832\") " pod="opendatahub/maas-controller-66d86b4f9b-vtsjr" Jul 06 20:20:12.973297 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:12.973267 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/90746b84-4f6e-4cd5-b9e6-a6f10a326832-webhook-cert\") pod \"maas-controller-66d86b4f9b-vtsjr\" (UID: \"90746b84-4f6e-4cd5-b9e6-a6f10a326832\") " pod="opendatahub/maas-controller-66d86b4f9b-vtsjr" Jul 06 20:20:12.979758 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:12.979733 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sx5js\" (UniqueName: \"kubernetes.io/projected/90746b84-4f6e-4cd5-b9e6-a6f10a326832-kube-api-access-sx5js\") pod \"maas-controller-66d86b4f9b-vtsjr\" (UID: \"90746b84-4f6e-4cd5-b9e6-a6f10a326832\") " pod="opendatahub/maas-controller-66d86b4f9b-vtsjr" Jul 06 20:20:13.141937 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:13.141855 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66d86b4f9b-vtsjr" Jul 06 20:20:13.285461 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:13.285430 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-66d86b4f9b-vtsjr"] Jul 06 20:20:13.286369 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:20:13.286341 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90746b84_4f6e_4cd5_b9e6_a6f10a326832.slice/crio-e08914883aa84617a120699085c91f71b608ff4bc5ab0f4a5b66566c88b2ad48 WatchSource:0}: Error finding container e08914883aa84617a120699085c91f71b608ff4bc5ab0f4a5b66566c88b2ad48: Status 404 returned error can't find the container with id e08914883aa84617a120699085c91f71b608ff4bc5ab0f4a5b66566c88b2ad48 Jul 06 20:20:13.374626 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:13.374595 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ef23b8f9-4342-4eea-8b9e-0155849250d0-webhook-cert\") pod \"maas-controller-57898744f7-h5mp2\" (UID: \"ef23b8f9-4342-4eea-8b9e-0155849250d0\") " pod="opendatahub/maas-controller-57898744f7-h5mp2" Jul 06 20:20:13.376974 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:13.376955 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ef23b8f9-4342-4eea-8b9e-0155849250d0-webhook-cert\") pod \"maas-controller-57898744f7-h5mp2\" (UID: \"ef23b8f9-4342-4eea-8b9e-0155849250d0\") " pod="opendatahub/maas-controller-57898744f7-h5mp2" Jul 06 20:20:13.554552 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:13.554516 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57898744f7-h5mp2" Jul 06 20:20:13.702160 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:13.702132 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-57898744f7-h5mp2"] Jul 06 20:20:13.704017 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:20:13.703988 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef23b8f9_4342_4eea_8b9e_0155849250d0.slice/crio-c51cdf41315eea2547b31ebc6e984dd031d02b0bc82d4df40b901e87c0d8f762 WatchSource:0}: Error finding container c51cdf41315eea2547b31ebc6e984dd031d02b0bc82d4df40b901e87c0d8f762: Status 404 returned error can't find the container with id c51cdf41315eea2547b31ebc6e984dd031d02b0bc82d4df40b901e87c0d8f762 Jul 06 20:20:13.976095 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:13.975917 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57898744f7-h5mp2" event={"ID":"ef23b8f9-4342-4eea-8b9e-0155849250d0","Type":"ContainerStarted","Data":"c51cdf41315eea2547b31ebc6e984dd031d02b0bc82d4df40b901e87c0d8f762"} Jul 06 20:20:13.977751 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:13.977719 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66d86b4f9b-vtsjr" event={"ID":"90746b84-4f6e-4cd5-b9e6-a6f10a326832","Type":"ContainerStarted","Data":"e08914883aa84617a120699085c91f71b608ff4bc5ab0f4a5b66566c88b2ad48"} Jul 06 20:20:15.995074 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:15.995031 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57898744f7-h5mp2" event={"ID":"ef23b8f9-4342-4eea-8b9e-0155849250d0","Type":"ContainerStarted","Data":"0da247ab8fc0c85c62e377ea34607b1e36fff1b1bd1ace3f3a690c22ec151deb"} Jul 06 20:20:15.995555 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:15.995093 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-57898744f7-h5mp2" Jul 06 20:20:15.996775 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:15.996747 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66d86b4f9b-vtsjr" event={"ID":"90746b84-4f6e-4cd5-b9e6-a6f10a326832","Type":"ContainerStarted","Data":"6058e246bdef8ee645957742d828a7e1a37c9f077cc68fbc66521df1a152db75"} Jul 06 20:20:15.996958 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:15.996812 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-66d86b4f9b-vtsjr" Jul 06 20:20:16.027547 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:16.027484 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-57898744f7-h5mp2" podStartSLOduration=2.189504474 podStartE2EDuration="4.027468921s" totalImagesPullingTime="1.837964447s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:20:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:20:16.026257919 +0000 UTC m=+726.441457726" watchObservedRunningTime="2026-07-06 20:20:16.027468921 +0000 UTC m=+726.442668726" Jul 06 20:20:16.082257 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:16.082198 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-66d86b4f9b-vtsjr" podStartSLOduration=1.8347806169999998 podStartE2EDuration="4.082180726s" totalImagesPullingTime="2.247400109s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:20:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:20:16.077971028 +0000 UTC m=+726.493170833" watchObservedRunningTime="2026-07-06 20:20:16.082180726 +0000 UTC m=+726.497380530" Jul 06 20:20:27.008577 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:27.008547 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-66d86b4f9b-vtsjr" Jul 06 20:20:27.009095 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:27.008628 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-57898744f7-h5mp2" Jul 06 20:20:27.088018 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:27.087979 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-57898744f7-h5mp2"] Jul 06 20:20:27.088254 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:27.088230 2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-57898744f7-h5mp2" podUID="ef23b8f9-4342-4eea-8b9e-0155849250d0" containerName="manager" containerID="cri-o://0da247ab8fc0c85c62e377ea34607b1e36fff1b1bd1ace3f3a690c22ec151deb" gracePeriod=30 Jul 06 20:20:27.351508 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:27.351483 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57898744f7-h5mp2" Jul 06 20:20:27.523240 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:27.523146 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6sdz8\" (UniqueName: \"kubernetes.io/projected/ef23b8f9-4342-4eea-8b9e-0155849250d0-kube-api-access-6sdz8\") pod \"ef23b8f9-4342-4eea-8b9e-0155849250d0\" (UID: \"ef23b8f9-4342-4eea-8b9e-0155849250d0\") " Jul 06 20:20:27.523240 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:27.523236 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ef23b8f9-4342-4eea-8b9e-0155849250d0-webhook-cert\") pod \"ef23b8f9-4342-4eea-8b9e-0155849250d0\" (UID: \"ef23b8f9-4342-4eea-8b9e-0155849250d0\") " Jul 06 20:20:27.525474 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:27.525439 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef23b8f9-4342-4eea-8b9e-0155849250d0-kube-api-access-6sdz8" (OuterVolumeSpecName: "kube-api-access-6sdz8") pod "ef23b8f9-4342-4eea-8b9e-0155849250d0" (UID: "ef23b8f9-4342-4eea-8b9e-0155849250d0"). InnerVolumeSpecName "kube-api-access-6sdz8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:20:27.525606 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:27.525517 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef23b8f9-4342-4eea-8b9e-0155849250d0-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "ef23b8f9-4342-4eea-8b9e-0155849250d0" (UID: "ef23b8f9-4342-4eea-8b9e-0155849250d0"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 20:20:27.624408 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:27.624364 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6sdz8\" (UniqueName: \"kubernetes.io/projected/ef23b8f9-4342-4eea-8b9e-0155849250d0-kube-api-access-6sdz8\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:20:27.624408 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:27.624402 2551 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ef23b8f9-4342-4eea-8b9e-0155849250d0-webhook-cert\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:20:28.068114 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:28.068077 2551 generic.go:356] "Generic (PLEG): container finished" podID="ef23b8f9-4342-4eea-8b9e-0155849250d0" containerID="0da247ab8fc0c85c62e377ea34607b1e36fff1b1bd1ace3f3a690c22ec151deb" exitCode=0 Jul 06 20:20:28.068587 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:28.068127 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57898744f7-h5mp2" Jul 06 20:20:28.068587 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:28.068160 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57898744f7-h5mp2" event={"ID":"ef23b8f9-4342-4eea-8b9e-0155849250d0","Type":"ContainerDied","Data":"0da247ab8fc0c85c62e377ea34607b1e36fff1b1bd1ace3f3a690c22ec151deb"} Jul 06 20:20:28.068587 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:28.068215 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57898744f7-h5mp2" event={"ID":"ef23b8f9-4342-4eea-8b9e-0155849250d0","Type":"ContainerDied","Data":"c51cdf41315eea2547b31ebc6e984dd031d02b0bc82d4df40b901e87c0d8f762"} Jul 06 20:20:28.068587 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:28.068239 2551 scope.go:117] "RemoveContainer" containerID="0da247ab8fc0c85c62e377ea34607b1e36fff1b1bd1ace3f3a690c22ec151deb" Jul 06 20:20:28.078283 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:28.078264 2551 scope.go:117] "RemoveContainer" containerID="0da247ab8fc0c85c62e377ea34607b1e36fff1b1bd1ace3f3a690c22ec151deb" Jul 06 20:20:28.078619 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:20:28.078598 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0da247ab8fc0c85c62e377ea34607b1e36fff1b1bd1ace3f3a690c22ec151deb\": container with ID starting with 0da247ab8fc0c85c62e377ea34607b1e36fff1b1bd1ace3f3a690c22ec151deb not found: ID does not exist" containerID="0da247ab8fc0c85c62e377ea34607b1e36fff1b1bd1ace3f3a690c22ec151deb" Jul 06 20:20:28.078672 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:28.078630 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0da247ab8fc0c85c62e377ea34607b1e36fff1b1bd1ace3f3a690c22ec151deb"} err="failed to get container status \"0da247ab8fc0c85c62e377ea34607b1e36fff1b1bd1ace3f3a690c22ec151deb\": rpc error: code = NotFound desc = could not find container \"0da247ab8fc0c85c62e377ea34607b1e36fff1b1bd1ace3f3a690c22ec151deb\": container with ID starting with 0da247ab8fc0c85c62e377ea34607b1e36fff1b1bd1ace3f3a690c22ec151deb not found: ID does not exist" Jul 06 20:20:28.092612 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:28.092580 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-57898744f7-h5mp2"] Jul 06 20:20:28.096142 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:28.096116 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-57898744f7-h5mp2"] Jul 06 20:20:28.165307 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:28.165270 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef23b8f9-4342-4eea-8b9e-0155849250d0" path="/var/lib/kubelet/pods/ef23b8f9-4342-4eea-8b9e-0155849250d0/volumes" Jul 06 20:20:47.707596 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:47.707565 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-6ffddcb95f-tkxlg"] Jul 06 20:20:47.708159 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:47.708141 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef23b8f9-4342-4eea-8b9e-0155849250d0" containerName="manager" Jul 06 20:20:47.708217 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:47.708161 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef23b8f9-4342-4eea-8b9e-0155849250d0" containerName="manager" Jul 06 20:20:47.708260 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:47.708234 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef23b8f9-4342-4eea-8b9e-0155849250d0" containerName="manager" Jul 06 20:20:47.711539 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:47.711507 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-6ffddcb95f-tkxlg" Jul 06 20:20:47.714057 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:47.714023 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jul 06 20:20:47.724314 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:47.724291 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-6ffddcb95f-tkxlg"] Jul 06 20:20:47.801352 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:47.801309 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36efbc10-cac3-4a19-a97f-49de85ea3066-maas-api-tls\") pod \"maas-api-6ffddcb95f-tkxlg\" (UID: \"36efbc10-cac3-4a19-a97f-49de85ea3066\") " pod="opendatahub/maas-api-6ffddcb95f-tkxlg" Jul 06 20:20:47.801575 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:47.801371 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36efbc10-cac3-4a19-a97f-49de85ea3066-openshift-service-ca\") pod \"maas-api-6ffddcb95f-tkxlg\" (UID: \"36efbc10-cac3-4a19-a97f-49de85ea3066\") " pod="opendatahub/maas-api-6ffddcb95f-tkxlg" Jul 06 20:20:47.801575 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:47.801509 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-979j6\" (UniqueName: \"kubernetes.io/projected/36efbc10-cac3-4a19-a97f-49de85ea3066-kube-api-access-979j6\") pod \"maas-api-6ffddcb95f-tkxlg\" (UID: \"36efbc10-cac3-4a19-a97f-49de85ea3066\") " pod="opendatahub/maas-api-6ffddcb95f-tkxlg" Jul 06 20:20:47.902869 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:47.902825 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36efbc10-cac3-4a19-a97f-49de85ea3066-maas-api-tls\") pod \"maas-api-6ffddcb95f-tkxlg\" (UID: \"36efbc10-cac3-4a19-a97f-49de85ea3066\") " pod="opendatahub/maas-api-6ffddcb95f-tkxlg" Jul 06 20:20:47.903089 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:47.902937 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36efbc10-cac3-4a19-a97f-49de85ea3066-openshift-service-ca\") pod \"maas-api-6ffddcb95f-tkxlg\" (UID: \"36efbc10-cac3-4a19-a97f-49de85ea3066\") " pod="opendatahub/maas-api-6ffddcb95f-tkxlg" Jul 06 20:20:47.903089 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:20:47.903042 2551 secret.go:189] Couldn't get secret opendatahub/maas-api-serving-cert: secret "maas-api-serving-cert" not found Jul 06 20:20:47.903089 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:47.903066 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-979j6\" (UniqueName: \"kubernetes.io/projected/36efbc10-cac3-4a19-a97f-49de85ea3066-kube-api-access-979j6\") pod \"maas-api-6ffddcb95f-tkxlg\" (UID: \"36efbc10-cac3-4a19-a97f-49de85ea3066\") " pod="opendatahub/maas-api-6ffddcb95f-tkxlg" Jul 06 20:20:47.903221 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:20:47.903113 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36efbc10-cac3-4a19-a97f-49de85ea3066-maas-api-tls podName:36efbc10-cac3-4a19-a97f-49de85ea3066 nodeName:}" failed. No retries permitted until 2026-07-06 20:20:48.403090975 +0000 UTC m=+758.818290773 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/36efbc10-cac3-4a19-a97f-49de85ea3066-maas-api-tls") pod "maas-api-6ffddcb95f-tkxlg" (UID: "36efbc10-cac3-4a19-a97f-49de85ea3066") : secret "maas-api-serving-cert" not found Jul 06 20:20:47.903543 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:47.903523 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36efbc10-cac3-4a19-a97f-49de85ea3066-openshift-service-ca\") pod \"maas-api-6ffddcb95f-tkxlg\" (UID: \"36efbc10-cac3-4a19-a97f-49de85ea3066\") " pod="opendatahub/maas-api-6ffddcb95f-tkxlg" Jul 06 20:20:47.912294 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:47.912266 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-979j6\" (UniqueName: \"kubernetes.io/projected/36efbc10-cac3-4a19-a97f-49de85ea3066-kube-api-access-979j6\") pod \"maas-api-6ffddcb95f-tkxlg\" (UID: \"36efbc10-cac3-4a19-a97f-49de85ea3066\") " pod="opendatahub/maas-api-6ffddcb95f-tkxlg" Jul 06 20:20:48.394705 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:48.394665 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59f49fd4f6-9dznl"] Jul 06 20:20:48.397599 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:48.397575 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59f49fd4f6-9dznl" Jul 06 20:20:48.401431 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:48.401408 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 06 20:20:48.401580 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:48.401562 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-kghrm\"" Jul 06 20:20:48.408090 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:48.408061 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1079f02-d8a4-42e9-8243-d6190e5debe8-plugins-config-volume\") pod \"payload-pre-processing-59f49fd4f6-9dznl\" (UID: \"a1079f02-d8a4-42e9-8243-d6190e5debe8\") " pod="openshift-ingress/payload-pre-processing-59f49fd4f6-9dznl" Jul 06 20:20:48.408219 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:48.408149 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36efbc10-cac3-4a19-a97f-49de85ea3066-maas-api-tls\") pod \"maas-api-6ffddcb95f-tkxlg\" (UID: \"36efbc10-cac3-4a19-a97f-49de85ea3066\") " pod="opendatahub/maas-api-6ffddcb95f-tkxlg" Jul 06 20:20:48.408292 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:48.408250 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22985\" (UniqueName: \"kubernetes.io/projected/a1079f02-d8a4-42e9-8243-d6190e5debe8-kube-api-access-22985\") pod \"payload-pre-processing-59f49fd4f6-9dznl\" (UID: \"a1079f02-d8a4-42e9-8243-d6190e5debe8\") " pod="openshift-ingress/payload-pre-processing-59f49fd4f6-9dznl" Jul 06 20:20:48.410566 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:48.410542 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36efbc10-cac3-4a19-a97f-49de85ea3066-maas-api-tls\") pod \"maas-api-6ffddcb95f-tkxlg\" (UID: \"36efbc10-cac3-4a19-a97f-49de85ea3066\") " pod="opendatahub/maas-api-6ffddcb95f-tkxlg" Jul 06 20:20:48.418012 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:48.417986 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59f49fd4f6-9dznl"] Jul 06 20:20:48.437388 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:48.437353 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-7bbcb654f4-bb8vm"] Jul 06 20:20:48.440131 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:48.440113 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7bbcb654f4-bb8vm" Jul 06 20:20:48.457273 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:48.457242 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7bbcb654f4-bb8vm"] Jul 06 20:20:48.509039 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:48.508993 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1f04227-efd2-4114-8b18-1fccfd54dd1b-plugins-config-volume\") pod \"payload-processing-7bbcb654f4-bb8vm\" (UID: \"e1f04227-efd2-4114-8b18-1fccfd54dd1b\") " pod="openshift-ingress/payload-processing-7bbcb654f4-bb8vm" Jul 06 20:20:48.509225 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:48.509065 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22985\" (UniqueName: \"kubernetes.io/projected/a1079f02-d8a4-42e9-8243-d6190e5debe8-kube-api-access-22985\") pod \"payload-pre-processing-59f49fd4f6-9dznl\" (UID: \"a1079f02-d8a4-42e9-8243-d6190e5debe8\") " pod="openshift-ingress/payload-pre-processing-59f49fd4f6-9dznl" Jul 06 20:20:48.509225 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:48.509191 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1079f02-d8a4-42e9-8243-d6190e5debe8-plugins-config-volume\") pod \"payload-pre-processing-59f49fd4f6-9dznl\" (UID: \"a1079f02-d8a4-42e9-8243-d6190e5debe8\") " pod="openshift-ingress/payload-pre-processing-59f49fd4f6-9dznl" Jul 06 20:20:48.509344 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:48.509240 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvdd4\" (UniqueName: \"kubernetes.io/projected/e1f04227-efd2-4114-8b18-1fccfd54dd1b-kube-api-access-pvdd4\") pod \"payload-processing-7bbcb654f4-bb8vm\" (UID: \"e1f04227-efd2-4114-8b18-1fccfd54dd1b\") " pod="openshift-ingress/payload-processing-7bbcb654f4-bb8vm" Jul 06 20:20:48.510171 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:48.510149 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1079f02-d8a4-42e9-8243-d6190e5debe8-plugins-config-volume\") pod \"payload-pre-processing-59f49fd4f6-9dznl\" (UID: \"a1079f02-d8a4-42e9-8243-d6190e5debe8\") " pod="openshift-ingress/payload-pre-processing-59f49fd4f6-9dznl" Jul 06 20:20:48.524355 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:48.524321 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-22985\" (UniqueName: \"kubernetes.io/projected/a1079f02-d8a4-42e9-8243-d6190e5debe8-kube-api-access-22985\") pod \"payload-pre-processing-59f49fd4f6-9dznl\" (UID: \"a1079f02-d8a4-42e9-8243-d6190e5debe8\") " pod="openshift-ingress/payload-pre-processing-59f49fd4f6-9dznl" Jul 06 20:20:48.610402 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:48.610370 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvdd4\" (UniqueName: \"kubernetes.io/projected/e1f04227-efd2-4114-8b18-1fccfd54dd1b-kube-api-access-pvdd4\") pod \"payload-processing-7bbcb654f4-bb8vm\" (UID: \"e1f04227-efd2-4114-8b18-1fccfd54dd1b\") " pod="openshift-ingress/payload-processing-7bbcb654f4-bb8vm" Jul 06 20:20:48.610598 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:48.610450 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1f04227-efd2-4114-8b18-1fccfd54dd1b-plugins-config-volume\") pod \"payload-processing-7bbcb654f4-bb8vm\" (UID: \"e1f04227-efd2-4114-8b18-1fccfd54dd1b\") " pod="openshift-ingress/payload-processing-7bbcb654f4-bb8vm" Jul 06 20:20:48.611209 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:48.611189 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1f04227-efd2-4114-8b18-1fccfd54dd1b-plugins-config-volume\") pod \"payload-processing-7bbcb654f4-bb8vm\" (UID: \"e1f04227-efd2-4114-8b18-1fccfd54dd1b\") " pod="openshift-ingress/payload-processing-7bbcb654f4-bb8vm" Jul 06 20:20:48.619371 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:48.619337 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvdd4\" (UniqueName: \"kubernetes.io/projected/e1f04227-efd2-4114-8b18-1fccfd54dd1b-kube-api-access-pvdd4\") pod \"payload-processing-7bbcb654f4-bb8vm\" (UID: \"e1f04227-efd2-4114-8b18-1fccfd54dd1b\") " pod="openshift-ingress/payload-processing-7bbcb654f4-bb8vm" Jul 06 20:20:48.623634 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:48.623614 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-6ffddcb95f-tkxlg" Jul 06 20:20:48.709772 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:48.709730 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59f49fd4f6-9dznl" Jul 06 20:20:48.751798 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:48.751757 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7bbcb654f4-bb8vm" Jul 06 20:20:48.796476 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:48.796313 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-6ffddcb95f-tkxlg"] Jul 06 20:20:48.799325 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:20:48.799264 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36efbc10_cac3_4a19_a97f_49de85ea3066.slice/crio-75440f4c0bc1f0b2cb3730aacf189261bb1d0a8c8c82ec8338d49be2d8966634 WatchSource:0}: Error finding container 75440f4c0bc1f0b2cb3730aacf189261bb1d0a8c8c82ec8338d49be2d8966634: Status 404 returned error can't find the container with id 75440f4c0bc1f0b2cb3730aacf189261bb1d0a8c8c82ec8338d49be2d8966634 Jul 06 20:20:48.896792 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:48.896569 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59f49fd4f6-9dznl"] Jul 06 20:20:48.923675 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:48.923630 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7bbcb654f4-bb8vm"] Jul 06 20:20:48.924706 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:20:48.924672 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1f04227_efd2_4114_8b18_1fccfd54dd1b.slice/crio-873a1f56fe27c35e8defde7a5819cac84353344d23ad1e70562f11a686602e1d WatchSource:0}: Error finding container 873a1f56fe27c35e8defde7a5819cac84353344d23ad1e70562f11a686602e1d: Status 404 returned error can't find the container with id 873a1f56fe27c35e8defde7a5819cac84353344d23ad1e70562f11a686602e1d Jul 06 20:20:49.205373 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:49.205335 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7bbcb654f4-bb8vm" event={"ID":"e1f04227-efd2-4114-8b18-1fccfd54dd1b","Type":"ContainerStarted","Data":"873a1f56fe27c35e8defde7a5819cac84353344d23ad1e70562f11a686602e1d"} Jul 06 20:20:49.206632 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:49.206603 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59f49fd4f6-9dznl" event={"ID":"a1079f02-d8a4-42e9-8243-d6190e5debe8","Type":"ContainerStarted","Data":"9510f8c366007f0d0fd1b844fcb6c30dc2955c76fd13800baf8f3818eaeb9a2c"} Jul 06 20:20:49.207775 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:49.207743 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-6ffddcb95f-tkxlg" event={"ID":"36efbc10-cac3-4a19-a97f-49de85ea3066","Type":"ContainerStarted","Data":"75440f4c0bc1f0b2cb3730aacf189261bb1d0a8c8c82ec8338d49be2d8966634"} Jul 06 20:20:53.267713 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:53.267658 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7bbcb654f4-bb8vm" event={"ID":"e1f04227-efd2-4114-8b18-1fccfd54dd1b","Type":"ContainerStarted","Data":"0af003cb81795d03af74de0fc0a0a78ca888c738a6cc042fb8e6b3f71c96adad"} Jul 06 20:20:53.268208 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:53.267804 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7bbcb654f4-bb8vm" Jul 06 20:20:53.269453 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:53.269423 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59f49fd4f6-9dznl" event={"ID":"a1079f02-d8a4-42e9-8243-d6190e5debe8","Type":"ContainerStarted","Data":"55d7488c6a64efebea5650fdeb08cf4b9ed5a2fe6186edc78bef7a68b04dd6cd"} Jul 06 20:20:53.269622 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:53.269478 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59f49fd4f6-9dznl" Jul 06 20:20:53.271059 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:53.271031 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-6ffddcb95f-tkxlg" event={"ID":"36efbc10-cac3-4a19-a97f-49de85ea3066","Type":"ContainerStarted","Data":"c4c020016de584d8493ed62fabc5c86e4a7605de507f3c595d275863283f6fba"} Jul 06 20:20:53.271195 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:53.271093 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-6ffddcb95f-tkxlg" Jul 06 20:20:53.285437 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:53.285384 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-7bbcb654f4-bb8vm" podStartSLOduration=1.605012887 podStartE2EDuration="5.285367676s" totalImagesPullingTime="3.680354789s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:20:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:20:53.284218418 +0000 UTC m=+763.699418244" watchObservedRunningTime="2026-07-06 20:20:53.285367676 +0000 UTC m=+763.700567480" Jul 06 20:20:53.300325 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:53.300271 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59f49fd4f6-9dznl" podStartSLOduration=1.601622007 podStartE2EDuration="5.300252459s" totalImagesPullingTime="3.698630452s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:20:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:20:53.298938541 +0000 UTC m=+763.714138372" watchObservedRunningTime="2026-07-06 20:20:53.300252459 +0000 UTC m=+763.715452263" Jul 06 20:20:53.315598 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:53.315539 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-6ffddcb95f-tkxlg" podStartSLOduration=2.550023129 podStartE2EDuration="6.315525995s" totalImagesPullingTime="3.765502866s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:20:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:20:53.314062072 +0000 UTC m=+763.729261868" watchObservedRunningTime="2026-07-06 20:20:53.315525995 +0000 UTC m=+763.730725800" Jul 06 20:20:56.297117 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:56.297087 2551 generic.go:356] "Generic (PLEG): container finished" podID="f45bd611-bbaa-4f58-a40a-528378e433bb" containerID="bc2cb4b4c37996f49c8062f55c4bdc94d065a58e67b86a5a3f948d49716caabf" exitCode=0 Jul 06 20:20:56.297505 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:56.297158 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-qj8pt" event={"ID":"f45bd611-bbaa-4f58-a40a-528378e433bb","Type":"ContainerDied","Data":"bc2cb4b4c37996f49c8062f55c4bdc94d065a58e67b86a5a3f948d49716caabf"} Jul 06 20:20:56.297658 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:56.297640 2551 scope.go:117] "RemoveContainer" containerID="bc2cb4b4c37996f49c8062f55c4bdc94d065a58e67b86a5a3f948d49716caabf" Jul 06 20:20:56.298759 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:56.298729 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:20:56.298830 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:56.298817 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:20:56.298868 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:56.298859 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:20:57.305213 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:57.305180 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-qj8pt" event={"ID":"f45bd611-bbaa-4f58-a40a-528378e433bb","Type":"ContainerStarted","Data":"6e82bfddc6dbdceb52623fa384582fbb3612f7716a2786e462d68b1e6584dc00"} Jul 06 20:20:57.329337 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:57.329305 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-qj8pt" Jul 06 20:20:57.331144 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:57.330806 2551 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-qj8pt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 06 20:20:57.331144 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:57.330861 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-qj8pt" podUID="f45bd611-bbaa-4f58-a40a-528378e433bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 06 20:20:58.330229 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:58.330182 2551 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-qj8pt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 06 20:20:58.330870 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:58.330267 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-qj8pt" podUID="f45bd611-bbaa-4f58-a40a-528378e433bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 06 20:20:59.285562 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:59.285531 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-6ffddcb95f-tkxlg" Jul 06 20:20:59.329508 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:59.329474 2551 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-qj8pt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 06 20:20:59.329710 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:20:59.329539 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-qj8pt" podUID="f45bd611-bbaa-4f58-a40a-528378e433bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 06 20:21:00.329155 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:00.329117 2551 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-qj8pt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 06 20:21:00.329597 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:00.329199 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-qj8pt" podUID="f45bd611-bbaa-4f58-a40a-528378e433bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 06 20:21:00.915106 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:00.915068 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 06 20:21:00.915308 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:00.915286 2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="keycloak-system/maas-keycloak-0" podUID="25fe3dc2-308a-4b09-ba2e-5e847e836d10" containerName="keycloak" containerID="cri-o://d8883b483786c83cbfbde42e172ecb1c3de7d2f5807dcc72a9e9909ffcaa44db" gracePeriod=30 Jul 06 20:21:01.241322 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:01.241230 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-qj8pt" Jul 06 20:21:01.334596 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:01.334560 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-qj8pt" Jul 06 20:21:01.336062 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:01.336039 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-qj8pt" Jul 06 20:21:02.989455 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:02.989426 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jul 06 20:21:03.095063 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:03.094966 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bphlt\" (UniqueName: \"kubernetes.io/projected/25fe3dc2-308a-4b09-ba2e-5e847e836d10-kube-api-access-bphlt\") pod \"25fe3dc2-308a-4b09-ba2e-5e847e836d10\" (UID: \"25fe3dc2-308a-4b09-ba2e-5e847e836d10\") " Jul 06 20:21:03.097174 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:03.097148 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25fe3dc2-308a-4b09-ba2e-5e847e836d10-kube-api-access-bphlt" (OuterVolumeSpecName: "kube-api-access-bphlt") pod "25fe3dc2-308a-4b09-ba2e-5e847e836d10" (UID: "25fe3dc2-308a-4b09-ba2e-5e847e836d10"). InnerVolumeSpecName "kube-api-access-bphlt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:21:03.196160 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:03.196120 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bphlt\" (UniqueName: \"kubernetes.io/projected/25fe3dc2-308a-4b09-ba2e-5e847e836d10-kube-api-access-bphlt\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:21:03.346716 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:03.346624 2551 generic.go:356] "Generic (PLEG): container finished" podID="25fe3dc2-308a-4b09-ba2e-5e847e836d10" containerID="d8883b483786c83cbfbde42e172ecb1c3de7d2f5807dcc72a9e9909ffcaa44db" exitCode=143 Jul 06 20:21:03.346716 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:03.346674 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jul 06 20:21:03.346932 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:03.346716 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"25fe3dc2-308a-4b09-ba2e-5e847e836d10","Type":"ContainerDied","Data":"d8883b483786c83cbfbde42e172ecb1c3de7d2f5807dcc72a9e9909ffcaa44db"} Jul 06 20:21:03.346932 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:03.346767 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"25fe3dc2-308a-4b09-ba2e-5e847e836d10","Type":"ContainerDied","Data":"794e609f341c00ed09b9adfeeeeed465a92cc148faed00d675fcad7610847bbc"} Jul 06 20:21:03.346932 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:03.346787 2551 scope.go:117] "RemoveContainer" containerID="d8883b483786c83cbfbde42e172ecb1c3de7d2f5807dcc72a9e9909ffcaa44db" Jul 06 20:21:03.357433 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:03.357406 2551 scope.go:117] "RemoveContainer" containerID="d8883b483786c83cbfbde42e172ecb1c3de7d2f5807dcc72a9e9909ffcaa44db" Jul 06 20:21:03.357713 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:21:03.357691 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8883b483786c83cbfbde42e172ecb1c3de7d2f5807dcc72a9e9909ffcaa44db\": container with ID starting with d8883b483786c83cbfbde42e172ecb1c3de7d2f5807dcc72a9e9909ffcaa44db not found: ID does not exist" containerID="d8883b483786c83cbfbde42e172ecb1c3de7d2f5807dcc72a9e9909ffcaa44db" Jul 06 20:21:03.357763 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:03.357723 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8883b483786c83cbfbde42e172ecb1c3de7d2f5807dcc72a9e9909ffcaa44db"} err="failed to get container status \"d8883b483786c83cbfbde42e172ecb1c3de7d2f5807dcc72a9e9909ffcaa44db\": rpc error: code = NotFound desc = could not find container \"d8883b483786c83cbfbde42e172ecb1c3de7d2f5807dcc72a9e9909ffcaa44db\": container with ID starting with d8883b483786c83cbfbde42e172ecb1c3de7d2f5807dcc72a9e9909ffcaa44db not found: ID does not exist" Jul 06 20:21:03.372804 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:03.372779 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 06 20:21:03.379497 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:03.379474 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 06 20:21:03.413951 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:03.413923 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 06 20:21:03.414380 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:03.414367 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25fe3dc2-308a-4b09-ba2e-5e847e836d10" containerName="keycloak" Jul 06 20:21:03.414423 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:03.414382 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="25fe3dc2-308a-4b09-ba2e-5e847e836d10" containerName="keycloak" Jul 06 20:21:03.414465 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:03.414456 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="25fe3dc2-308a-4b09-ba2e-5e847e836d10" containerName="keycloak" Jul 06 20:21:03.417410 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:03.417391 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jul 06 20:21:03.420322 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:03.420298 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"keycloak-test-realms\"" Jul 06 20:21:03.420457 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:03.420301 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-znqkw\"" Jul 06 20:21:03.420457 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:03.420302 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jul 06 20:21:03.420457 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:03.420302 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jul 06 20:21:03.420457 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:03.420425 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jul 06 20:21:03.428876 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:03.428853 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 06 20:21:03.499627 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:03.499589 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dxxt\" (UniqueName: \"kubernetes.io/projected/a792203d-0ab2-44db-b237-6d1b4be679e7-kube-api-access-7dxxt\") pod \"maas-keycloak-0\" (UID: \"a792203d-0ab2-44db-b237-6d1b4be679e7\") " pod="keycloak-system/maas-keycloak-0" Jul 06 20:21:03.499830 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:03.499676 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/a792203d-0ab2-44db-b237-6d1b4be679e7-test-realms\") pod \"maas-keycloak-0\" (UID: \"a792203d-0ab2-44db-b237-6d1b4be679e7\") " pod="keycloak-system/maas-keycloak-0" Jul 06 20:21:03.601153 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:03.601036 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dxxt\" (UniqueName: \"kubernetes.io/projected/a792203d-0ab2-44db-b237-6d1b4be679e7-kube-api-access-7dxxt\") pod \"maas-keycloak-0\" (UID: \"a792203d-0ab2-44db-b237-6d1b4be679e7\") " pod="keycloak-system/maas-keycloak-0" Jul 06 20:21:03.601153 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:03.601120 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/a792203d-0ab2-44db-b237-6d1b4be679e7-test-realms\") pod \"maas-keycloak-0\" (UID: \"a792203d-0ab2-44db-b237-6d1b4be679e7\") " pod="keycloak-system/maas-keycloak-0" Jul 06 20:21:03.601831 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:03.601810 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/a792203d-0ab2-44db-b237-6d1b4be679e7-test-realms\") pod \"maas-keycloak-0\" (UID: \"a792203d-0ab2-44db-b237-6d1b4be679e7\") " pod="keycloak-system/maas-keycloak-0" Jul 06 20:21:03.610648 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:03.610611 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dxxt\" (UniqueName: \"kubernetes.io/projected/a792203d-0ab2-44db-b237-6d1b4be679e7-kube-api-access-7dxxt\") pod \"maas-keycloak-0\" (UID: \"a792203d-0ab2-44db-b237-6d1b4be679e7\") " pod="keycloak-system/maas-keycloak-0" Jul 06 20:21:03.728537 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:03.728495 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jul 06 20:21:03.877139 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:03.877114 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 06 20:21:03.878846 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:21:03.878822 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda792203d_0ab2_44db_b237_6d1b4be679e7.slice/crio-ea039cbda1d2cd2cd49e9ac1ada68b9c1dd26eaf6033db22b803f89a999b640d WatchSource:0}: Error finding container ea039cbda1d2cd2cd49e9ac1ada68b9c1dd26eaf6033db22b803f89a999b640d: Status 404 returned error can't find the container with id ea039cbda1d2cd2cd49e9ac1ada68b9c1dd26eaf6033db22b803f89a999b640d Jul 06 20:21:04.165317 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:04.165231 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25fe3dc2-308a-4b09-ba2e-5e847e836d10" path="/var/lib/kubelet/pods/25fe3dc2-308a-4b09-ba2e-5e847e836d10/volumes" Jul 06 20:21:04.281845 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:04.281814 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7bbcb654f4-bb8vm" Jul 06 20:21:04.282013 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:04.281924 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59f49fd4f6-9dznl" Jul 06 20:21:04.354907 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:04.354855 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"a792203d-0ab2-44db-b237-6d1b4be679e7","Type":"ContainerStarted","Data":"cdef6e1bf74ddeb9c7a80fce6adc193beb5bdb160b615959302e602529eda3f5"} Jul 06 20:21:04.355094 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:04.354942 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"a792203d-0ab2-44db-b237-6d1b4be679e7","Type":"ContainerStarted","Data":"ea039cbda1d2cd2cd49e9ac1ada68b9c1dd26eaf6033db22b803f89a999b640d"} Jul 06 20:21:04.377631 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:04.377562 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=1.010309559 podStartE2EDuration="1.377536827s" totalImagesPullingTime="367.227268ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:21:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:21:04.376007705 +0000 UTC m=+774.791207511" watchObservedRunningTime="2026-07-06 20:21:04.377536827 +0000 UTC m=+774.792736634" Jul 06 20:21:04.729575 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:04.729528 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jul 06 20:21:04.731413 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:04.731376 2551 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="a792203d-0ab2-44db-b237-6d1b4be679e7" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.64:9000/health/started\": dial tcp 10.133.0.64:9000: connect: connection refused" Jul 06 20:21:05.729683 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:05.729632 2551 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="a792203d-0ab2-44db-b237-6d1b4be679e7" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.64:9000/health/started\": dial tcp 10.133.0.64:9000: connect: connection refused" Jul 06 20:21:06.729330 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:06.729273 2551 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="a792203d-0ab2-44db-b237-6d1b4be679e7" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.64:9000/health/started\": dial tcp 10.133.0.64:9000: connect: connection refused" Jul 06 20:21:07.729325 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:07.729270 2551 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="a792203d-0ab2-44db-b237-6d1b4be679e7" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.64:9000/health/started\": dial tcp 10.133.0.64:9000: connect: connection refused" Jul 06 20:21:08.729489 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:08.729429 2551 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="a792203d-0ab2-44db-b237-6d1b4be679e7" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.64:9000/health/started\": dial tcp 10.133.0.64:9000: connect: connection refused" Jul 06 20:21:09.729392 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:09.729335 2551 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="a792203d-0ab2-44db-b237-6d1b4be679e7" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.64:9000/health/started\": dial tcp 10.133.0.64:9000: connect: connection refused" Jul 06 20:21:10.729596 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:10.729547 2551 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="a792203d-0ab2-44db-b237-6d1b4be679e7" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.64:9000/health/started\": dial tcp 10.133.0.64:9000: connect: connection refused" Jul 06 20:21:11.730016 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:11.729960 2551 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="a792203d-0ab2-44db-b237-6d1b4be679e7" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.64:9000/health/started\": dial tcp 10.133.0.64:9000: connect: connection refused" Jul 06 20:21:12.729220 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:12.729166 2551 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="a792203d-0ab2-44db-b237-6d1b4be679e7" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.64:9000/health/started\": dial tcp 10.133.0.64:9000: connect: connection refused" Jul 06 20:21:13.729616 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:13.729578 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jul 06 20:21:13.730161 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:13.729973 2551 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="a792203d-0ab2-44db-b237-6d1b4be679e7" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.64:9000/health/started\": dial tcp 10.133.0.64:9000: connect: connection refused" Jul 06 20:21:14.729146 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:14.729092 2551 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="a792203d-0ab2-44db-b237-6d1b4be679e7" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.64:9000/health/started\": dial tcp 10.133.0.64:9000: connect: connection refused" Jul 06 20:21:15.729876 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:15.729822 2551 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="a792203d-0ab2-44db-b237-6d1b4be679e7" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.64:9000/health/started\": dial tcp 10.133.0.64:9000: connect: connection refused" Jul 06 20:21:16.729762 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:16.729716 2551 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="a792203d-0ab2-44db-b237-6d1b4be679e7" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.64:9000/health/started\": dial tcp 10.133.0.64:9000: connect: connection refused" Jul 06 20:21:17.823780 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:17.823727 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jul 06 20:21:17.839211 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:17.839163 2551 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="a792203d-0ab2-44db-b237-6d1b4be679e7" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 20:21:27.828588 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:27.828534 2551 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="a792203d-0ab2-44db-b237-6d1b4be679e7" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 20:21:37.830166 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:37.830058 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jul 06 20:21:48.591475 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:48.591437 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-79544f47c7-rfh8m"] Jul 06 20:21:48.595211 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:48.595190 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79544f47c7-rfh8m" Jul 06 20:21:48.597881 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:48.597852 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"authorino-oidc-ca\"" Jul 06 20:21:48.603425 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:48.603217 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-79544f47c7-rfh8m"] Jul 06 20:21:48.791154 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:48.791120 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/43504ba9-a368-4c66-93d5-67ec231b8858-tls-cert\") pod \"authorino-79544f47c7-rfh8m\" (UID: \"43504ba9-a368-4c66-93d5-67ec231b8858\") " pod="kuadrant-system/authorino-79544f47c7-rfh8m" Jul 06 20:21:48.791353 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:48.791163 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2gq8\" (UniqueName: \"kubernetes.io/projected/43504ba9-a368-4c66-93d5-67ec231b8858-kube-api-access-f2gq8\") pod \"authorino-79544f47c7-rfh8m\" (UID: \"43504ba9-a368-4c66-93d5-67ec231b8858\") " pod="kuadrant-system/authorino-79544f47c7-rfh8m" Jul 06 20:21:48.791353 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:48.791297 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/43504ba9-a368-4c66-93d5-67ec231b8858-oidc-ca\") pod \"authorino-79544f47c7-rfh8m\" (UID: \"43504ba9-a368-4c66-93d5-67ec231b8858\") " pod="kuadrant-system/authorino-79544f47c7-rfh8m" Jul 06 20:21:48.892239 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:48.892142 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2gq8\" (UniqueName: \"kubernetes.io/projected/43504ba9-a368-4c66-93d5-67ec231b8858-kube-api-access-f2gq8\") pod \"authorino-79544f47c7-rfh8m\" (UID: \"43504ba9-a368-4c66-93d5-67ec231b8858\") " pod="kuadrant-system/authorino-79544f47c7-rfh8m" Jul 06 20:21:48.892239 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:48.892208 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/43504ba9-a368-4c66-93d5-67ec231b8858-oidc-ca\") pod \"authorino-79544f47c7-rfh8m\" (UID: \"43504ba9-a368-4c66-93d5-67ec231b8858\") " pod="kuadrant-system/authorino-79544f47c7-rfh8m" Jul 06 20:21:48.892427 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:48.892275 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/43504ba9-a368-4c66-93d5-67ec231b8858-tls-cert\") pod \"authorino-79544f47c7-rfh8m\" (UID: \"43504ba9-a368-4c66-93d5-67ec231b8858\") " pod="kuadrant-system/authorino-79544f47c7-rfh8m" Jul 06 20:21:48.892988 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:48.892966 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/43504ba9-a368-4c66-93d5-67ec231b8858-oidc-ca\") pod \"authorino-79544f47c7-rfh8m\" (UID: \"43504ba9-a368-4c66-93d5-67ec231b8858\") " pod="kuadrant-system/authorino-79544f47c7-rfh8m" Jul 06 20:21:48.894762 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:48.894740 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/43504ba9-a368-4c66-93d5-67ec231b8858-tls-cert\") pod \"authorino-79544f47c7-rfh8m\" (UID: \"43504ba9-a368-4c66-93d5-67ec231b8858\") " pod="kuadrant-system/authorino-79544f47c7-rfh8m" Jul 06 20:21:48.901297 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:48.901274 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2gq8\" (UniqueName: \"kubernetes.io/projected/43504ba9-a368-4c66-93d5-67ec231b8858-kube-api-access-f2gq8\") pod \"authorino-79544f47c7-rfh8m\" (UID: \"43504ba9-a368-4c66-93d5-67ec231b8858\") " pod="kuadrant-system/authorino-79544f47c7-rfh8m" Jul 06 20:21:48.907090 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:48.907069 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79544f47c7-rfh8m" Jul 06 20:21:49.059055 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:49.057684 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-79544f47c7-rfh8m"] Jul 06 20:21:49.061513 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:21:49.061466 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod43504ba9_a368_4c66_93d5_67ec231b8858.slice/crio-4871abc337c00ac8a63fd01af428ab9b346894e8d65c8aa8882d14bd49e25dd3 WatchSource:0}: Error finding container 4871abc337c00ac8a63fd01af428ab9b346894e8d65c8aa8882d14bd49e25dd3: Status 404 returned error can't find the container with id 4871abc337c00ac8a63fd01af428ab9b346894e8d65c8aa8882d14bd49e25dd3 Jul 06 20:21:49.685633 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:49.685589 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79544f47c7-rfh8m" event={"ID":"43504ba9-a368-4c66-93d5-67ec231b8858","Type":"ContainerStarted","Data":"f45a24da85ff98abecd4966a2cff4f4f9c873b10972891ce40261de22ffc3a48"} Jul 06 20:21:49.686031 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:49.685647 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79544f47c7-rfh8m" event={"ID":"43504ba9-a368-4c66-93d5-67ec231b8858","Type":"ContainerStarted","Data":"4871abc337c00ac8a63fd01af428ab9b346894e8d65c8aa8882d14bd49e25dd3"} Jul 06 20:21:49.704638 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:49.704585 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-79544f47c7-rfh8m" podStartSLOduration=1.169100311 podStartE2EDuration="1.704567985s" totalImagesPullingTime="535.467674ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:21:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:21:49.701203092 +0000 UTC m=+820.116402898" watchObservedRunningTime="2026-07-06 20:21:49.704567985 +0000 UTC m=+820.119767790" Jul 06 20:21:49.728589 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:49.728556 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-74bcb76b9-8gn8q"] Jul 06 20:21:49.728807 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:49.728761 2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-74bcb76b9-8gn8q" podUID="ffe3859d-c527-4c64-b18a-30469b83889a" containerName="authorino" containerID="cri-o://be51ddec2160e1492c6508cc9dd9038865d84a830fd00334c8fda356df6f8ce6" gracePeriod=30 Jul 06 20:21:49.984330 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:49.984302 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-74bcb76b9-8gn8q" Jul 06 20:21:50.103327 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:50.103281 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cgq42\" (UniqueName: \"kubernetes.io/projected/ffe3859d-c527-4c64-b18a-30469b83889a-kube-api-access-cgq42\") pod \"ffe3859d-c527-4c64-b18a-30469b83889a\" (UID: \"ffe3859d-c527-4c64-b18a-30469b83889a\") " Jul 06 20:21:50.103327 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:50.103334 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ffe3859d-c527-4c64-b18a-30469b83889a-tls-cert\") pod \"ffe3859d-c527-4c64-b18a-30469b83889a\" (UID: \"ffe3859d-c527-4c64-b18a-30469b83889a\") " Jul 06 20:21:50.105492 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:50.105459 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffe3859d-c527-4c64-b18a-30469b83889a-kube-api-access-cgq42" (OuterVolumeSpecName: "kube-api-access-cgq42") pod "ffe3859d-c527-4c64-b18a-30469b83889a" (UID: "ffe3859d-c527-4c64-b18a-30469b83889a"). InnerVolumeSpecName "kube-api-access-cgq42". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:21:50.115724 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:50.115692 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ffe3859d-c527-4c64-b18a-30469b83889a-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "ffe3859d-c527-4c64-b18a-30469b83889a" (UID: "ffe3859d-c527-4c64-b18a-30469b83889a"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 20:21:50.204226 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:50.204188 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cgq42\" (UniqueName: \"kubernetes.io/projected/ffe3859d-c527-4c64-b18a-30469b83889a-kube-api-access-cgq42\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:21:50.204226 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:50.204223 2551 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ffe3859d-c527-4c64-b18a-30469b83889a-tls-cert\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:21:50.692847 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:50.692807 2551 generic.go:356] "Generic (PLEG): container finished" podID="ffe3859d-c527-4c64-b18a-30469b83889a" containerID="be51ddec2160e1492c6508cc9dd9038865d84a830fd00334c8fda356df6f8ce6" exitCode=0 Jul 06 20:21:50.693282 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:50.692876 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-74bcb76b9-8gn8q" Jul 06 20:21:50.693282 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:50.692928 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-74bcb76b9-8gn8q" event={"ID":"ffe3859d-c527-4c64-b18a-30469b83889a","Type":"ContainerDied","Data":"be51ddec2160e1492c6508cc9dd9038865d84a830fd00334c8fda356df6f8ce6"} Jul 06 20:21:50.693282 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:50.692975 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-74bcb76b9-8gn8q" event={"ID":"ffe3859d-c527-4c64-b18a-30469b83889a","Type":"ContainerDied","Data":"3d4e0b745ea60fe6cabcec0c2446e16fdb80be66a0c3a15dc7b53918a37fd5a0"} Jul 06 20:21:50.693282 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:50.692994 2551 scope.go:117] "RemoveContainer" containerID="be51ddec2160e1492c6508cc9dd9038865d84a830fd00334c8fda356df6f8ce6" Jul 06 20:21:50.702751 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:50.702730 2551 scope.go:117] "RemoveContainer" containerID="be51ddec2160e1492c6508cc9dd9038865d84a830fd00334c8fda356df6f8ce6" Jul 06 20:21:50.703102 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:21:50.703083 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be51ddec2160e1492c6508cc9dd9038865d84a830fd00334c8fda356df6f8ce6\": container with ID starting with be51ddec2160e1492c6508cc9dd9038865d84a830fd00334c8fda356df6f8ce6 not found: ID does not exist" containerID="be51ddec2160e1492c6508cc9dd9038865d84a830fd00334c8fda356df6f8ce6" Jul 06 20:21:50.703159 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:50.703112 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be51ddec2160e1492c6508cc9dd9038865d84a830fd00334c8fda356df6f8ce6"} err="failed to get container status \"be51ddec2160e1492c6508cc9dd9038865d84a830fd00334c8fda356df6f8ce6\": rpc error: code = NotFound desc = could not find container \"be51ddec2160e1492c6508cc9dd9038865d84a830fd00334c8fda356df6f8ce6\": container with ID starting with be51ddec2160e1492c6508cc9dd9038865d84a830fd00334c8fda356df6f8ce6 not found: ID does not exist" Jul 06 20:21:50.711347 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:50.711317 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-74bcb76b9-8gn8q"] Jul 06 20:21:50.715519 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:50.715495 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-74bcb76b9-8gn8q"] Jul 06 20:21:52.165347 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:21:52.165313 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ffe3859d-c527-4c64-b18a-30469b83889a" path="/var/lib/kubelet/pods/ffe3859d-c527-4c64-b18a-30469b83889a/volumes" Jul 06 20:23:16.012834 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:23:16.012803 2551 kubelet.go:1648] "Image garbage collection succeeded" Jul 06 20:24:13.456461 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:24:13.456427 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-69b4996c48-6hvqm"] Jul 06 20:24:13.456950 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:24:13.456872 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffe3859d-c527-4c64-b18a-30469b83889a" containerName="authorino" Jul 06 20:24:13.456950 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:24:13.456883 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffe3859d-c527-4c64-b18a-30469b83889a" containerName="authorino" Jul 06 20:24:13.457049 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:24:13.457001 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="ffe3859d-c527-4c64-b18a-30469b83889a" containerName="authorino" Jul 06 20:24:13.460115 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:24:13.460097 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-69b4996c48-6hvqm" Jul 06 20:24:13.472174 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:24:13.472144 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-69b4996c48-6hvqm"] Jul 06 20:24:13.472750 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:24:13.472722 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhh2q\" (UniqueName: \"kubernetes.io/projected/9b710a57-1591-476e-95a8-d93adf65c48d-kube-api-access-fhh2q\") pod \"authorino-69b4996c48-6hvqm\" (UID: \"9b710a57-1591-476e-95a8-d93adf65c48d\") " pod="kuadrant-system/authorino-69b4996c48-6hvqm" Jul 06 20:24:13.472926 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:24:13.472784 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/9b710a57-1591-476e-95a8-d93adf65c48d-oidc-ca\") pod \"authorino-69b4996c48-6hvqm\" (UID: \"9b710a57-1591-476e-95a8-d93adf65c48d\") " pod="kuadrant-system/authorino-69b4996c48-6hvqm" Jul 06 20:24:13.472986 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:24:13.472952 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9b710a57-1591-476e-95a8-d93adf65c48d-tls-cert\") pod \"authorino-69b4996c48-6hvqm\" (UID: \"9b710a57-1591-476e-95a8-d93adf65c48d\") " pod="kuadrant-system/authorino-69b4996c48-6hvqm" Jul 06 20:24:13.574097 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:24:13.574049 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhh2q\" (UniqueName: \"kubernetes.io/projected/9b710a57-1591-476e-95a8-d93adf65c48d-kube-api-access-fhh2q\") pod \"authorino-69b4996c48-6hvqm\" (UID: \"9b710a57-1591-476e-95a8-d93adf65c48d\") " pod="kuadrant-system/authorino-69b4996c48-6hvqm" Jul 06 20:24:13.574097 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:24:13.574109 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/9b710a57-1591-476e-95a8-d93adf65c48d-oidc-ca\") pod \"authorino-69b4996c48-6hvqm\" (UID: \"9b710a57-1591-476e-95a8-d93adf65c48d\") " pod="kuadrant-system/authorino-69b4996c48-6hvqm" Jul 06 20:24:13.574324 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:24:13.574174 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9b710a57-1591-476e-95a8-d93adf65c48d-tls-cert\") pod \"authorino-69b4996c48-6hvqm\" (UID: \"9b710a57-1591-476e-95a8-d93adf65c48d\") " pod="kuadrant-system/authorino-69b4996c48-6hvqm" Jul 06 20:24:13.574813 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:24:13.574787 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/9b710a57-1591-476e-95a8-d93adf65c48d-oidc-ca\") pod \"authorino-69b4996c48-6hvqm\" (UID: \"9b710a57-1591-476e-95a8-d93adf65c48d\") " pod="kuadrant-system/authorino-69b4996c48-6hvqm" Jul 06 20:24:13.576582 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:24:13.576555 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9b710a57-1591-476e-95a8-d93adf65c48d-tls-cert\") pod \"authorino-69b4996c48-6hvqm\" (UID: \"9b710a57-1591-476e-95a8-d93adf65c48d\") " pod="kuadrant-system/authorino-69b4996c48-6hvqm" Jul 06 20:24:13.590152 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:24:13.590123 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhh2q\" (UniqueName: \"kubernetes.io/projected/9b710a57-1591-476e-95a8-d93adf65c48d-kube-api-access-fhh2q\") pod \"authorino-69b4996c48-6hvqm\" (UID: \"9b710a57-1591-476e-95a8-d93adf65c48d\") " pod="kuadrant-system/authorino-69b4996c48-6hvqm" Jul 06 20:24:13.771539 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:24:13.771497 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-69b4996c48-6hvqm" Jul 06 20:24:13.915982 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:24:13.915953 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-69b4996c48-6hvqm"] Jul 06 20:24:13.917503 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:24:13.917470 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9b710a57_1591_476e_95a8_d93adf65c48d.slice/crio-444613bb92916f551fc3c330559766d4fcfdf31d049a1e75454ab6df425e43d8 WatchSource:0}: Error finding container 444613bb92916f551fc3c330559766d4fcfdf31d049a1e75454ab6df425e43d8: Status 404 returned error can't find the container with id 444613bb92916f551fc3c330559766d4fcfdf31d049a1e75454ab6df425e43d8 Jul 06 20:24:13.918879 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:24:13.918853 2551 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 20:24:14.607209 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:24:14.607101 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-69b4996c48-6hvqm" event={"ID":"9b710a57-1591-476e-95a8-d93adf65c48d","Type":"ContainerStarted","Data":"76cf914c82295e7afeca3e1ee70fa7929c11fb23ac6d888481f4d9549200e9d2"} Jul 06 20:24:14.607209 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:24:14.607165 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-69b4996c48-6hvqm" event={"ID":"9b710a57-1591-476e-95a8-d93adf65c48d","Type":"ContainerStarted","Data":"444613bb92916f551fc3c330559766d4fcfdf31d049a1e75454ab6df425e43d8"} Jul 06 20:24:14.628544 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:24:14.628475 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-69b4996c48-6hvqm" podStartSLOduration=1.1899261270000001 podStartE2EDuration="1.628452932s" totalImagesPullingTime="438.526805ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:24:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:24:14.625251763 +0000 UTC m=+965.040451568" watchObservedRunningTime="2026-07-06 20:24:14.628452932 +0000 UTC m=+965.043652740" Jul 06 20:24:14.668916 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:24:14.667099 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-79544f47c7-rfh8m"] Jul 06 20:24:14.668916 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:24:14.667329 2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-79544f47c7-rfh8m" podUID="43504ba9-a368-4c66-93d5-67ec231b8858" containerName="authorino" containerID="cri-o://f45a24da85ff98abecd4966a2cff4f4f9c873b10972891ce40261de22ffc3a48" gracePeriod=30 Jul 06 20:24:14.957386 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:24:14.957358 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79544f47c7-rfh8m" Jul 06 20:24:14.987187 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:24:14.987150 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/43504ba9-a368-4c66-93d5-67ec231b8858-oidc-ca\") pod \"43504ba9-a368-4c66-93d5-67ec231b8858\" (UID: \"43504ba9-a368-4c66-93d5-67ec231b8858\") " Jul 06 20:24:14.987391 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:24:14.987205 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f2gq8\" (UniqueName: \"kubernetes.io/projected/43504ba9-a368-4c66-93d5-67ec231b8858-kube-api-access-f2gq8\") pod \"43504ba9-a368-4c66-93d5-67ec231b8858\" (UID: \"43504ba9-a368-4c66-93d5-67ec231b8858\") " Jul 06 20:24:14.987391 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:24:14.987306 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/43504ba9-a368-4c66-93d5-67ec231b8858-tls-cert\") pod \"43504ba9-a368-4c66-93d5-67ec231b8858\" (UID: \"43504ba9-a368-4c66-93d5-67ec231b8858\") " Jul 06 20:24:14.990659 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:24:14.990614 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43504ba9-a368-4c66-93d5-67ec231b8858-kube-api-access-f2gq8" (OuterVolumeSpecName: "kube-api-access-f2gq8") pod "43504ba9-a368-4c66-93d5-67ec231b8858" (UID: "43504ba9-a368-4c66-93d5-67ec231b8858"). InnerVolumeSpecName "kube-api-access-f2gq8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:24:14.993864 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:24:14.993825 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43504ba9-a368-4c66-93d5-67ec231b8858-oidc-ca" (OuterVolumeSpecName: "oidc-ca") pod "43504ba9-a368-4c66-93d5-67ec231b8858" (UID: "43504ba9-a368-4c66-93d5-67ec231b8858"). InnerVolumeSpecName "oidc-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:24:15.001633 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:24:15.001598 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43504ba9-a368-4c66-93d5-67ec231b8858-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "43504ba9-a368-4c66-93d5-67ec231b8858" (UID: "43504ba9-a368-4c66-93d5-67ec231b8858"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 20:24:15.088783 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:24:15.088744 2551 reconciler_common.go:299] "Volume detached for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/43504ba9-a368-4c66-93d5-67ec231b8858-oidc-ca\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:24:15.088783 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:24:15.088781 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f2gq8\" (UniqueName: \"kubernetes.io/projected/43504ba9-a368-4c66-93d5-67ec231b8858-kube-api-access-f2gq8\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:24:15.088783 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:24:15.088790 2551 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/43504ba9-a368-4c66-93d5-67ec231b8858-tls-cert\") on node \"ip-10-0-137-178.ec2.internal\" DevicePath \"\"" Jul 06 20:24:15.616641 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:24:15.616593 2551 generic.go:356] "Generic (PLEG): container finished" podID="43504ba9-a368-4c66-93d5-67ec231b8858" containerID="f45a24da85ff98abecd4966a2cff4f4f9c873b10972891ce40261de22ffc3a48" exitCode=0 Jul 06 20:24:15.617158 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:24:15.616652 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79544f47c7-rfh8m" Jul 06 20:24:15.617158 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:24:15.616681 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79544f47c7-rfh8m" event={"ID":"43504ba9-a368-4c66-93d5-67ec231b8858","Type":"ContainerDied","Data":"f45a24da85ff98abecd4966a2cff4f4f9c873b10972891ce40261de22ffc3a48"} Jul 06 20:24:15.617158 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:24:15.616748 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79544f47c7-rfh8m" event={"ID":"43504ba9-a368-4c66-93d5-67ec231b8858","Type":"ContainerDied","Data":"4871abc337c00ac8a63fd01af428ab9b346894e8d65c8aa8882d14bd49e25dd3"} Jul 06 20:24:15.617158 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:24:15.616770 2551 scope.go:117] "RemoveContainer" containerID="f45a24da85ff98abecd4966a2cff4f4f9c873b10972891ce40261de22ffc3a48" Jul 06 20:24:15.628569 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:24:15.628546 2551 scope.go:117] "RemoveContainer" containerID="f45a24da85ff98abecd4966a2cff4f4f9c873b10972891ce40261de22ffc3a48" Jul 06 20:24:15.628977 ip-10-0-137-178 kubenswrapper[2551]: E0706 20:24:15.628946 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f45a24da85ff98abecd4966a2cff4f4f9c873b10972891ce40261de22ffc3a48\": container with ID starting with f45a24da85ff98abecd4966a2cff4f4f9c873b10972891ce40261de22ffc3a48 not found: ID does not exist" containerID="f45a24da85ff98abecd4966a2cff4f4f9c873b10972891ce40261de22ffc3a48" Jul 06 20:24:15.629083 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:24:15.628986 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f45a24da85ff98abecd4966a2cff4f4f9c873b10972891ce40261de22ffc3a48"} err="failed to get container status \"f45a24da85ff98abecd4966a2cff4f4f9c873b10972891ce40261de22ffc3a48\": rpc error: code = NotFound desc = could not find container \"f45a24da85ff98abecd4966a2cff4f4f9c873b10972891ce40261de22ffc3a48\": container with ID starting with f45a24da85ff98abecd4966a2cff4f4f9c873b10972891ce40261de22ffc3a48 not found: ID does not exist" Jul 06 20:24:15.651817 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:24:15.651766 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-79544f47c7-rfh8m"] Jul 06 20:24:15.655684 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:24:15.655651 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-79544f47c7-rfh8m"] Jul 06 20:24:16.166268 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:24:16.166231 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43504ba9-a368-4c66-93d5-67ec231b8858" path="/var/lib/kubelet/pods/43504ba9-a368-4c66-93d5-67ec231b8858/volumes" Jul 06 20:24:55.880661 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:24:55.880627 2551 generic.go:356] "Generic (PLEG): container finished" podID="36efbc10-cac3-4a19-a97f-49de85ea3066" containerID="c4c020016de584d8493ed62fabc5c86e4a7605de507f3c595d275863283f6fba" exitCode=137 Jul 06 20:24:55.881123 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:24:55.880696 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-6ffddcb95f-tkxlg" event={"ID":"36efbc10-cac3-4a19-a97f-49de85ea3066","Type":"ContainerDied","Data":"c4c020016de584d8493ed62fabc5c86e4a7605de507f3c595d275863283f6fba"} Jul 06 20:24:55.881123 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:24:55.881085 2551 scope.go:117] "RemoveContainer" containerID="c4c020016de584d8493ed62fabc5c86e4a7605de507f3c595d275863283f6fba" Jul 06 20:24:56.080555 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:24:56.080521 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-69b4996c48-6hvqm_9b710a57-1591-476e-95a8-d93adf65c48d/authorino/0.log" Jul 06 20:24:56.889320 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:24:56.889282 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-6ffddcb95f-tkxlg" event={"ID":"36efbc10-cac3-4a19-a97f-49de85ea3066","Type":"ContainerStarted","Data":"ffaf6b603ab7f33bfe62924cc294f558c7f92bca46eb3a74f891ee00bf1114f9"} Jul 06 20:24:56.889699 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:24:56.889528 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-6ffddcb95f-tkxlg" Jul 06 20:25:01.074885 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:01.074858 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-7qd2m_c0ee877c-70e6-4960-9924-c3aeb9f21455/manager/0.log" Jul 06 20:25:01.199148 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:01.199114 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-6nlkr_a6ef5c09-60d4-44fd-b968-f0bd1770147d/manager/0.log" Jul 06 20:25:01.316034 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:01.316004 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-6ffddcb95f-tkxlg_36efbc10-cac3-4a19-a97f-49de85ea3066/maas-api/1.log" Jul 06 20:25:01.442327 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:01.442240 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-66d86b4f9b-vtsjr_90746b84-4f6e-4cd5-b9e6-a6f10a326832/manager/0.log" Jul 06 20:25:01.560399 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:01.560370 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-gp85h_3fe7b2ec-32e7-4b5e-bfad-7f0f1ff752c2/server/0.log" Jul 06 20:25:01.684486 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:01.684457 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-r5svr_e7240fb9-24ab-412a-b7dc-d12f95a61060/manager/0.log" Jul 06 20:25:02.104334 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:02.104301 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7f6455564d-rwbgc_944d69e8-62db-410d-b9e6-a9e476aef550/copy-manifests/0.log" Jul 06 20:25:02.112596 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:02.112570 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7f6455564d-rwbgc_944d69e8-62db-410d-b9e6-a9e476aef550/manager/0.log" Jul 06 20:25:02.899334 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:02.899304 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-6ffddcb95f-tkxlg" Jul 06 20:25:03.050398 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:03.050368 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqd6k_5ea8e837-257b-4fde-915b-96062fc3bb8f/util/0.log" Jul 06 20:25:03.057380 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:03.057356 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqd6k_5ea8e837-257b-4fde-915b-96062fc3bb8f/pull/0.log" Jul 06 20:25:03.065181 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:03.065160 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqd6k_5ea8e837-257b-4fde-915b-96062fc3bb8f/extract/0.log" Jul 06 20:25:03.189117 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:03.189040 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027c5q_61c662a3-a1eb-4fcb-9861-747c57829e38/util/0.log" Jul 06 20:25:03.197661 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:03.197629 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027c5q_61c662a3-a1eb-4fcb-9861-747c57829e38/pull/0.log" Jul 06 20:25:03.204860 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:03.204838 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027c5q_61c662a3-a1eb-4fcb-9861-747c57829e38/extract/0.log" Jul 06 20:25:03.325701 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:03.325665 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732g7dd_a46a28db-8e4c-458a-ba7b-c5c5ea614b97/extract/0.log" Jul 06 20:25:03.332287 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:03.332268 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732g7dd_a46a28db-8e4c-458a-ba7b-c5c5ea614b97/util/0.log" Jul 06 20:25:03.339700 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:03.339680 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732g7dd_a46a28db-8e4c-458a-ba7b-c5c5ea614b97/pull/0.log" Jul 06 20:25:03.456357 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:03.456273 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bc9fh_3bf83d9c-23f6-4f11-8c0f-d70f27447216/extract/0.log" Jul 06 20:25:03.463668 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:03.463635 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bc9fh_3bf83d9c-23f6-4f11-8c0f-d70f27447216/util/0.log" Jul 06 20:25:03.472457 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:03.472434 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bc9fh_3bf83d9c-23f6-4f11-8c0f-d70f27447216/pull/0.log" Jul 06 20:25:03.603847 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:03.603810 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-69b4996c48-6hvqm_9b710a57-1591-476e-95a8-d93adf65c48d/authorino/0.log" Jul 06 20:25:03.867225 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:03.867192 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-smk9d_f48a1ab3-cef0-4cc0-8041-a5d97e784452/manager/0.log" Jul 06 20:25:03.984724 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:03.984695 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-gz2pn_09faf81c-561c-4eaf-86d1-a6e5856291c8/kuadrant-console-plugin/0.log" Jul 06 20:25:04.105142 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:04.105110 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-6zck8_286a63f8-e79e-434e-b82b-214061777c43/registry-server/0.log" Jul 06 20:25:04.235340 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:04.235260 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-2xfsm_426dcf4a-6ebc-4101-b15f-3d0f08d27bca/manager/0.log" Jul 06 20:25:04.500839 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:04.500816 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-zjvkk_e8251546-8656-4cdb-b307-69f62aba522c/manager/0.log" Jul 06 20:25:04.885000 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:04.884911 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678xwrmt_6dd6cbd6-1df4-4233-bca9-6e44587c8759/istio-proxy/0.log" Jul 06 20:25:05.127333 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:05.127305 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-68b56f95f5-6krcp_fad5f926-d0b6-43b0-b124-92a9d7478625/kube-auth-proxy/0.log" Jul 06 20:25:05.379334 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:05.379296 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-qj8pt_f45bd611-bbaa-4f58-a40a-528378e433bb/istio-proxy/1.log" Jul 06 20:25:05.505677 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:05.505645 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59f49fd4f6-9dznl_a1079f02-d8a4-42e9-8243-d6190e5debe8/payload-pre-processing/0.log" Jul 06 20:25:05.630317 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:05.630233 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bbcb654f4-bb8vm_e1f04227-efd2-4114-8b18-1fccfd54dd1b/payload-processing/0.log" Jul 06 20:25:05.763967 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:05.763936 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-668d9f84f5-rrtln_735538e5-306c-44a6-a934-30f463187f08/router/0.log" Jul 06 20:25:13.830678 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:13.830643 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-g6t4k_d6909872-c920-4e38-a17a-11ad51d50379/global-pull-secret-syncer/0.log" Jul 06 20:25:14.009303 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:14.009271 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-5b2wn_1bbf9b1e-36f5-487b-a66e-55dc425fd1a0/konnectivity-agent/0.log" Jul 06 20:25:14.105474 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:14.105388 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-178.ec2.internal_ac36f75d0ec38f2ed1aabc8706b8a395/haproxy/0.log" Jul 06 20:25:18.957788 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:18.957759 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqd6k_5ea8e837-257b-4fde-915b-96062fc3bb8f/extract/0.log" Jul 06 20:25:18.982694 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:18.982668 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqd6k_5ea8e837-257b-4fde-915b-96062fc3bb8f/util/0.log" Jul 06 20:25:19.010433 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:19.010390 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqd6k_5ea8e837-257b-4fde-915b-96062fc3bb8f/pull/0.log" Jul 06 20:25:19.044724 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:19.044695 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027c5q_61c662a3-a1eb-4fcb-9861-747c57829e38/extract/0.log" Jul 06 20:25:19.081826 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:19.081793 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027c5q_61c662a3-a1eb-4fcb-9861-747c57829e38/util/0.log" Jul 06 20:25:19.109059 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:19.109035 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027c5q_61c662a3-a1eb-4fcb-9861-747c57829e38/pull/0.log" Jul 06 20:25:19.148534 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:19.148503 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732g7dd_a46a28db-8e4c-458a-ba7b-c5c5ea614b97/extract/0.log" Jul 06 20:25:19.177885 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:19.177853 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732g7dd_a46a28db-8e4c-458a-ba7b-c5c5ea614b97/util/0.log" Jul 06 20:25:19.212142 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:19.212059 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732g7dd_a46a28db-8e4c-458a-ba7b-c5c5ea614b97/pull/0.log" Jul 06 20:25:19.286276 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:19.286242 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bc9fh_3bf83d9c-23f6-4f11-8c0f-d70f27447216/extract/0.log" Jul 06 20:25:19.316749 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:19.316721 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bc9fh_3bf83d9c-23f6-4f11-8c0f-d70f27447216/util/0.log" Jul 06 20:25:19.341148 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:19.341120 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bc9fh_3bf83d9c-23f6-4f11-8c0f-d70f27447216/pull/0.log" Jul 06 20:25:19.619762 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:19.619719 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-69b4996c48-6hvqm_9b710a57-1591-476e-95a8-d93adf65c48d/authorino/0.log" Jul 06 20:25:19.681322 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:19.681287 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-smk9d_f48a1ab3-cef0-4cc0-8041-a5d97e784452/manager/0.log" Jul 06 20:25:19.726044 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:19.726017 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-gz2pn_09faf81c-561c-4eaf-86d1-a6e5856291c8/kuadrant-console-plugin/0.log" Jul 06 20:25:19.770593 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:19.770561 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-6zck8_286a63f8-e79e-434e-b82b-214061777c43/registry-server/0.log" Jul 06 20:25:19.819099 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:19.819068 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-2xfsm_426dcf4a-6ebc-4101-b15f-3d0f08d27bca/manager/0.log" Jul 06 20:25:19.917943 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:19.917823 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-zjvkk_e8251546-8656-4cdb-b307-69f62aba522c/manager/0.log" Jul 06 20:25:21.609367 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:21.609329 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6af805a7-e59a-4c68-8b0c-39625fbba718/alertmanager/0.log" Jul 06 20:25:21.633598 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:21.633565 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6af805a7-e59a-4c68-8b0c-39625fbba718/config-reloader/0.log" Jul 06 20:25:21.654808 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:21.654769 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6af805a7-e59a-4c68-8b0c-39625fbba718/kube-rbac-proxy-web/0.log" Jul 06 20:25:21.680524 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:21.680498 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6af805a7-e59a-4c68-8b0c-39625fbba718/kube-rbac-proxy/0.log" Jul 06 20:25:21.703243 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:21.703217 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6af805a7-e59a-4c68-8b0c-39625fbba718/kube-rbac-proxy-metric/0.log" Jul 06 20:25:21.726514 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:21.726488 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6af805a7-e59a-4c68-8b0c-39625fbba718/prom-label-proxy/0.log" Jul 06 20:25:21.749524 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:21.749493 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6af805a7-e59a-4c68-8b0c-39625fbba718/init-config-reloader/0.log" Jul 06 20:25:21.909110 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:21.909027 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5456bc64bb-dlqgd_56e7150a-0f69-48ff-8e0d-0029a85a1341/metrics-server/0.log" Jul 06 20:25:22.134581 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:22.134553 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-v2rzd_7226da11-2638-40c4-92fd-d5e2801dc9ac/node-exporter/0.log" Jul 06 20:25:22.162534 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:22.162459 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-v2rzd_7226da11-2638-40c4-92fd-d5e2801dc9ac/kube-rbac-proxy/0.log" Jul 06 20:25:22.188333 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:22.188306 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-v2rzd_7226da11-2638-40c4-92fd-d5e2801dc9ac/init-textfile/0.log" Jul 06 20:25:22.220062 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:22.220024 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-78wq2_c82a93e6-feb4-4977-889d-175921fd1395/kube-rbac-proxy-main/0.log" Jul 06 20:25:22.242076 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:22.242053 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-78wq2_c82a93e6-feb4-4977-889d-175921fd1395/kube-rbac-proxy-self/0.log" Jul 06 20:25:22.262601 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:22.262575 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-78wq2_c82a93e6-feb4-4977-889d-175921fd1395/openshift-state-metrics/0.log" Jul 06 20:25:22.578625 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:22.578598 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7b95cf59d8-42hsw_a1ae0621-1054-4d52-83d9-e3cb51bc509a/telemeter-client/0.log" Jul 06 20:25:22.629258 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:22.629231 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7b95cf59d8-42hsw_a1ae0621-1054-4d52-83d9-e3cb51bc509a/reload/0.log" Jul 06 20:25:22.664313 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:22.664279 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7b95cf59d8-42hsw_a1ae0621-1054-4d52-83d9-e3cb51bc509a/kube-rbac-proxy/0.log" Jul 06 20:25:23.591434 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:23.591393 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-2868l/perf-node-gather-daemonset-7hz9c"] Jul 06 20:25:23.591969 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:23.591947 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43504ba9-a368-4c66-93d5-67ec231b8858" containerName="authorino" Jul 06 20:25:23.591969 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:23.591970 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="43504ba9-a368-4c66-93d5-67ec231b8858" containerName="authorino" Jul 06 20:25:23.592150 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:23.592077 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="43504ba9-a368-4c66-93d5-67ec231b8858" containerName="authorino" Jul 06 20:25:23.595654 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:23.595636 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2868l/perf-node-gather-daemonset-7hz9c" Jul 06 20:25:23.598275 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:23.598251 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-2868l\"/\"kube-root-ca.crt\"" Jul 06 20:25:23.598275 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:23.598264 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-2868l\"/\"openshift-service-ca.crt\"" Jul 06 20:25:23.599042 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:23.599029 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-2868l\"/\"default-dockercfg-59xnn\"" Jul 06 20:25:23.606288 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:23.606264 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2868l/perf-node-gather-daemonset-7hz9c"] Jul 06 20:25:23.701203 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:23.701166 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a2850ee5-4831-46bf-ae97-cea8a0a0130b-lib-modules\") pod \"perf-node-gather-daemonset-7hz9c\" (UID: \"a2850ee5-4831-46bf-ae97-cea8a0a0130b\") " pod="openshift-must-gather-2868l/perf-node-gather-daemonset-7hz9c" Jul 06 20:25:23.701695 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:23.701230 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a2850ee5-4831-46bf-ae97-cea8a0a0130b-sys\") pod \"perf-node-gather-daemonset-7hz9c\" (UID: \"a2850ee5-4831-46bf-ae97-cea8a0a0130b\") " pod="openshift-must-gather-2868l/perf-node-gather-daemonset-7hz9c" Jul 06 20:25:23.701695 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:23.701287 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vkp8\" (UniqueName: \"kubernetes.io/projected/a2850ee5-4831-46bf-ae97-cea8a0a0130b-kube-api-access-9vkp8\") pod \"perf-node-gather-daemonset-7hz9c\" (UID: \"a2850ee5-4831-46bf-ae97-cea8a0a0130b\") " pod="openshift-must-gather-2868l/perf-node-gather-daemonset-7hz9c" Jul 06 20:25:23.701695 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:23.701312 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a2850ee5-4831-46bf-ae97-cea8a0a0130b-podres\") pod \"perf-node-gather-daemonset-7hz9c\" (UID: \"a2850ee5-4831-46bf-ae97-cea8a0a0130b\") " pod="openshift-must-gather-2868l/perf-node-gather-daemonset-7hz9c" Jul 06 20:25:23.701695 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:23.701350 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a2850ee5-4831-46bf-ae97-cea8a0a0130b-proc\") pod \"perf-node-gather-daemonset-7hz9c\" (UID: \"a2850ee5-4831-46bf-ae97-cea8a0a0130b\") " pod="openshift-must-gather-2868l/perf-node-gather-daemonset-7hz9c" Jul 06 20:25:23.802407 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:23.802373 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a2850ee5-4831-46bf-ae97-cea8a0a0130b-lib-modules\") pod \"perf-node-gather-daemonset-7hz9c\" (UID: \"a2850ee5-4831-46bf-ae97-cea8a0a0130b\") " pod="openshift-must-gather-2868l/perf-node-gather-daemonset-7hz9c" Jul 06 20:25:23.802581 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:23.802443 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a2850ee5-4831-46bf-ae97-cea8a0a0130b-sys\") pod \"perf-node-gather-daemonset-7hz9c\" (UID: \"a2850ee5-4831-46bf-ae97-cea8a0a0130b\") " pod="openshift-must-gather-2868l/perf-node-gather-daemonset-7hz9c" Jul 06 20:25:23.802581 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:23.802495 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vkp8\" (UniqueName: \"kubernetes.io/projected/a2850ee5-4831-46bf-ae97-cea8a0a0130b-kube-api-access-9vkp8\") pod \"perf-node-gather-daemonset-7hz9c\" (UID: \"a2850ee5-4831-46bf-ae97-cea8a0a0130b\") " pod="openshift-must-gather-2868l/perf-node-gather-daemonset-7hz9c" Jul 06 20:25:23.802581 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:23.802523 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a2850ee5-4831-46bf-ae97-cea8a0a0130b-podres\") pod \"perf-node-gather-daemonset-7hz9c\" (UID: \"a2850ee5-4831-46bf-ae97-cea8a0a0130b\") " pod="openshift-must-gather-2868l/perf-node-gather-daemonset-7hz9c" Jul 06 20:25:23.802581 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:23.802554 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a2850ee5-4831-46bf-ae97-cea8a0a0130b-lib-modules\") pod \"perf-node-gather-daemonset-7hz9c\" (UID: \"a2850ee5-4831-46bf-ae97-cea8a0a0130b\") " pod="openshift-must-gather-2868l/perf-node-gather-daemonset-7hz9c" Jul 06 20:25:23.802581 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:23.802564 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a2850ee5-4831-46bf-ae97-cea8a0a0130b-proc\") pod \"perf-node-gather-daemonset-7hz9c\" (UID: \"a2850ee5-4831-46bf-ae97-cea8a0a0130b\") " pod="openshift-must-gather-2868l/perf-node-gather-daemonset-7hz9c" Jul 06 20:25:23.802805 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:23.802577 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a2850ee5-4831-46bf-ae97-cea8a0a0130b-sys\") pod \"perf-node-gather-daemonset-7hz9c\" (UID: \"a2850ee5-4831-46bf-ae97-cea8a0a0130b\") " pod="openshift-must-gather-2868l/perf-node-gather-daemonset-7hz9c" Jul 06 20:25:23.802805 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:23.802666 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a2850ee5-4831-46bf-ae97-cea8a0a0130b-proc\") pod \"perf-node-gather-daemonset-7hz9c\" (UID: \"a2850ee5-4831-46bf-ae97-cea8a0a0130b\") " pod="openshift-must-gather-2868l/perf-node-gather-daemonset-7hz9c" Jul 06 20:25:23.802805 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:23.802678 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a2850ee5-4831-46bf-ae97-cea8a0a0130b-podres\") pod \"perf-node-gather-daemonset-7hz9c\" (UID: \"a2850ee5-4831-46bf-ae97-cea8a0a0130b\") " pod="openshift-must-gather-2868l/perf-node-gather-daemonset-7hz9c" Jul 06 20:25:23.811081 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:23.811054 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vkp8\" (UniqueName: \"kubernetes.io/projected/a2850ee5-4831-46bf-ae97-cea8a0a0130b-kube-api-access-9vkp8\") pod \"perf-node-gather-daemonset-7hz9c\" (UID: \"a2850ee5-4831-46bf-ae97-cea8a0a0130b\") " pod="openshift-must-gather-2868l/perf-node-gather-daemonset-7hz9c" Jul 06 20:25:23.907271 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:23.907163 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2868l/perf-node-gather-daemonset-7hz9c" Jul 06 20:25:24.263208 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:24.263183 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2868l/perf-node-gather-daemonset-7hz9c"] Jul 06 20:25:24.264276 ip-10-0-137-178 kubenswrapper[2551]: W0706 20:25:24.264243 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda2850ee5_4831_46bf_ae97_cea8a0a0130b.slice/crio-6adf5e5e56eee0c45a31bed99414b6fa6b10114166d72b777269cfbede04d926 WatchSource:0}: Error finding container 6adf5e5e56eee0c45a31bed99414b6fa6b10114166d72b777269cfbede04d926: Status 404 returned error can't find the container with id 6adf5e5e56eee0c45a31bed99414b6fa6b10114166d72b777269cfbede04d926 Jul 06 20:25:25.072370 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:25.072333 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2868l/perf-node-gather-daemonset-7hz9c" event={"ID":"a2850ee5-4831-46bf-ae97-cea8a0a0130b","Type":"ContainerStarted","Data":"d00fcdc53264a0290f514f86be7ff3301bf547999bb2b49bea9028b1560f93da"} Jul 06 20:25:25.072860 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:25.072395 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2868l/perf-node-gather-daemonset-7hz9c" event={"ID":"a2850ee5-4831-46bf-ae97-cea8a0a0130b","Type":"ContainerStarted","Data":"6adf5e5e56eee0c45a31bed99414b6fa6b10114166d72b777269cfbede04d926"} Jul 06 20:25:25.072860 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:25.072425 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-2868l/perf-node-gather-daemonset-7hz9c" Jul 06 20:25:25.091636 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:25.091583 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-2868l/perf-node-gather-daemonset-7hz9c" podStartSLOduration=2.091566367 podStartE2EDuration="2.091566367s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:25:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:25:25.090582482 +0000 UTC m=+1035.505782289" watchObservedRunningTime="2026-07-06 20:25:25.091566367 +0000 UTC m=+1035.506766172" Jul 06 20:25:25.243396 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:25.243360 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-85d867cd45-829fc_febd04fa-ce6e-48ce-9d9e-461fc9885088/console/0.log" Jul 06 20:25:25.915550 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:25.915522 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-847qg_8c2d1989-b3d7-48a4-8417-23e639cbc855/volume-data-source-validator/0.log" Jul 06 20:25:27.048022 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:27.043965 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qwn7v_148d8232-818e-46d9-b74b-94f6d03bf080/dns/0.log" Jul 06 20:25:27.068473 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:27.068444 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qwn7v_148d8232-818e-46d9-b74b-94f6d03bf080/kube-rbac-proxy/0.log" Jul 06 20:25:27.241289 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:27.241256 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-477lt_854b635c-f1e7-47d5-bf88-b5a80b572b1b/dns-node-resolver/0.log" Jul 06 20:25:27.897289 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:27.897257 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-94665fb6-4xkqt_5d20463d-fd91-4376-90aa-b38ed6f9b1c0/registry/0.log" Jul 06 20:25:27.968716 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:27.968688 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mv9hq_9cd8d42e-db03-477c-8ef9-bfed943eadb3/node-ca/0.log" Jul 06 20:25:28.992269 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:28.992238 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678xwrmt_6dd6cbd6-1df4-4233-bca9-6e44587c8759/istio-proxy/0.log" Jul 06 20:25:29.069535 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:29.069507 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-68b56f95f5-6krcp_fad5f926-d0b6-43b0-b124-92a9d7478625/kube-auth-proxy/0.log" Jul 06 20:25:29.151489 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:29.151455 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-qj8pt_f45bd611-bbaa-4f58-a40a-528378e433bb/istio-proxy/0.log" Jul 06 20:25:29.224066 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:29.224035 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-qj8pt_f45bd611-bbaa-4f58-a40a-528378e433bb/istio-proxy/1.log" Jul 06 20:25:29.250028 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:29.249949 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59f49fd4f6-9dznl_a1079f02-d8a4-42e9-8243-d6190e5debe8/payload-pre-processing/0.log" Jul 06 20:25:29.317604 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:29.317568 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bbcb654f4-bb8vm_e1f04227-efd2-4114-8b18-1fccfd54dd1b/payload-processing/0.log" Jul 06 20:25:29.343106 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:29.343081 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-668d9f84f5-rrtln_735538e5-306c-44a6-a934-30f463187f08/router/0.log" Jul 06 20:25:30.066333 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:30.066301 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-z67pq_cf4f10b3-8b8d-45c2-9f64-8a37aa591b68/serve-healthcheck-canary/0.log" Jul 06 20:25:30.647113 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:30.647061 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-jxbvp_e54c25e9-3542-48e1-9c6c-de6bd9b3f3d7/insights-operator/0.log" Jul 06 20:25:30.648420 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:30.648402 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-jxbvp_e54c25e9-3542-48e1-9c6c-de6bd9b3f3d7/insights-operator/1.log" Jul 06 20:25:30.828275 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:30.821819 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zd4zj_f60316f2-a0cc-4ac1-9c8b-eaccb5977acd/kube-rbac-proxy/0.log" Jul 06 20:25:30.846236 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:30.846209 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zd4zj_f60316f2-a0cc-4ac1-9c8b-eaccb5977acd/exporter/0.log" Jul 06 20:25:30.867458 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:30.867431 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zd4zj_f60316f2-a0cc-4ac1-9c8b-eaccb5977acd/extractor/0.log" Jul 06 20:25:31.089676 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:31.089651 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-2868l/perf-node-gather-daemonset-7hz9c" Jul 06 20:25:33.436089 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:33.436007 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-7qd2m_c0ee877c-70e6-4960-9924-c3aeb9f21455/manager/0.log" Jul 06 20:25:33.534692 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:33.534661 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-6nlkr_a6ef5c09-60d4-44fd-b968-f0bd1770147d/manager/0.log" Jul 06 20:25:33.590066 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:33.590036 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-6ffddcb95f-tkxlg_36efbc10-cac3-4a19-a97f-49de85ea3066/maas-api/1.log" Jul 06 20:25:33.609707 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:33.609679 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-6ffddcb95f-tkxlg_36efbc10-cac3-4a19-a97f-49de85ea3066/maas-api/0.log" Jul 06 20:25:33.686783 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:33.686693 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-66d86b4f9b-vtsjr_90746b84-4f6e-4cd5-b9e6-a6f10a326832/manager/0.log" Jul 06 20:25:33.751791 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:33.751764 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-gp85h_3fe7b2ec-32e7-4b5e-bfad-7f0f1ff752c2/server/0.log" Jul 06 20:25:33.836092 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:33.836062 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-r5svr_e7240fb9-24ab-412a-b7dc-d12f95a61060/manager/0.log" Jul 06 20:25:34.064841 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:34.064807 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678xwrmt_6dd6cbd6-1df4-4233-bca9-6e44587c8759/istio-proxy/0.log" Jul 06 20:25:34.140901 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:34.140858 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-68b56f95f5-6krcp_fad5f926-d0b6-43b0-b124-92a9d7478625/kube-auth-proxy/0.log" Jul 06 20:25:34.169170 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:34.169142 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7f6455564d-rwbgc_944d69e8-62db-410d-b9e6-a9e476aef550/manager/0.log" Jul 06 20:25:34.186012 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:34.185984 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-qj8pt_f45bd611-bbaa-4f58-a40a-528378e433bb/istio-proxy/0.log" Jul 06 20:25:34.200818 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:34.200787 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7f6455564d-rwbgc_944d69e8-62db-410d-b9e6-a9e476aef550/copy-manifests/0.log" Jul 06 20:25:34.263097 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:34.263063 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-qj8pt_f45bd611-bbaa-4f58-a40a-528378e433bb/istio-proxy/1.log" Jul 06 20:25:34.282260 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:34.282235 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59f49fd4f6-9dznl_a1079f02-d8a4-42e9-8243-d6190e5debe8/payload-pre-processing/0.log" Jul 06 20:25:34.351685 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:34.351581 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bbcb654f4-bb8vm_e1f04227-efd2-4114-8b18-1fccfd54dd1b/payload-processing/0.log" Jul 06 20:25:34.374952 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:34.374927 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-668d9f84f5-rrtln_735538e5-306c-44a6-a934-30f463187f08/router/0.log" Jul 06 20:25:35.608817 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:35.608789 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678xwrmt_6dd6cbd6-1df4-4233-bca9-6e44587c8759/istio-proxy/0.log" Jul 06 20:25:35.661463 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:35.661440 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-68b56f95f5-6krcp_fad5f926-d0b6-43b0-b124-92a9d7478625/kube-auth-proxy/0.log" Jul 06 20:25:35.708536 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:35.708491 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-qj8pt_f45bd611-bbaa-4f58-a40a-528378e433bb/istio-proxy/0.log" Jul 06 20:25:35.788947 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:35.788885 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-qj8pt_f45bd611-bbaa-4f58-a40a-528378e433bb/istio-proxy/1.log" Jul 06 20:25:35.801343 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:35.801316 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59f49fd4f6-9dznl_a1079f02-d8a4-42e9-8243-d6190e5debe8/payload-pre-processing/0.log" Jul 06 20:25:35.860779 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:35.860692 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bbcb654f4-bb8vm_e1f04227-efd2-4114-8b18-1fccfd54dd1b/payload-processing/0.log" Jul 06 20:25:35.875447 ip-10-0-137-178 kubenswrapper[2551]: I0706 20:25:35.875424 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-668d9f84f5-rrtln_735538e5-306c-44a6-a934-30f463187f08/router/0.log"