Jul 13 07:00:17.058142 ip-10-0-141-37 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 13 07:00:17.058316 ip-10-0-141-37 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 13 07:00:17.058418 ip-10-0-141-37 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 13 07:00:17.058934 ip-10-0-141-37 systemd[1]: Failed to start Kubernetes Kubelet. Jul 13 07:00:27.182322 ip-10-0-141-37 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 13 07:00:27.182338 ip-10-0-141-37 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 4834f445884e4eb9b4e364c6a1c56f2e -- Jul 13 07:02:03.566207 ip-10-0-141-37 systemd[1]: Starting Kubernetes Kubelet... Jul 13 07:02:04.144596 ip-10-0-141-37 kubenswrapper[2586]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 13 07:02:04.144596 ip-10-0-141-37 kubenswrapper[2586]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 13 07:02:04.144596 ip-10-0-141-37 kubenswrapper[2586]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 13 07:02:04.144596 ip-10-0-141-37 kubenswrapper[2586]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 13 07:02:04.144596 ip-10-0-141-37 kubenswrapper[2586]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 13 07:02:04.144596 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.053022 2586 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 13 07:02:04.144596 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057616 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 07:02:04.144596 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057633 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 07:02:04.144596 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057638 2586 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 07:02:04.144596 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057641 2586 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 07:02:04.144596 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057644 2586 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 07:02:04.144596 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057647 2586 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 07:02:04.144596 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057650 2586 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 07:02:04.144596 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057654 2586 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 07:02:04.146484 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057658 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 07:02:04.146484 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057661 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 07:02:04.146484 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057665 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 07:02:04.146484 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057668 2586 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 07:02:04.146484 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057671 2586 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 07:02:04.146484 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057674 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 07:02:04.146484 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057677 2586 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 07:02:04.146484 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057680 2586 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 07:02:04.146484 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057683 2586 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 07:02:04.146484 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057686 2586 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 07:02:04.146484 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057692 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 07:02:04.146484 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057695 2586 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 07:02:04.146484 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057698 2586 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 07:02:04.146484 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057701 2586 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 07:02:04.146484 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057704 2586 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 07:02:04.146484 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057708 2586 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 07:02:04.146484 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057712 2586 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 07:02:04.146484 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057715 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 07:02:04.146484 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057718 2586 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 07:02:04.146484 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057722 2586 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 07:02:04.138707 ip-10-0-141-37 systemd[1]: Started Kubernetes Kubelet. Jul 13 07:02:04.147335 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057725 2586 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 07:02:04.147335 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057727 2586 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 07:02:04.147335 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057730 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 07:02:04.147335 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057733 2586 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 07:02:04.147335 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057735 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 07:02:04.147335 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057738 2586 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 07:02:04.147335 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057741 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 07:02:04.147335 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057743 2586 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 07:02:04.147335 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057746 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 07:02:04.147335 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057750 2586 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 07:02:04.147335 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057753 2586 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 07:02:04.147335 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057756 2586 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 07:02:04.147335 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057759 2586 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 07:02:04.147335 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057762 2586 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 07:02:04.147335 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057765 2586 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 07:02:04.147335 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057767 2586 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 07:02:04.147335 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057770 2586 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 07:02:04.147335 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057772 2586 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 07:02:04.147335 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057775 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 07:02:04.147335 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057778 2586 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 07:02:04.148403 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057782 2586 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 07:02:04.148403 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057785 2586 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 07:02:04.148403 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057788 2586 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 07:02:04.148403 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057790 2586 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 07:02:04.148403 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057793 2586 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 07:02:04.148403 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057796 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 07:02:04.148403 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057799 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 07:02:04.148403 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057801 2586 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 07:02:04.148403 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057804 2586 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 07:02:04.148403 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057806 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 07:02:04.148403 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057808 2586 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 07:02:04.148403 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057811 2586 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 07:02:04.148403 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057814 2586 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 07:02:04.148403 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057817 2586 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 07:02:04.148403 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057820 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 07:02:04.148403 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057824 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 07:02:04.148403 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057828 2586 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 07:02:04.148403 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057831 2586 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 07:02:04.148403 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057836 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 07:02:04.149281 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057840 2586 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 07:02:04.149281 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057855 2586 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 07:02:04.149281 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057860 2586 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 07:02:04.149281 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057863 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 07:02:04.149281 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057867 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 07:02:04.149281 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057870 2586 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 07:02:04.149281 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057873 2586 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 07:02:04.149281 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057876 2586 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 07:02:04.149281 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057878 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 07:02:04.149281 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057881 2586 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 07:02:04.149281 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057883 2586 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 07:02:04.149281 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057887 2586 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 07:02:04.149281 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057891 2586 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 07:02:04.149281 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057894 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 07:02:04.149281 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057897 2586 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 07:02:04.149281 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057899 2586 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 07:02:04.149281 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057902 2586 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 07:02:04.149281 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057904 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 07:02:04.149281 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057907 2586 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 07:02:04.149281 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057910 2586 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 07:02:04.150353 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057912 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 07:02:04.150353 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057915 2586 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 07:02:04.150353 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057917 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 07:02:04.150353 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057920 2586 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 07:02:04.150353 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057922 2586 feature_gate.go:328] unrecognized feature gate: Example Jul 13 07:02:04.150353 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057925 2586 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 07:02:04.150353 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057927 2586 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 07:02:04.150353 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057930 2586 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 07:02:04.150353 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057932 2586 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 07:02:04.150353 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057935 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 07:02:04.150353 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057939 2586 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 07:02:04.150353 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.057942 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 07:02:04.150353 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058709 2586 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 07:02:04.150353 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058717 2586 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 07:02:04.150353 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058721 2586 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 07:02:04.150353 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058724 2586 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 07:02:04.150353 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058727 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 07:02:04.150353 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058730 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 07:02:04.150353 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058733 2586 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 07:02:04.151477 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058736 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 07:02:04.151477 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058739 2586 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 07:02:04.151477 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058741 2586 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 07:02:04.151477 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058744 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 07:02:04.151477 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058746 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 07:02:04.151477 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058749 2586 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 07:02:04.151477 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058752 2586 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 07:02:04.151477 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058755 2586 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 07:02:04.151477 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058757 2586 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 07:02:04.151477 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058760 2586 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 07:02:04.151477 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058762 2586 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 07:02:04.151477 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058765 2586 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 07:02:04.151477 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058768 2586 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 07:02:04.151477 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058770 2586 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 07:02:04.151477 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058773 2586 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 07:02:04.151477 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058775 2586 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 07:02:04.151477 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058778 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 07:02:04.151477 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058781 2586 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 07:02:04.151477 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058783 2586 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 07:02:04.151477 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058786 2586 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 07:02:04.152494 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058788 2586 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 07:02:04.152494 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058791 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 07:02:04.152494 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058793 2586 feature_gate.go:328] unrecognized feature gate: Example Jul 13 07:02:04.152494 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058796 2586 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 07:02:04.152494 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058800 2586 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 07:02:04.152494 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058802 2586 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 07:02:04.152494 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058806 2586 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 07:02:04.152494 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058808 2586 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 07:02:04.152494 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058812 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 07:02:04.152494 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058816 2586 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 07:02:04.152494 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058818 2586 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 07:02:04.152494 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058821 2586 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 07:02:04.152494 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058825 2586 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 07:02:04.152494 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058829 2586 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 07:02:04.152494 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058831 2586 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 07:02:04.152494 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058834 2586 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 07:02:04.152494 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058837 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 07:02:04.152494 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058840 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 07:02:04.152494 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058861 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 07:02:04.153343 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058865 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 07:02:04.153343 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058868 2586 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 07:02:04.153343 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058870 2586 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 07:02:04.153343 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058873 2586 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 07:02:04.153343 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058876 2586 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 07:02:04.153343 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058878 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 07:02:04.153343 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058881 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 07:02:04.153343 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058885 2586 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 07:02:04.153343 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058888 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 07:02:04.153343 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058891 2586 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 07:02:04.153343 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058894 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 07:02:04.153343 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058897 2586 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 07:02:04.153343 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058900 2586 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 07:02:04.153343 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058902 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 07:02:04.153343 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058905 2586 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 07:02:04.153343 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058908 2586 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 07:02:04.153343 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058910 2586 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 07:02:04.153343 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058917 2586 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 07:02:04.153343 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058919 2586 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 07:02:04.153343 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058922 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 07:02:04.154263 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058924 2586 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 07:02:04.154263 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058927 2586 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 07:02:04.154263 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058930 2586 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 07:02:04.154263 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058932 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 07:02:04.154263 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058936 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 07:02:04.154263 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058938 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 07:02:04.154263 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058941 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 07:02:04.154263 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058943 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 07:02:04.154263 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058946 2586 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 07:02:04.154263 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058948 2586 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 07:02:04.154263 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058951 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 07:02:04.154263 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058954 2586 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 07:02:04.154263 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058956 2586 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 07:02:04.154263 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058960 2586 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 07:02:04.154263 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058962 2586 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 07:02:04.154263 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058964 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 07:02:04.154263 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058967 2586 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 07:02:04.154263 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058970 2586 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 07:02:04.154263 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058972 2586 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 07:02:04.154263 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058975 2586 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 07:02:04.155335 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058977 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 07:02:04.155335 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058980 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 07:02:04.155335 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058982 2586 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 07:02:04.155335 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058984 2586 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 07:02:04.155335 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058987 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 07:02:04.155335 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058990 2586 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 07:02:04.155335 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058992 2586 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 07:02:04.155335 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058995 2586 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 07:02:04.155335 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.058997 2586 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 07:02:04.155335 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059001 2586 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 07:02:04.155335 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059004 2586 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 07:02:04.155335 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059006 2586 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 07:02:04.155335 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059010 2586 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 07:02:04.155335 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059452 2586 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 07:02:04.155335 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059457 2586 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 07:02:04.155335 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059460 2586 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 07:02:04.155335 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059464 2586 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 07:02:04.155335 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059467 2586 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 07:02:04.155335 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059470 2586 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 07:02:04.156236 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059472 2586 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 07:02:04.156236 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059476 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 07:02:04.156236 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059478 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 07:02:04.156236 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059481 2586 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 07:02:04.156236 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059483 2586 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 07:02:04.156236 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059486 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 07:02:04.156236 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059489 2586 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 07:02:04.156236 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059492 2586 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 07:02:04.156236 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059495 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 07:02:04.156236 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059498 2586 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 07:02:04.156236 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059500 2586 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 07:02:04.156236 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059503 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 07:02:04.156236 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059506 2586 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 07:02:04.156236 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059508 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 07:02:04.156236 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059511 2586 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 07:02:04.156236 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059514 2586 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 07:02:04.156236 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059517 2586 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 07:02:04.156236 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059519 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 07:02:04.156236 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059522 2586 feature_gate.go:328] unrecognized feature gate: Example Jul 13 07:02:04.156236 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059525 2586 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 07:02:04.156236 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059527 2586 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 07:02:04.156932 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059530 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 07:02:04.156932 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059534 2586 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 07:02:04.156932 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059536 2586 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 07:02:04.156932 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059539 2586 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 07:02:04.156932 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059542 2586 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 07:02:04.156932 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059544 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 07:02:04.156932 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059547 2586 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 07:02:04.156932 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059549 2586 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 07:02:04.156932 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059552 2586 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 07:02:04.156932 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059554 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 07:02:04.156932 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059557 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 07:02:04.156932 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059559 2586 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 07:02:04.156932 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059561 2586 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 07:02:04.156932 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059564 2586 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 07:02:04.156932 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059567 2586 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 07:02:04.156932 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059569 2586 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 07:02:04.156932 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059572 2586 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 07:02:04.156932 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059574 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 07:02:04.156932 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059577 2586 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 07:02:04.156932 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059580 2586 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 07:02:04.157834 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059583 2586 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 07:02:04.157834 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059585 2586 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 07:02:04.157834 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059589 2586 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 07:02:04.157834 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059593 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 07:02:04.157834 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059596 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 07:02:04.157834 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059598 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 07:02:04.157834 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059601 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 07:02:04.157834 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059603 2586 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 07:02:04.157834 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059605 2586 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 07:02:04.157834 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059608 2586 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 07:02:04.157834 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059610 2586 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 07:02:04.157834 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059613 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 07:02:04.157834 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059615 2586 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 07:02:04.157834 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059619 2586 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 07:02:04.157834 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059622 2586 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 07:02:04.157834 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059625 2586 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 07:02:04.157834 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059628 2586 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 07:02:04.157834 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059630 2586 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 07:02:04.157834 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059633 2586 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 07:02:04.158821 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059635 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 07:02:04.158821 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059638 2586 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 07:02:04.158821 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059641 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 07:02:04.158821 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059643 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 07:02:04.158821 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059648 2586 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 07:02:04.158821 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059650 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 07:02:04.158821 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059654 2586 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 07:02:04.158821 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059657 2586 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 07:02:04.158821 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059660 2586 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 07:02:04.158821 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059663 2586 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 07:02:04.158821 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059666 2586 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 07:02:04.158821 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059669 2586 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 07:02:04.158821 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059672 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 07:02:04.158821 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059674 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 07:02:04.158821 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059677 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 07:02:04.158821 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059679 2586 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 07:02:04.158821 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059681 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 07:02:04.158821 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059684 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 07:02:04.158821 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059686 2586 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 07:02:04.158821 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059689 2586 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 07:02:04.159701 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059692 2586 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 07:02:04.159701 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059695 2586 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 07:02:04.159701 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059697 2586 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 07:02:04.159701 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059700 2586 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 07:02:04.159701 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059702 2586 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 07:02:04.159701 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059705 2586 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 07:02:04.159701 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059709 2586 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 07:02:04.159701 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059712 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 07:02:04.159701 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059714 2586 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 07:02:04.159701 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059717 2586 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 07:02:04.159701 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059720 2586 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 07:02:04.159701 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059723 2586 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 07:02:04.159701 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.059727 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 07:02:04.159701 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.059799 2586 flags.go:64] FLAG: --address="0.0.0.0" Jul 13 07:02:04.159701 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.059813 2586 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 13 07:02:04.159701 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.059820 2586 flags.go:64] FLAG: --anonymous-auth="true" Jul 13 07:02:04.159701 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.059826 2586 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 13 07:02:04.159701 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.059830 2586 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 13 07:02:04.159701 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.059834 2586 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 13 07:02:04.159701 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.059838 2586 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 13 07:02:04.161154 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.059855 2586 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 13 07:02:04.161154 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.059858 2586 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 13 07:02:04.161154 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.059861 2586 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 13 07:02:04.161154 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.059865 2586 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 13 07:02:04.161154 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.059869 2586 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 13 07:02:04.161154 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.059874 2586 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 13 07:02:04.161154 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.059877 2586 flags.go:64] FLAG: --cgroup-root="" Jul 13 07:02:04.161154 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.059880 2586 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 13 07:02:04.161154 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.059883 2586 flags.go:64] FLAG: --client-ca-file="" Jul 13 07:02:04.161154 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.059886 2586 flags.go:64] FLAG: --cloud-provider="external" Jul 13 07:02:04.161154 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.059889 2586 flags.go:64] FLAG: --cluster-dns="[]" Jul 13 07:02:04.161154 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.059894 2586 flags.go:64] FLAG: --cluster-domain="" Jul 13 07:02:04.161154 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.059897 2586 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 13 07:02:04.161154 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.059900 2586 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 13 07:02:04.161154 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.059903 2586 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 13 07:02:04.161154 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.059906 2586 flags.go:64] FLAG: --container-log-max-files="5" Jul 13 07:02:04.161154 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.059910 2586 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 13 07:02:04.161154 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.059913 2586 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 13 07:02:04.161154 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.059917 2586 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 13 07:02:04.161154 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.059921 2586 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 13 07:02:04.161154 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.059924 2586 flags.go:64] FLAG: --contention-profiling="false" Jul 13 07:02:04.161154 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.059927 2586 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 13 07:02:04.161154 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.059930 2586 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 13 07:02:04.161154 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.059934 2586 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 13 07:02:04.162246 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.059937 2586 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 13 07:02:04.162246 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.059941 2586 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 13 07:02:04.162246 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.059944 2586 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 13 07:02:04.162246 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.059947 2586 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 13 07:02:04.162246 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.059951 2586 flags.go:64] FLAG: --enable-load-reader="false" Jul 13 07:02:04.162246 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.059954 2586 flags.go:64] FLAG: --enable-server="true" Jul 13 07:02:04.162246 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.059957 2586 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 13 07:02:04.162246 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.059961 2586 flags.go:64] FLAG: --event-burst="100" Jul 13 07:02:04.162246 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.059964 2586 flags.go:64] FLAG: --event-qps="50" Jul 13 07:02:04.162246 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.059967 2586 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 13 07:02:04.162246 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.059970 2586 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 13 07:02:04.162246 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.059973 2586 flags.go:64] FLAG: --eviction-hard="" Jul 13 07:02:04.162246 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.059977 2586 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 13 07:02:04.162246 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.059980 2586 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 13 07:02:04.162246 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.059983 2586 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 13 07:02:04.162246 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.059987 2586 flags.go:64] FLAG: --eviction-soft="" Jul 13 07:02:04.162246 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.059990 2586 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 13 07:02:04.162246 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.059992 2586 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 13 07:02:04.162246 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.059995 2586 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 13 07:02:04.162246 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.059998 2586 flags.go:64] FLAG: --experimental-mounter-path="" Jul 13 07:02:04.162246 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060001 2586 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 13 07:02:04.162246 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060004 2586 flags.go:64] FLAG: --fail-swap-on="true" Jul 13 07:02:04.162246 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060007 2586 flags.go:64] FLAG: --feature-gates="" Jul 13 07:02:04.162246 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060011 2586 flags.go:64] FLAG: --file-check-frequency="20s" Jul 13 07:02:04.162246 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060014 2586 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 13 07:02:04.164381 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060017 2586 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 13 07:02:04.164381 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060022 2586 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 13 07:02:04.164381 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060025 2586 flags.go:64] FLAG: --healthz-port="10248" Jul 13 07:02:04.164381 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060028 2586 flags.go:64] FLAG: --help="false" Jul 13 07:02:04.164381 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060031 2586 flags.go:64] FLAG: --hostname-override="ip-10-0-141-37.ec2.internal" Jul 13 07:02:04.164381 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060034 2586 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 13 07:02:04.164381 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060037 2586 flags.go:64] FLAG: --http-check-frequency="20s" Jul 13 07:02:04.164381 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060040 2586 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 13 07:02:04.164381 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060043 2586 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 13 07:02:04.164381 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060047 2586 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 13 07:02:04.164381 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060049 2586 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 13 07:02:04.164381 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060054 2586 flags.go:64] FLAG: --image-service-endpoint="" Jul 13 07:02:04.164381 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060056 2586 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 13 07:02:04.164381 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060059 2586 flags.go:64] FLAG: --kube-api-burst="100" Jul 13 07:02:04.164381 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060062 2586 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 13 07:02:04.164381 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060065 2586 flags.go:64] FLAG: --kube-api-qps="50" Jul 13 07:02:04.164381 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060068 2586 flags.go:64] FLAG: --kube-reserved="" Jul 13 07:02:04.164381 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060071 2586 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 13 07:02:04.164381 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060074 2586 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 13 07:02:04.164381 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060077 2586 flags.go:64] FLAG: --kubelet-cgroups="" Jul 13 07:02:04.164381 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060079 2586 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 13 07:02:04.164381 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060082 2586 flags.go:64] FLAG: --lock-file="" Jul 13 07:02:04.164381 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060086 2586 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 13 07:02:04.164381 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060089 2586 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 13 07:02:04.166008 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060092 2586 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 13 07:02:04.166008 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060098 2586 flags.go:64] FLAG: --log-json-split-stream="false" Jul 13 07:02:04.166008 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060100 2586 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 13 07:02:04.166008 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060103 2586 flags.go:64] FLAG: --log-text-split-stream="false" Jul 13 07:02:04.166008 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060106 2586 flags.go:64] FLAG: --logging-format="text" Jul 13 07:02:04.166008 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060109 2586 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 13 07:02:04.166008 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060112 2586 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 13 07:02:04.166008 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060115 2586 flags.go:64] FLAG: --manifest-url="" Jul 13 07:02:04.166008 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060118 2586 flags.go:64] FLAG: --manifest-url-header="" Jul 13 07:02:04.166008 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060123 2586 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 13 07:02:04.166008 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060126 2586 flags.go:64] FLAG: --max-open-files="1000000" Jul 13 07:02:04.166008 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060130 2586 flags.go:64] FLAG: --max-pods="110" Jul 13 07:02:04.166008 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060133 2586 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 13 07:02:04.166008 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060136 2586 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 13 07:02:04.166008 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060139 2586 flags.go:64] FLAG: --memory-manager-policy="None" Jul 13 07:02:04.166008 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060142 2586 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 13 07:02:04.166008 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060145 2586 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 13 07:02:04.166008 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060156 2586 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 13 07:02:04.166008 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060160 2586 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 13 07:02:04.166008 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060170 2586 flags.go:64] FLAG: --node-status-max-images="50" Jul 13 07:02:04.166008 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060173 2586 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 13 07:02:04.166008 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060176 2586 flags.go:64] FLAG: --oom-score-adj="-999" Jul 13 07:02:04.166008 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060179 2586 flags.go:64] FLAG: --pod-cidr="" Jul 13 07:02:04.169686 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060182 2586 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 13 07:02:04.169686 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060187 2586 flags.go:64] FLAG: --pod-manifest-path="" Jul 13 07:02:04.169686 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060190 2586 flags.go:64] FLAG: --pod-max-pids="-1" Jul 13 07:02:04.169686 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060193 2586 flags.go:64] FLAG: --pods-per-core="0" Jul 13 07:02:04.169686 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060196 2586 flags.go:64] FLAG: --port="10250" Jul 13 07:02:04.169686 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060199 2586 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 13 07:02:04.169686 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060202 2586 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0c28fa189ea41c1e6" Jul 13 07:02:04.169686 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060205 2586 flags.go:64] FLAG: --qos-reserved="" Jul 13 07:02:04.169686 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060209 2586 flags.go:64] FLAG: --read-only-port="10255" Jul 13 07:02:04.169686 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060212 2586 flags.go:64] FLAG: --register-node="true" Jul 13 07:02:04.169686 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060215 2586 flags.go:64] FLAG: --register-with-taints="" Jul 13 07:02:04.169686 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060218 2586 flags.go:64] FLAG: --registry-burst="10" Jul 13 07:02:04.169686 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060221 2586 flags.go:64] FLAG: --registry-qps="5" Jul 13 07:02:04.169686 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060225 2586 flags.go:64] FLAG: --reserved-cpus="" Jul 13 07:02:04.169686 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060227 2586 flags.go:64] FLAG: --reserved-memory="" Jul 13 07:02:04.169686 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060231 2586 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 13 07:02:04.169686 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060234 2586 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 13 07:02:04.169686 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060237 2586 flags.go:64] FLAG: --rotate-certificates="false" Jul 13 07:02:04.169686 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060241 2586 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 13 07:02:04.169686 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060245 2586 flags.go:64] FLAG: --runonce="false" Jul 13 07:02:04.169686 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060248 2586 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 13 07:02:04.169686 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060251 2586 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 13 07:02:04.169686 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060254 2586 flags.go:64] FLAG: --seccomp-default="false" Jul 13 07:02:04.169686 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060257 2586 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 13 07:02:04.169686 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060260 2586 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 13 07:02:04.171943 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060263 2586 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 13 07:02:04.171943 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060266 2586 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 13 07:02:04.171943 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060269 2586 flags.go:64] FLAG: --storage-driver-password="root" Jul 13 07:02:04.171943 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060273 2586 flags.go:64] FLAG: --storage-driver-secure="false" Jul 13 07:02:04.171943 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060276 2586 flags.go:64] FLAG: --storage-driver-table="stats" Jul 13 07:02:04.171943 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060279 2586 flags.go:64] FLAG: --storage-driver-user="root" Jul 13 07:02:04.171943 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060282 2586 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 13 07:02:04.171943 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060285 2586 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 13 07:02:04.171943 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060288 2586 flags.go:64] FLAG: --system-cgroups="" Jul 13 07:02:04.171943 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060291 2586 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 13 07:02:04.171943 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060297 2586 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 13 07:02:04.171943 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060299 2586 flags.go:64] FLAG: --tls-cert-file="" Jul 13 07:02:04.171943 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060302 2586 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 13 07:02:04.171943 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060307 2586 flags.go:64] FLAG: --tls-min-version="" Jul 13 07:02:04.171943 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060310 2586 flags.go:64] FLAG: --tls-private-key-file="" Jul 13 07:02:04.171943 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060313 2586 flags.go:64] FLAG: --topology-manager-policy="none" Jul 13 07:02:04.171943 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060316 2586 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 13 07:02:04.171943 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060319 2586 flags.go:64] FLAG: --topology-manager-scope="container" Jul 13 07:02:04.171943 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060322 2586 flags.go:64] FLAG: --v="2" Jul 13 07:02:04.171943 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060326 2586 flags.go:64] FLAG: --version="false" Jul 13 07:02:04.171943 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060330 2586 flags.go:64] FLAG: --vmodule="" Jul 13 07:02:04.171943 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060334 2586 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 13 07:02:04.171943 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.060337 2586 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 13 07:02:04.171943 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060456 2586 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 07:02:04.171943 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060460 2586 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 07:02:04.175917 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060463 2586 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 07:02:04.175917 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060468 2586 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 07:02:04.175917 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060471 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 07:02:04.175917 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060476 2586 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 07:02:04.175917 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060479 2586 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 07:02:04.175917 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060482 2586 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 07:02:04.175917 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060485 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 07:02:04.175917 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060488 2586 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 07:02:04.175917 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060491 2586 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 07:02:04.175917 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060493 2586 feature_gate.go:328] unrecognized feature gate: Example Jul 13 07:02:04.175917 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060498 2586 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 07:02:04.175917 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060501 2586 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 07:02:04.175917 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060503 2586 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 07:02:04.175917 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060506 2586 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 07:02:04.175917 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060509 2586 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 07:02:04.175917 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060511 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 07:02:04.175917 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060514 2586 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 07:02:04.175917 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060516 2586 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 07:02:04.175917 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060519 2586 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 07:02:04.175917 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060521 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 07:02:04.176781 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060524 2586 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 07:02:04.176781 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060526 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 07:02:04.176781 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060529 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 07:02:04.176781 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060531 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 07:02:04.176781 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060534 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 07:02:04.176781 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060537 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 07:02:04.176781 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060539 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 07:02:04.176781 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060541 2586 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 07:02:04.176781 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060544 2586 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 07:02:04.176781 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060546 2586 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 07:02:04.176781 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060549 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 07:02:04.176781 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060552 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 07:02:04.176781 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060554 2586 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 07:02:04.176781 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060558 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 07:02:04.176781 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060560 2586 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 07:02:04.176781 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060563 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 07:02:04.176781 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060565 2586 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 07:02:04.176781 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060568 2586 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 07:02:04.176781 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060571 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 07:02:04.176781 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060574 2586 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 07:02:04.178648 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060577 2586 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 07:02:04.178648 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060579 2586 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 07:02:04.178648 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060584 2586 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 07:02:04.178648 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060587 2586 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 07:02:04.178648 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060590 2586 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 07:02:04.178648 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060592 2586 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 07:02:04.178648 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060594 2586 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 07:02:04.178648 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060597 2586 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 07:02:04.178648 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060600 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 07:02:04.178648 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060602 2586 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 07:02:04.178648 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060605 2586 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 07:02:04.178648 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060607 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 07:02:04.178648 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060610 2586 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 07:02:04.178648 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060612 2586 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 07:02:04.178648 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060615 2586 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 07:02:04.178648 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060617 2586 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 07:02:04.178648 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060620 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 07:02:04.178648 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060623 2586 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 07:02:04.178648 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060625 2586 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 07:02:04.180053 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060628 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 07:02:04.180053 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060630 2586 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 07:02:04.180053 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060633 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 07:02:04.180053 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060635 2586 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 07:02:04.180053 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060638 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 07:02:04.180053 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060640 2586 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 07:02:04.180053 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060644 2586 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 07:02:04.180053 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060646 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 07:02:04.180053 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060649 2586 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 07:02:04.180053 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060651 2586 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 07:02:04.180053 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060654 2586 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 07:02:04.180053 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060656 2586 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 07:02:04.180053 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060659 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 07:02:04.180053 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060662 2586 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 07:02:04.180053 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060664 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 07:02:04.180053 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060668 2586 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 07:02:04.180053 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060671 2586 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 07:02:04.180053 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060675 2586 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 07:02:04.180053 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060678 2586 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 07:02:04.180053 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060681 2586 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 07:02:04.181645 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060684 2586 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 07:02:04.181645 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060687 2586 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 07:02:04.181645 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060690 2586 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 07:02:04.181645 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060693 2586 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 07:02:04.181645 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060696 2586 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 07:02:04.181645 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060699 2586 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 07:02:04.181645 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060703 2586 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 07:02:04.181645 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060706 2586 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 07:02:04.181645 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060708 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 07:02:04.181645 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060711 2586 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 07:02:04.181645 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060713 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 07:02:04.181645 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060716 2586 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 07:02:04.181645 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060719 2586 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 07:02:04.181645 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060722 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 07:02:04.181645 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060724 2586 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 07:02:04.181645 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060727 2586 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 07:02:04.181645 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060729 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 07:02:04.181645 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.060732 2586 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 07:02:04.182400 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.061380 2586 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 13 07:02:04.182400 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.068649 2586 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 13 07:02:04.182400 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.068667 2586 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 13 07:02:04.182400 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.068686 2586 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 13 07:02:04.182400 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.068695 2586 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 13 07:02:04.182400 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068775 2586 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 07:02:04.182400 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068782 2586 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 07:02:04.182400 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068787 2586 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 07:02:04.182400 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068791 2586 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 07:02:04.182400 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068794 2586 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 07:02:04.182400 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068797 2586 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 07:02:04.182400 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068800 2586 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 07:02:04.182400 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068803 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 07:02:04.182400 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068806 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 07:02:04.183664 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068809 2586 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 07:02:04.183664 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068815 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 07:02:04.183664 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068818 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 07:02:04.183664 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068821 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 07:02:04.183664 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068824 2586 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 07:02:04.183664 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068826 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 07:02:04.183664 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068829 2586 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 07:02:04.183664 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068832 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 07:02:04.183664 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068834 2586 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 07:02:04.183664 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068837 2586 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 07:02:04.183664 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068840 2586 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 07:02:04.183664 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068864 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 07:02:04.183664 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068868 2586 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 07:02:04.183664 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068871 2586 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 07:02:04.183664 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068875 2586 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 07:02:04.183664 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068879 2586 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 07:02:04.183664 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068883 2586 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 07:02:04.183664 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068887 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 07:02:04.183664 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068891 2586 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 07:02:04.183664 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068894 2586 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 07:02:04.184488 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068897 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 07:02:04.184488 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068900 2586 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 07:02:04.184488 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068903 2586 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 07:02:04.184488 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068905 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 07:02:04.184488 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068908 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 07:02:04.184488 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068910 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 07:02:04.184488 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068913 2586 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 07:02:04.184488 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068916 2586 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 07:02:04.184488 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068918 2586 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 07:02:04.184488 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068921 2586 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 07:02:04.184488 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068925 2586 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 07:02:04.184488 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068929 2586 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 07:02:04.184488 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068932 2586 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 07:02:04.184488 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068936 2586 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 07:02:04.184488 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068939 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 07:02:04.184488 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068942 2586 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 07:02:04.184488 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068944 2586 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 07:02:04.184488 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068947 2586 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 07:02:04.184488 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068949 2586 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 07:02:04.185528 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068952 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 07:02:04.185528 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068954 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 07:02:04.185528 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068957 2586 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 07:02:04.185528 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068959 2586 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 07:02:04.185528 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068962 2586 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 07:02:04.185528 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068970 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 07:02:04.185528 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068974 2586 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 07:02:04.185528 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068976 2586 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 07:02:04.185528 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068979 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 07:02:04.185528 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068982 2586 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 07:02:04.185528 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068984 2586 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 07:02:04.185528 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068987 2586 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 07:02:04.185528 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068990 2586 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 07:02:04.185528 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068992 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 07:02:04.185528 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068996 2586 feature_gate.go:328] unrecognized feature gate: Example Jul 13 07:02:04.185528 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.068999 2586 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 07:02:04.185528 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069003 2586 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 07:02:04.185528 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069007 2586 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 07:02:04.185528 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069010 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 07:02:04.185528 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069012 2586 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 07:02:04.186359 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069015 2586 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 07:02:04.186359 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069018 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 07:02:04.186359 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069020 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 07:02:04.186359 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069022 2586 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 07:02:04.186359 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069025 2586 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 07:02:04.186359 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069027 2586 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 07:02:04.186359 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069031 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 07:02:04.186359 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069034 2586 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 07:02:04.186359 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069036 2586 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 07:02:04.186359 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069039 2586 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 07:02:04.186359 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069045 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 07:02:04.186359 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069047 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 07:02:04.186359 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069050 2586 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 07:02:04.186359 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069055 2586 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 07:02:04.186359 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069057 2586 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 07:02:04.186359 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069060 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 07:02:04.186359 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069062 2586 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 07:02:04.186359 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069065 2586 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 07:02:04.186359 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069073 2586 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 07:02:04.186359 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069076 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 07:02:04.187677 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069079 2586 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 07:02:04.187677 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069081 2586 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 07:02:04.187677 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069084 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 07:02:04.187677 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069086 2586 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 07:02:04.187677 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069089 2586 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 07:02:04.187677 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069091 2586 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 07:02:04.187677 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069094 2586 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 07:02:04.187677 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069096 2586 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 07:02:04.187677 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069099 2586 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 07:02:04.187677 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069102 2586 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 07:02:04.187677 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069104 2586 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 07:02:04.187677 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.069110 2586 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 13 07:02:04.187677 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069263 2586 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 07:02:04.187677 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069269 2586 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 07:02:04.187677 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069272 2586 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 07:02:04.188298 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069275 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 07:02:04.188298 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069277 2586 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 07:02:04.188298 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069280 2586 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 07:02:04.188298 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069284 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 07:02:04.188298 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069287 2586 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 07:02:04.188298 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069290 2586 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 07:02:04.188298 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069292 2586 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 07:02:04.188298 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069294 2586 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 07:02:04.188298 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069297 2586 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 07:02:04.188298 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069299 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 07:02:04.188298 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069302 2586 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 07:02:04.188298 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069307 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 07:02:04.188298 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069309 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 07:02:04.188298 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069312 2586 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 07:02:04.188298 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069314 2586 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 07:02:04.188298 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069317 2586 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 07:02:04.188298 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069325 2586 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 07:02:04.188298 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069328 2586 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 07:02:04.188298 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069331 2586 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 07:02:04.382022 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069333 2586 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 07:02:04.382022 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069336 2586 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 07:02:04.382022 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069338 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 07:02:04.382022 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069341 2586 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 07:02:04.382022 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069343 2586 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 07:02:04.382022 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069346 2586 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 07:02:04.382022 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069349 2586 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 07:02:04.382022 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069351 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 07:02:04.382022 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069354 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 07:02:04.382022 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069357 2586 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 07:02:04.382022 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069359 2586 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 07:02:04.382022 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069361 2586 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 07:02:04.382022 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069364 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 07:02:04.382022 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069373 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 07:02:04.382022 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069375 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 07:02:04.382022 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069378 2586 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 07:02:04.382022 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069381 2586 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 07:02:04.382022 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069384 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 07:02:04.382022 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069387 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 07:02:04.382022 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069389 2586 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 07:02:04.395500 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069391 2586 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 07:02:04.395500 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069394 2586 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 07:02:04.395500 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069396 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 07:02:04.395500 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069400 2586 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 07:02:04.395500 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069405 2586 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 07:02:04.395500 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069409 2586 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 07:02:04.395500 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069412 2586 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 07:02:04.395500 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069415 2586 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 07:02:04.395500 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069418 2586 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 07:02:04.395500 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069421 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 07:02:04.395500 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069429 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 07:02:04.395500 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069432 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 07:02:04.395500 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069434 2586 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 07:02:04.395500 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069438 2586 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 07:02:04.395500 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069440 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 07:02:04.395500 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069443 2586 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 07:02:04.395500 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069445 2586 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 07:02:04.395500 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069448 2586 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 07:02:04.395500 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069450 2586 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 07:02:04.396190 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069453 2586 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 07:02:04.396190 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069456 2586 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 07:02:04.396190 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069459 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 07:02:04.396190 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069462 2586 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 07:02:04.396190 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069465 2586 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 07:02:04.396190 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069468 2586 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 07:02:04.396190 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069470 2586 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 07:02:04.396190 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069473 2586 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 07:02:04.396190 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069476 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 07:02:04.396190 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069479 2586 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 07:02:04.396190 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069482 2586 feature_gate.go:328] unrecognized feature gate: Example Jul 13 07:02:04.396190 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069484 2586 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 07:02:04.396190 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069487 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 07:02:04.396190 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069489 2586 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 07:02:04.396190 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069492 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 07:02:04.396190 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069494 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 07:02:04.396190 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069497 2586 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 07:02:04.396190 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069501 2586 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 07:02:04.396190 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069504 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 07:02:04.396190 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069506 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 07:02:04.396876 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069509 2586 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 07:02:04.396876 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069511 2586 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 07:02:04.396876 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069513 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 07:02:04.396876 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069516 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 07:02:04.396876 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069525 2586 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 07:02:04.396876 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069527 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 07:02:04.396876 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069530 2586 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 07:02:04.396876 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069533 2586 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 07:02:04.396876 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069535 2586 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 07:02:04.396876 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069538 2586 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 07:02:04.396876 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069540 2586 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 07:02:04.396876 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069543 2586 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 07:02:04.396876 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069545 2586 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 07:02:04.396876 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069547 2586 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 07:02:04.396876 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069550 2586 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 07:02:04.396876 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069552 2586 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 07:02:04.396876 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069555 2586 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 07:02:04.396876 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:04.069557 2586 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 07:02:04.397383 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.069562 2586 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 13 07:02:04.397383 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.069696 2586 server.go:962] "Client rotation is on, will bootstrap in background" Jul 13 07:02:04.397383 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.073141 2586 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 13 07:02:04.397383 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.074701 2586 server.go:1019] "Starting client certificate rotation" Jul 13 07:02:04.397383 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.074817 2586 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 13 07:02:04.397383 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.074865 2586 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 13 07:02:04.397383 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.078001 2586 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 13 07:02:04.397383 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.082503 2586 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 13 07:02:04.397383 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.095272 2586 log.go:25] "Validated CRI v1 runtime API" Jul 13 07:02:04.397383 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.097791 2586 log.go:25] "Validated CRI v1 image API" Jul 13 07:02:04.397383 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.099449 2586 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 13 07:02:04.397383 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.103000 2586 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 7b2c98aa-e1cd-4b84-baa1-4fc01e2af70f:/dev/nvme0n1p4 c0e1b914-8872-4c8e-a6da-c6275e235540:/dev/nvme0n1p3] Jul 13 07:02:04.397776 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.103017 2586 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 13 07:02:04.397776 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.105788 2586 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 13 07:02:04.397971 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.108533 2586 manager.go:216] Machine: {Timestamp:2026-07-13 07:02:04.107061922 +0000 UTC m=+0.393461138 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3116285 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec25c4fd4a7c4249682c4b36ed24cc90 SystemUUID:ec25c4fd-4a7c-4249-682c-4b36ed24cc90 BootID:4834f445-884e-4eb9-b4e3-64c6a1c56f2e Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:16:d1:14:f8:83 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:16:d1:14:f8:83 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:c2:06:64:48:54:f8 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 13 07:02:04.397971 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.108646 2586 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 13 07:02:04.397971 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.108784 2586 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 13 07:02:04.397971 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.112703 2586 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 13 07:02:04.397971 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.112736 2586 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-37.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 13 07:02:04.397971 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.114241 2586 topology_manager.go:138] "Creating topology manager with none policy" Jul 13 07:02:04.397971 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.114254 2586 container_manager_linux.go:309] "Creating device plugin manager" Jul 13 07:02:04.397971 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.114263 2586 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 13 07:02:04.397971 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.115042 2586 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 13 07:02:04.397971 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.116576 2586 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 13 07:02:04.397971 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.119966 2586 state_mem.go:36] "Initialized new in-memory state store" Jul 13 07:02:04.397971 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.120109 2586 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 13 07:02:04.397971 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.120141 2586 kubelet.go:486] "Attempting to sync node with API server" Jul 13 07:02:04.397971 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.120183 2586 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 13 07:02:04.397971 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.120202 2586 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 13 07:02:04.397971 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.120219 2586 kubelet.go:398] "Adding apiserver pod source" Jul 13 07:02:04.397971 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.120232 2586 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 13 07:02:04.397971 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.120649 2586 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 13 07:02:04.398512 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.120670 2586 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 13 07:02:04.398512 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.123097 2586 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 13 07:02:04.398512 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.125601 2586 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 13 07:02:04.398512 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.125630 2586 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 13 07:02:04.398512 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.125832 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 13 07:02:04.398512 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.125866 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 13 07:02:04.398512 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.125875 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 13 07:02:04.398512 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.125880 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 13 07:02:04.398512 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.125889 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 13 07:02:04.398512 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.125896 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 13 07:02:04.398512 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.126472 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 13 07:02:04.398512 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.126486 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 13 07:02:04.398512 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.126496 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 13 07:02:04.398512 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.126505 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 13 07:02:04.398512 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.126527 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 13 07:02:04.398512 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.126541 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 13 07:02:04.398512 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.127553 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 13 07:02:04.398512 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.127564 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 13 07:02:04.398512 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.131131 2586 server.go:1268] "Started kubelet" Jul 13 07:02:04.398512 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.131222 2586 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 13 07:02:04.398512 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.137404 2586 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-37.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 13 07:02:04.398512 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:04.138189 2586 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-37.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 13 07:02:04.398512 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:04.138490 2586 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 13 07:02:04.398512 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.138535 2586 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 13 07:02:04.398512 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.138622 2586 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 13 07:02:04.398512 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.140254 2586 server.go:310] "Adding debug handlers to kubelet server" Jul 13 07:02:04.398512 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.142619 2586 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 13 07:02:04.398512 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:04.143989 2586 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 13 07:02:04.398512 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:04.143597 2586 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-37.ec2.internal.18c1c75bb65faa47 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-37.ec2.internal,UID:ip-10-0-141-37.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-37.ec2.internal,},FirstTimestamp:2026-07-13 07:02:04.131101255 +0000 UTC m=+0.417500471,LastTimestamp:2026-07-13 07:02:04.131101255 +0000 UTC m=+0.417500471,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-37.ec2.internal,}" Jul 13 07:02:04.398512 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.148749 2586 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 13 07:02:04.398512 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.148772 2586 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 13 07:02:04.398512 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:04.148905 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-37.ec2.internal\" not found" Jul 13 07:02:04.398512 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.149559 2586 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 13 07:02:04.398512 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.149618 2586 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 13 07:02:04.398512 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.149634 2586 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 13 07:02:04.398512 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.149729 2586 reconstruct.go:97] "Volume reconstruction finished" Jul 13 07:02:04.398512 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.149738 2586 reconciler.go:29] "Reconciler: start to sync state" Jul 13 07:02:04.398512 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.150308 2586 factory.go:55] Registering systemd factory Jul 13 07:02:04.398512 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.150326 2586 factory.go:223] Registration of the systemd container factory successfully Jul 13 07:02:04.398512 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.150553 2586 factory.go:153] Registering CRI-O factory Jul 13 07:02:04.398512 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.150567 2586 factory.go:223] Registration of the crio container factory successfully Jul 13 07:02:04.399719 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.150643 2586 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 13 07:02:04.399719 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.150669 2586 factory.go:103] Registering Raw factory Jul 13 07:02:04.399719 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.150684 2586 manager.go:1196] Started watching for new ooms in manager Jul 13 07:02:04.399719 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.151068 2586 manager.go:349] Starting recovery of all containers Jul 13 07:02:04.399719 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.152567 2586 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rcwk8" Jul 13 07:02:04.399719 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.161123 2586 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rcwk8" Jul 13 07:02:04.399719 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:04.163820 2586 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 13 07:02:04.399719 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:04.165995 2586 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-37.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 13 07:02:04.399719 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.167635 2586 manager.go:354] Recovery completed Jul 13 07:02:04.399719 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:04.169539 2586 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 13 07:02:04.399719 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.172986 2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 07:02:04.399719 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.175936 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-37.ec2.internal" event="NodeHasSufficientMemory" Jul 13 07:02:04.399719 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.176097 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-37.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 07:02:04.399719 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.176107 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-37.ec2.internal" event="NodeHasSufficientPID" Jul 13 07:02:04.399719 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.186972 2586 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 13 07:02:04.399719 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.186985 2586 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 13 07:02:04.399719 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.186993 2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 07:02:04.399719 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.187008 2586 state_mem.go:36] "Initialized new in-memory state store" Jul 13 07:02:04.399719 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.188019 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-37.ec2.internal" event="NodeHasSufficientMemory" Jul 13 07:02:04.399719 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.188049 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-37.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 07:02:04.399719 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.188064 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-37.ec2.internal" event="NodeHasSufficientPID" Jul 13 07:02:04.399719 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.188898 2586 policy_none.go:49] "None policy: Start" Jul 13 07:02:04.399719 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.188913 2586 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 13 07:02:04.399719 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.188925 2586 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 13 07:02:04.399719 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.189854 2586 policy_none.go:47] "Start" Jul 13 07:02:04.399719 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:04.190788 2586 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-37.ec2.internal.18c1c75bb90df75c default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-37.ec2.internal,UID:ip-10-0-141-37.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-37.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-37.ec2.internal,},FirstTimestamp:2026-07-13 07:02:04.176078684 +0000 UTC m=+0.462477903,LastTimestamp:2026-07-13 07:02:04.176078684 +0000 UTC m=+0.462477903,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-37.ec2.internal,}" Jul 13 07:02:04.399719 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.240566 2586 manager.go:337] "Starting Device Plugin manager" Jul 13 07:02:04.399719 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:04.240601 2586 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 13 07:02:04.399719 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.240611 2586 server.go:87] "Starting device plugin registration server" Jul 13 07:02:04.399719 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.240889 2586 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 13 07:02:04.399719 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.240901 2586 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 13 07:02:04.399719 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.241006 2586 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 13 07:02:04.399719 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.241097 2586 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 13 07:02:04.399719 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.241106 2586 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 13 07:02:04.399719 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:04.241629 2586 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 13 07:02:04.399719 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:04.241667 2586 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-37.ec2.internal\" not found" Jul 13 07:02:04.399719 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.341277 2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 07:02:04.402570 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.343479 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-37.ec2.internal" event="NodeHasSufficientMemory" Jul 13 07:02:04.402570 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.343508 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-37.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 07:02:04.402570 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.343519 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-37.ec2.internal" event="NodeHasSufficientPID" Jul 13 07:02:04.402570 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.343547 2586 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-37.ec2.internal" Jul 13 07:02:04.402570 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.346417 2586 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 13 07:02:04.402570 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.347718 2586 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 13 07:02:04.402570 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.347761 2586 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 13 07:02:04.402570 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.347788 2586 kubelet.go:2448] "Starting kubelet main sync loop" Jul 13 07:02:04.402570 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:04.347828 2586 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 13 07:02:04.402570 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.350069 2586 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 07:02:04.402570 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.354682 2586 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-37.ec2.internal" Jul 13 07:02:04.402570 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:04.354704 2586 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-37.ec2.internal\": node \"ip-10-0-141-37.ec2.internal\" not found" Jul 13 07:02:04.402570 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:04.386869 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-37.ec2.internal\" not found" Jul 13 07:02:06.034637 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.448908 2586 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-37.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-37.ec2.internal"] Jul 13 07:02:06.034637 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.449025 2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 07:02:06.034637 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.450617 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-37.ec2.internal" event="NodeHasSufficientMemory" Jul 13 07:02:06.034637 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.450646 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-37.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 07:02:06.034637 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.450658 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-37.ec2.internal" event="NodeHasSufficientPID" Jul 13 07:02:06.034637 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.451922 2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 07:02:06.034637 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.452027 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-37.ec2.internal" Jul 13 07:02:06.034637 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.452068 2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 07:02:06.034637 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.452704 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-37.ec2.internal" event="NodeHasSufficientMemory" Jul 13 07:02:06.034637 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.452708 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-37.ec2.internal" event="NodeHasSufficientMemory" Jul 13 07:02:06.034637 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.452739 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-37.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 07:02:06.034637 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.452753 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-37.ec2.internal" event="NodeHasSufficientPID" Jul 13 07:02:06.034637 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.452756 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-37.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 07:02:06.034637 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.452767 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-37.ec2.internal" event="NodeHasSufficientPID" Jul 13 07:02:06.034637 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.453733 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-37.ec2.internal" Jul 13 07:02:06.034637 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.453764 2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 07:02:06.034637 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.454397 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-37.ec2.internal" event="NodeHasSufficientMemory" Jul 13 07:02:06.034637 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.454419 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-37.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 07:02:06.034637 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:04.454430 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-37.ec2.internal" event="NodeHasSufficientPID" Jul 13 07:02:06.034637 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:04.470450 2586 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-37.ec2.internal\" not found" node="ip-10-0-141-37.ec2.internal" Jul 13 07:02:06.034637 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:04.474434 2586 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-37.ec2.internal\" not found" node="ip-10-0-141-37.ec2.internal" Jul 13 07:02:06.034637 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:04.487088 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-37.ec2.internal\" not found" Jul 13 07:02:06.034637 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:06.001295 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0de86516d8a93e213ba2d06529343779-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-37.ec2.internal\" (UID: \"0de86516d8a93e213ba2d06529343779\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-37.ec2.internal" Jul 13 07:02:06.034637 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:06.001340 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0de86516d8a93e213ba2d06529343779-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-37.ec2.internal\" (UID: \"0de86516d8a93e213ba2d06529343779\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-37.ec2.internal" Jul 13 07:02:06.041141 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:06.038545 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-37.ec2.internal\" not found" Jul 13 07:02:06.144646 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:06.101935 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0de86516d8a93e213ba2d06529343779-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-37.ec2.internal\" (UID: \"0de86516d8a93e213ba2d06529343779\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-37.ec2.internal" Jul 13 07:02:06.144646 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:06.101968 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0de86516d8a93e213ba2d06529343779-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-37.ec2.internal\" (UID: \"0de86516d8a93e213ba2d06529343779\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-37.ec2.internal" Jul 13 07:02:06.144646 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:06.101987 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6df327e3e80f10552c93b38d2fd449c4-config\") pod \"kube-apiserver-proxy-ip-10-0-141-37.ec2.internal\" (UID: \"6df327e3e80f10552c93b38d2fd449c4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-37.ec2.internal" Jul 13 07:02:06.144646 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:06.102036 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0de86516d8a93e213ba2d06529343779-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-37.ec2.internal\" (UID: \"0de86516d8a93e213ba2d06529343779\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-37.ec2.internal" Jul 13 07:02:06.144646 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:06.102046 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0de86516d8a93e213ba2d06529343779-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-37.ec2.internal\" (UID: \"0de86516d8a93e213ba2d06529343779\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-37.ec2.internal" Jul 13 07:02:06.144646 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:06.139052 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-37.ec2.internal\" not found" Jul 13 07:02:06.258028 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:06.202403 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6df327e3e80f10552c93b38d2fd449c4-config\") pod \"kube-apiserver-proxy-ip-10-0-141-37.ec2.internal\" (UID: \"6df327e3e80f10552c93b38d2fd449c4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-37.ec2.internal" Jul 13 07:02:06.258028 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:06.202455 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6df327e3e80f10552c93b38d2fd449c4-config\") pod \"kube-apiserver-proxy-ip-10-0-141-37.ec2.internal\" (UID: \"6df327e3e80f10552c93b38d2fd449c4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-37.ec2.internal" Jul 13 07:02:06.258028 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:06.223539 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-37.ec2.internal" Jul 13 07:02:06.258028 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:06.227669 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-37.ec2.internal" Jul 13 07:02:06.258028 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:06.240096 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-37.ec2.internal\" not found" Jul 13 07:02:06.373591 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:06.340615 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-37.ec2.internal\" not found" Jul 13 07:02:06.467719 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:06.441007 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-37.ec2.internal\" not found" Jul 13 07:02:06.524709 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:06.524670 2586 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 13 07:02:06.524855 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:06.524833 2586 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 13 07:02:06.541941 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:06.541917 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-37.ec2.internal\" not found" Jul 13 07:02:06.621095 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:06.621053 2586 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-12 06:57:05 +0000 UTC" deadline="2028-03-31 05:12:26.200113556 +0000 UTC" Jul 13 07:02:06.621095 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:06.621088 2586 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15046h10m19.579030416s" Jul 13 07:02:06.642285 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:06.642207 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-37.ec2.internal\" not found" Jul 13 07:02:06.742891 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:06.742852 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-37.ec2.internal\" not found" Jul 13 07:02:06.742891 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:06.742876 2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 07:02:06.744053 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:06.744038 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-37.ec2.internal" event="NodeHasSufficientMemory" Jul 13 07:02:06.744109 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:06.744067 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-37.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 07:02:06.744109 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:06.744077 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-37.ec2.internal" event="NodeHasSufficientPID" Jul 13 07:02:06.823608 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:06.823577 2586 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 07:02:06.864605 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:06.864578 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0de86516d8a93e213ba2d06529343779.slice/crio-5c4889cbfb8955532686a7795b2b6c12a4ac792ddfe083c2c8fbc6f72de2ebe6 WatchSource:0}: Error finding container 5c4889cbfb8955532686a7795b2b6c12a4ac792ddfe083c2c8fbc6f72de2ebe6: Status 404 returned error can't find the container with id 5c4889cbfb8955532686a7795b2b6c12a4ac792ddfe083c2c8fbc6f72de2ebe6 Jul 13 07:02:06.867787 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:06.867772 2586 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 07:02:06.882122 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:06.882100 2586 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 07:02:06.900600 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:06.900551 2586 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-37.ec2.internal" Jul 13 07:02:06.914108 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:06.914089 2586 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 13 07:02:06.916226 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:06.916214 2586 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-37.ec2.internal" Jul 13 07:02:06.922342 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:06.922327 2586 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 13 07:02:06.926798 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:06.926776 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6df327e3e80f10552c93b38d2fd449c4.slice/crio-69c8665a96052afd3cbb43be0af397f30bac4da0b8db2423f07c75ae464ec37c WatchSource:0}: Error finding container 69c8665a96052afd3cbb43be0af397f30bac4da0b8db2423f07c75ae464ec37c: Status 404 returned error can't find the container with id 69c8665a96052afd3cbb43be0af397f30bac4da0b8db2423f07c75ae464ec37c Jul 13 07:02:07.042864 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.042834 2586 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 07:02:07.572773 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.572506 2586 apiserver.go:52] "Watching apiserver" Jul 13 07:02:07.579290 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.578807 2586 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 13 07:02:07.582057 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.581593 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-xs7q5","kube-system/kube-apiserver-proxy-ip-10-0-141-37.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m28rr","openshift-cluster-node-tuning-operator/tuned-6l8np","openshift-dns/node-resolver-g6kgx","openshift-multus/multus-additional-cni-plugins-56rlp","openshift-network-diagnostics/network-check-target-6xdjl","openshift-network-operator/iptables-alerter-z7jkb","kube-system/global-pull-secret-syncer-lb8th","openshift-image-registry/node-ca-vxcrv","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-37.ec2.internal","openshift-multus/multus-g9z6n","openshift-multus/network-metrics-daemon-znhkv","openshift-ovn-kubernetes/ovnkube-node-7b964"] Jul 13 07:02:07.583963 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.583936 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6xdjl" Jul 13 07:02:07.584080 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:07.584024 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6xdjl" podUID="0ade3045-6b94-41a9-baeb-db7ce7a84c7a" Jul 13 07:02:07.585796 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.585772 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-z7jkb" Jul 13 07:02:07.587793 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.587775 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m28rr" Jul 13 07:02:07.588141 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.588120 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-sclkx\"" Jul 13 07:02:07.588225 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.588200 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 13 07:02:07.588323 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.588303 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 13 07:02:07.588405 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.588392 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 13 07:02:07.589813 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.589792 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 13 07:02:07.589913 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.589849 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 13 07:02:07.589913 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.589902 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-cgsfg\"" Jul 13 07:02:07.590022 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.589976 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 13 07:02:07.590104 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.590088 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6l8np" Jul 13 07:02:07.591900 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.591882 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-4cjjx\"" Jul 13 07:02:07.592049 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.592036 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 13 07:02:07.592205 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.592189 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-znhkv" Jul 13 07:02:07.592269 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.592190 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 13 07:02:07.592319 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:07.592257 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-znhkv" podUID="638764ba-e970-42f1-83ec-cf5c7075d61e" Jul 13 07:02:07.597487 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.597275 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-56rlp" Jul 13 07:02:07.600743 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.599406 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 13 07:02:07.600743 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.599649 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-f4kdj\"" Jul 13 07:02:07.600743 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.599717 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 13 07:02:07.600743 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.599720 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 13 07:02:07.600743 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.599813 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 13 07:02:07.600743 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.599873 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 13 07:02:07.600743 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.599977 2586 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 13 07:02:07.600743 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.600056 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lb8th" Jul 13 07:02:07.600743 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:07.600111 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lb8th" podUID="c3724c5e-1a2b-410b-a745-b0f90bf27358" Jul 13 07:02:07.601849 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.601530 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xs7q5" Jul 13 07:02:07.603359 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.603337 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 13 07:02:07.603612 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.603596 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 13 07:02:07.603843 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.603797 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-lqh7p\"" Jul 13 07:02:07.604156 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.604138 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vxcrv" Jul 13 07:02:07.607766 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.606401 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 13 07:02:07.607766 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.606444 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-cj4fd\"" Jul 13 07:02:07.607766 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.606665 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 13 07:02:07.607766 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.606800 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 13 07:02:07.607766 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.607726 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-g9z6n" Jul 13 07:02:07.609957 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.609350 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 13 07:02:07.609957 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.609696 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-9mrd4\"" Jul 13 07:02:07.611384 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.610432 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2abf1e96-1ba2-4cc1-a632-48e28b374830-konnectivity-ca\") pod \"konnectivity-agent-xs7q5\" (UID: \"2abf1e96-1ba2-4cc1-a632-48e28b374830\") " pod="kube-system/konnectivity-agent-xs7q5" Jul 13 07:02:07.611384 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.610454 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d7d6c54b-cee3-4b7c-9b72-2afe08125102-iptables-alerter-script\") pod \"iptables-alerter-z7jkb\" (UID: \"d7d6c54b-cee3-4b7c-9b72-2afe08125102\") " pod="openshift-network-operator/iptables-alerter-z7jkb" Jul 13 07:02:07.611384 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.610472 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d7d6c54b-cee3-4b7c-9b72-2afe08125102-host-slash\") pod \"iptables-alerter-z7jkb\" (UID: \"d7d6c54b-cee3-4b7c-9b72-2afe08125102\") " pod="openshift-network-operator/iptables-alerter-z7jkb" Jul 13 07:02:07.611384 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.610489 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c3724c5e-1a2b-410b-a745-b0f90bf27358-original-pull-secret\") pod \"global-pull-secret-syncer-lb8th\" (UID: \"c3724c5e-1a2b-410b-a745-b0f90bf27358\") " pod="kube-system/global-pull-secret-syncer-lb8th" Jul 13 07:02:07.611384 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.610505 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9dfz\" (UniqueName: \"kubernetes.io/projected/e1b48537-330d-49c0-80be-80fe72862d9d-kube-api-access-k9dfz\") pod \"tuned-6l8np\" (UID: \"e1b48537-330d-49c0-80be-80fe72862d9d\") " pod="openshift-cluster-node-tuning-operator/tuned-6l8np" Jul 13 07:02:07.611384 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.610520 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98fzf\" (UniqueName: \"kubernetes.io/projected/2d5a84b8-155e-4dc0-8526-19a67235d08f-kube-api-access-98fzf\") pod \"multus-additional-cni-plugins-56rlp\" (UID: \"2d5a84b8-155e-4dc0-8526-19a67235d08f\") " pod="openshift-multus/multus-additional-cni-plugins-56rlp" Jul 13 07:02:07.611384 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.610540 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e1b48537-330d-49c0-80be-80fe72862d9d-etc-sysctl-d\") pod \"tuned-6l8np\" (UID: \"e1b48537-330d-49c0-80be-80fe72862d9d\") " pod="openshift-cluster-node-tuning-operator/tuned-6l8np" Jul 13 07:02:07.611384 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.610553 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e1b48537-330d-49c0-80be-80fe72862d9d-lib-modules\") pod \"tuned-6l8np\" (UID: \"e1b48537-330d-49c0-80be-80fe72862d9d\") " pod="openshift-cluster-node-tuning-operator/tuned-6l8np" Jul 13 07:02:07.611384 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.610574 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e1b48537-330d-49c0-80be-80fe72862d9d-var-lib-kubelet\") pod \"tuned-6l8np\" (UID: \"e1b48537-330d-49c0-80be-80fe72862d9d\") " pod="openshift-cluster-node-tuning-operator/tuned-6l8np" Jul 13 07:02:07.611384 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.610597 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2d5a84b8-155e-4dc0-8526-19a67235d08f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-56rlp\" (UID: \"2d5a84b8-155e-4dc0-8526-19a67235d08f\") " pod="openshift-multus/multus-additional-cni-plugins-56rlp" Jul 13 07:02:07.611384 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.610640 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c3724c5e-1a2b-410b-a745-b0f90bf27358-kubelet-config\") pod \"global-pull-secret-syncer-lb8th\" (UID: \"c3724c5e-1a2b-410b-a745-b0f90bf27358\") " pod="kube-system/global-pull-secret-syncer-lb8th" Jul 13 07:02:07.611384 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.610672 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4198fdf7-f7df-4dc0-99e0-92b0c4874587-registration-dir\") pod \"aws-ebs-csi-driver-node-m28rr\" (UID: \"4198fdf7-f7df-4dc0-99e0-92b0c4874587\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m28rr" Jul 13 07:02:07.611384 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.610712 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e1b48537-330d-49c0-80be-80fe72862d9d-etc-modprobe-d\") pod \"tuned-6l8np\" (UID: \"e1b48537-330d-49c0-80be-80fe72862d9d\") " pod="openshift-cluster-node-tuning-operator/tuned-6l8np" Jul 13 07:02:07.611384 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.610735 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e1b48537-330d-49c0-80be-80fe72862d9d-sys\") pod \"tuned-6l8np\" (UID: \"e1b48537-330d-49c0-80be-80fe72862d9d\") " pod="openshift-cluster-node-tuning-operator/tuned-6l8np" Jul 13 07:02:07.611384 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.610757 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4198fdf7-f7df-4dc0-99e0-92b0c4874587-device-dir\") pod \"aws-ebs-csi-driver-node-m28rr\" (UID: \"4198fdf7-f7df-4dc0-99e0-92b0c4874587\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m28rr" Jul 13 07:02:07.611384 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.610780 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4198fdf7-f7df-4dc0-99e0-92b0c4874587-sys-fs\") pod \"aws-ebs-csi-driver-node-m28rr\" (UID: \"4198fdf7-f7df-4dc0-99e0-92b0c4874587\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m28rr" Jul 13 07:02:07.612144 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.610808 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4h5nt\" (UniqueName: \"kubernetes.io/projected/d7d6c54b-cee3-4b7c-9b72-2afe08125102-kube-api-access-4h5nt\") pod \"iptables-alerter-z7jkb\" (UID: \"d7d6c54b-cee3-4b7c-9b72-2afe08125102\") " pod="openshift-network-operator/iptables-alerter-z7jkb" Jul 13 07:02:07.612144 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.610835 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2d5a84b8-155e-4dc0-8526-19a67235d08f-os-release\") pod \"multus-additional-cni-plugins-56rlp\" (UID: \"2d5a84b8-155e-4dc0-8526-19a67235d08f\") " pod="openshift-multus/multus-additional-cni-plugins-56rlp" Jul 13 07:02:07.612144 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.610859 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2d5a84b8-155e-4dc0-8526-19a67235d08f-cni-binary-copy\") pod \"multus-additional-cni-plugins-56rlp\" (UID: \"2d5a84b8-155e-4dc0-8526-19a67235d08f\") " pod="openshift-multus/multus-additional-cni-plugins-56rlp" Jul 13 07:02:07.612144 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.610888 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c3724c5e-1a2b-410b-a745-b0f90bf27358-dbus\") pod \"global-pull-secret-syncer-lb8th\" (UID: \"c3724c5e-1a2b-410b-a745-b0f90bf27358\") " pod="kube-system/global-pull-secret-syncer-lb8th" Jul 13 07:02:07.612144 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.610911 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxcr7\" (UniqueName: \"kubernetes.io/projected/4198fdf7-f7df-4dc0-99e0-92b0c4874587-kube-api-access-rxcr7\") pod \"aws-ebs-csi-driver-node-m28rr\" (UID: \"4198fdf7-f7df-4dc0-99e0-92b0c4874587\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m28rr" Jul 13 07:02:07.612144 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.610952 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e1b48537-330d-49c0-80be-80fe72862d9d-etc-sysconfig\") pod \"tuned-6l8np\" (UID: \"e1b48537-330d-49c0-80be-80fe72862d9d\") " pod="openshift-cluster-node-tuning-operator/tuned-6l8np" Jul 13 07:02:07.612144 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.611010 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhst6\" (UniqueName: \"kubernetes.io/projected/638764ba-e970-42f1-83ec-cf5c7075d61e-kube-api-access-zhst6\") pod \"network-metrics-daemon-znhkv\" (UID: \"638764ba-e970-42f1-83ec-cf5c7075d61e\") " pod="openshift-multus/network-metrics-daemon-znhkv" Jul 13 07:02:07.612144 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.611046 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2d5a84b8-155e-4dc0-8526-19a67235d08f-cnibin\") pod \"multus-additional-cni-plugins-56rlp\" (UID: \"2d5a84b8-155e-4dc0-8526-19a67235d08f\") " pod="openshift-multus/multus-additional-cni-plugins-56rlp" Jul 13 07:02:07.612144 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.611089 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2d5a84b8-155e-4dc0-8526-19a67235d08f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-56rlp\" (UID: \"2d5a84b8-155e-4dc0-8526-19a67235d08f\") " pod="openshift-multus/multus-additional-cni-plugins-56rlp" Jul 13 07:02:07.612144 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.611112 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4198fdf7-f7df-4dc0-99e0-92b0c4874587-socket-dir\") pod \"aws-ebs-csi-driver-node-m28rr\" (UID: \"4198fdf7-f7df-4dc0-99e0-92b0c4874587\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m28rr" Jul 13 07:02:07.612144 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.611135 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e1b48537-330d-49c0-80be-80fe72862d9d-etc-sysctl-conf\") pod \"tuned-6l8np\" (UID: \"e1b48537-330d-49c0-80be-80fe72862d9d\") " pod="openshift-cluster-node-tuning-operator/tuned-6l8np" Jul 13 07:02:07.612144 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.611157 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e1b48537-330d-49c0-80be-80fe72862d9d-etc-systemd\") pod \"tuned-6l8np\" (UID: \"e1b48537-330d-49c0-80be-80fe72862d9d\") " pod="openshift-cluster-node-tuning-operator/tuned-6l8np" Jul 13 07:02:07.612144 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.611185 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4198fdf7-f7df-4dc0-99e0-92b0c4874587-etc-selinux\") pod \"aws-ebs-csi-driver-node-m28rr\" (UID: \"4198fdf7-f7df-4dc0-99e0-92b0c4874587\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m28rr" Jul 13 07:02:07.612144 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.611199 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e1b48537-330d-49c0-80be-80fe72862d9d-run\") pod \"tuned-6l8np\" (UID: \"e1b48537-330d-49c0-80be-80fe72862d9d\") " pod="openshift-cluster-node-tuning-operator/tuned-6l8np" Jul 13 07:02:07.612144 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.611627 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-g6kgx" Jul 13 07:02:07.612144 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.611645 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e1b48537-330d-49c0-80be-80fe72862d9d-host\") pod \"tuned-6l8np\" (UID: \"e1b48537-330d-49c0-80be-80fe72862d9d\") " pod="openshift-cluster-node-tuning-operator/tuned-6l8np" Jul 13 07:02:07.612144 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.611671 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e1b48537-330d-49c0-80be-80fe72862d9d-tmp\") pod \"tuned-6l8np\" (UID: \"e1b48537-330d-49c0-80be-80fe72862d9d\") " pod="openshift-cluster-node-tuning-operator/tuned-6l8np" Jul 13 07:02:07.613267 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.611709 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4198fdf7-f7df-4dc0-99e0-92b0c4874587-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m28rr\" (UID: \"4198fdf7-f7df-4dc0-99e0-92b0c4874587\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m28rr" Jul 13 07:02:07.613267 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.611738 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldwqr\" (UniqueName: \"kubernetes.io/projected/0ade3045-6b94-41a9-baeb-db7ce7a84c7a-kube-api-access-ldwqr\") pod \"network-check-target-6xdjl\" (UID: \"0ade3045-6b94-41a9-baeb-db7ce7a84c7a\") " pod="openshift-network-diagnostics/network-check-target-6xdjl" Jul 13 07:02:07.613267 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.611765 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2d5a84b8-155e-4dc0-8526-19a67235d08f-system-cni-dir\") pod \"multus-additional-cni-plugins-56rlp\" (UID: \"2d5a84b8-155e-4dc0-8526-19a67235d08f\") " pod="openshift-multus/multus-additional-cni-plugins-56rlp" Jul 13 07:02:07.613267 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.611791 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2d5a84b8-155e-4dc0-8526-19a67235d08f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-56rlp\" (UID: \"2d5a84b8-155e-4dc0-8526-19a67235d08f\") " pod="openshift-multus/multus-additional-cni-plugins-56rlp" Jul 13 07:02:07.613267 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.611833 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e1b48537-330d-49c0-80be-80fe72862d9d-etc-kubernetes\") pod \"tuned-6l8np\" (UID: \"e1b48537-330d-49c0-80be-80fe72862d9d\") " pod="openshift-cluster-node-tuning-operator/tuned-6l8np" Jul 13 07:02:07.613267 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.611856 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e1b48537-330d-49c0-80be-80fe72862d9d-etc-tuned\") pod \"tuned-6l8np\" (UID: \"e1b48537-330d-49c0-80be-80fe72862d9d\") " pod="openshift-cluster-node-tuning-operator/tuned-6l8np" Jul 13 07:02:07.613267 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.611867 2586 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 13 07:02:07.613267 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.611880 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/638764ba-e970-42f1-83ec-cf5c7075d61e-metrics-certs\") pod \"network-metrics-daemon-znhkv\" (UID: \"638764ba-e970-42f1-83ec-cf5c7075d61e\") " pod="openshift-multus/network-metrics-daemon-znhkv" Jul 13 07:02:07.613267 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.611902 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2abf1e96-1ba2-4cc1-a632-48e28b374830-agent-certs\") pod \"konnectivity-agent-xs7q5\" (UID: \"2abf1e96-1ba2-4cc1-a632-48e28b374830\") " pod="kube-system/konnectivity-agent-xs7q5" Jul 13 07:02:07.613715 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.613289 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 13 07:02:07.613715 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.613641 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.613817 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.613785 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-xvm9m\"" Jul 13 07:02:07.614087 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.614070 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 13 07:02:07.615556 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.615535 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-c6nz8\"" Jul 13 07:02:07.615644 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.615596 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 13 07:02:07.621184 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.621163 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 13 07:02:07.621736 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.621294 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 13 07:02:07.621736 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.621444 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 13 07:02:07.621736 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.621713 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 13 07:02:07.621736 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.621723 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 13 07:02:07.634789 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.634770 2586 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-dvl8w" Jul 13 07:02:07.644286 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.644267 2586 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-dvl8w" Jul 13 07:02:07.701277 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.701250 2586 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 13 07:02:07.713024 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.713000 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e1b48537-330d-49c0-80be-80fe72862d9d-etc-systemd\") pod \"tuned-6l8np\" (UID: \"e1b48537-330d-49c0-80be-80fe72862d9d\") " pod="openshift-cluster-node-tuning-operator/tuned-6l8np" Jul 13 07:02:07.713148 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.713036 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/89973bd4-81f7-4536-841e-1571b0fd9506-multus-daemon-config\") pod \"multus-g9z6n\" (UID: \"89973bd4-81f7-4536-841e-1571b0fd9506\") " pod="openshift-multus/multus-g9z6n" Jul 13 07:02:07.713148 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.713065 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4198fdf7-f7df-4dc0-99e0-92b0c4874587-socket-dir\") pod \"aws-ebs-csi-driver-node-m28rr\" (UID: \"4198fdf7-f7df-4dc0-99e0-92b0c4874587\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m28rr" Jul 13 07:02:07.713148 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.713096 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e1b48537-330d-49c0-80be-80fe72862d9d-etc-sysctl-conf\") pod \"tuned-6l8np\" (UID: \"e1b48537-330d-49c0-80be-80fe72862d9d\") " pod="openshift-cluster-node-tuning-operator/tuned-6l8np" Jul 13 07:02:07.713148 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.713112 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/be89589f-0a40-4e74-9ecf-aeac7e231126-host\") pod \"node-ca-vxcrv\" (UID: \"be89589f-0a40-4e74-9ecf-aeac7e231126\") " pod="openshift-image-registry/node-ca-vxcrv" Jul 13 07:02:07.713148 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.713128 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e1b48537-330d-49c0-80be-80fe72862d9d-etc-systemd\") pod \"tuned-6l8np\" (UID: \"e1b48537-330d-49c0-80be-80fe72862d9d\") " pod="openshift-cluster-node-tuning-operator/tuned-6l8np" Jul 13 07:02:07.713397 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.713132 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/89973bd4-81f7-4536-841e-1571b0fd9506-host-run-netns\") pod \"multus-g9z6n\" (UID: \"89973bd4-81f7-4536-841e-1571b0fd9506\") " pod="openshift-multus/multus-g9z6n" Jul 13 07:02:07.713397 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.713191 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3d71897a-4b2e-4e4e-9525-ccd89bd18458-var-lib-openvswitch\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.713397 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.713218 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3d71897a-4b2e-4e4e-9525-ccd89bd18458-host-slash\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.713397 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.713250 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e1b48537-330d-49c0-80be-80fe72862d9d-run\") pod \"tuned-6l8np\" (UID: \"e1b48537-330d-49c0-80be-80fe72862d9d\") " pod="openshift-cluster-node-tuning-operator/tuned-6l8np" Jul 13 07:02:07.713397 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.713254 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4198fdf7-f7df-4dc0-99e0-92b0c4874587-socket-dir\") pod \"aws-ebs-csi-driver-node-m28rr\" (UID: \"4198fdf7-f7df-4dc0-99e0-92b0c4874587\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m28rr" Jul 13 07:02:07.713397 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.713277 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/89973bd4-81f7-4536-841e-1571b0fd9506-host-var-lib-cni-multus\") pod \"multus-g9z6n\" (UID: \"89973bd4-81f7-4536-841e-1571b0fd9506\") " pod="openshift-multus/multus-g9z6n" Jul 13 07:02:07.713397 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.713302 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/89973bd4-81f7-4536-841e-1571b0fd9506-hostroot\") pod \"multus-g9z6n\" (UID: \"89973bd4-81f7-4536-841e-1571b0fd9506\") " pod="openshift-multus/multus-g9z6n" Jul 13 07:02:07.713397 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.713348 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3d71897a-4b2e-4e4e-9525-ccd89bd18458-run-systemd\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.713397 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.713370 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3d71897a-4b2e-4e4e-9525-ccd89bd18458-node-log\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.713397 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.713392 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3d71897a-4b2e-4e4e-9525-ccd89bd18458-ovnkube-script-lib\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.713906 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.713407 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e1b48537-330d-49c0-80be-80fe72862d9d-etc-sysctl-conf\") pod \"tuned-6l8np\" (UID: \"e1b48537-330d-49c0-80be-80fe72862d9d\") " pod="openshift-cluster-node-tuning-operator/tuned-6l8np" Jul 13 07:02:07.713906 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.713417 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4198fdf7-f7df-4dc0-99e0-92b0c4874587-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m28rr\" (UID: \"4198fdf7-f7df-4dc0-99e0-92b0c4874587\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m28rr" Jul 13 07:02:07.713906 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.713463 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e1b48537-330d-49c0-80be-80fe72862d9d-run\") pod \"tuned-6l8np\" (UID: \"e1b48537-330d-49c0-80be-80fe72862d9d\") " pod="openshift-cluster-node-tuning-operator/tuned-6l8np" Jul 13 07:02:07.713906 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.713467 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldwqr\" (UniqueName: \"kubernetes.io/projected/0ade3045-6b94-41a9-baeb-db7ce7a84c7a-kube-api-access-ldwqr\") pod \"network-check-target-6xdjl\" (UID: \"0ade3045-6b94-41a9-baeb-db7ce7a84c7a\") " pod="openshift-network-diagnostics/network-check-target-6xdjl" Jul 13 07:02:07.713906 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.713496 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4198fdf7-f7df-4dc0-99e0-92b0c4874587-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m28rr\" (UID: \"4198fdf7-f7df-4dc0-99e0-92b0c4874587\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m28rr" Jul 13 07:02:07.713906 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.713504 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2d5a84b8-155e-4dc0-8526-19a67235d08f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-56rlp\" (UID: \"2d5a84b8-155e-4dc0-8526-19a67235d08f\") " pod="openshift-multus/multus-additional-cni-plugins-56rlp" Jul 13 07:02:07.713906 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.713532 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e1b48537-330d-49c0-80be-80fe72862d9d-etc-kubernetes\") pod \"tuned-6l8np\" (UID: \"e1b48537-330d-49c0-80be-80fe72862d9d\") " pod="openshift-cluster-node-tuning-operator/tuned-6l8np" Jul 13 07:02:07.713906 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.713557 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/89973bd4-81f7-4536-841e-1571b0fd9506-system-cni-dir\") pod \"multus-g9z6n\" (UID: \"89973bd4-81f7-4536-841e-1571b0fd9506\") " pod="openshift-multus/multus-g9z6n" Jul 13 07:02:07.713906 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.713579 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/89973bd4-81f7-4536-841e-1571b0fd9506-host-run-multus-certs\") pod \"multus-g9z6n\" (UID: \"89973bd4-81f7-4536-841e-1571b0fd9506\") " pod="openshift-multus/multus-g9z6n" Jul 13 07:02:07.713906 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.713602 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3d71897a-4b2e-4e4e-9525-ccd89bd18458-ovnkube-config\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.713906 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.713624 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3d71897a-4b2e-4e4e-9525-ccd89bd18458-ovn-node-metrics-cert\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.713906 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.713647 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/638764ba-e970-42f1-83ec-cf5c7075d61e-metrics-certs\") pod \"network-metrics-daemon-znhkv\" (UID: \"638764ba-e970-42f1-83ec-cf5c7075d61e\") " pod="openshift-multus/network-metrics-daemon-znhkv" Jul 13 07:02:07.713906 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.713670 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d7d6c54b-cee3-4b7c-9b72-2afe08125102-iptables-alerter-script\") pod \"iptables-alerter-z7jkb\" (UID: \"d7d6c54b-cee3-4b7c-9b72-2afe08125102\") " pod="openshift-network-operator/iptables-alerter-z7jkb" Jul 13 07:02:07.713906 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.713712 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c3724c5e-1a2b-410b-a745-b0f90bf27358-original-pull-secret\") pod \"global-pull-secret-syncer-lb8th\" (UID: \"c3724c5e-1a2b-410b-a745-b0f90bf27358\") " pod="kube-system/global-pull-secret-syncer-lb8th" Jul 13 07:02:07.713906 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.713736 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3d71897a-4b2e-4e4e-9525-ccd89bd18458-host-kubelet\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.713906 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.713752 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e1b48537-330d-49c0-80be-80fe72862d9d-etc-kubernetes\") pod \"tuned-6l8np\" (UID: \"e1b48537-330d-49c0-80be-80fe72862d9d\") " pod="openshift-cluster-node-tuning-operator/tuned-6l8np" Jul 13 07:02:07.713906 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.713757 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3d71897a-4b2e-4e4e-9525-ccd89bd18458-systemd-units\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.714698 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.713789 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3d71897a-4b2e-4e4e-9525-ccd89bd18458-host-cni-netd\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.714698 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:07.713885 2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 07:02:07.714698 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.713924 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/89973bd4-81f7-4536-841e-1571b0fd9506-host-run-k8s-cni-cncf-io\") pod \"multus-g9z6n\" (UID: \"89973bd4-81f7-4536-841e-1571b0fd9506\") " pod="openshift-multus/multus-g9z6n" Jul 13 07:02:07.714698 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.713949 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/89973bd4-81f7-4536-841e-1571b0fd9506-multus-conf-dir\") pod \"multus-g9z6n\" (UID: \"89973bd4-81f7-4536-841e-1571b0fd9506\") " pod="openshift-multus/multus-g9z6n" Jul 13 07:02:07.714698 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.713979 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3d71897a-4b2e-4e4e-9525-ccd89bd18458-run-openvswitch\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.714698 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.714020 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3d71897a-4b2e-4e4e-9525-ccd89bd18458-host-cni-bin\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.714698 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.714046 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2d5a84b8-155e-4dc0-8526-19a67235d08f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-56rlp\" (UID: \"2d5a84b8-155e-4dc0-8526-19a67235d08f\") " pod="openshift-multus/multus-additional-cni-plugins-56rlp" Jul 13 07:02:07.714698 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.714078 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4198fdf7-f7df-4dc0-99e0-92b0c4874587-registration-dir\") pod \"aws-ebs-csi-driver-node-m28rr\" (UID: \"4198fdf7-f7df-4dc0-99e0-92b0c4874587\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m28rr" Jul 13 07:02:07.714698 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.714103 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3d71897a-4b2e-4e4e-9525-ccd89bd18458-env-overrides\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.714698 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.714124 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2d5a84b8-155e-4dc0-8526-19a67235d08f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-56rlp\" (UID: \"2d5a84b8-155e-4dc0-8526-19a67235d08f\") " pod="openshift-multus/multus-additional-cni-plugins-56rlp" Jul 13 07:02:07.714698 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.714167 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4198fdf7-f7df-4dc0-99e0-92b0c4874587-sys-fs\") pod \"aws-ebs-csi-driver-node-m28rr\" (UID: \"4198fdf7-f7df-4dc0-99e0-92b0c4874587\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m28rr" Jul 13 07:02:07.714698 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.714191 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4h5nt\" (UniqueName: \"kubernetes.io/projected/d7d6c54b-cee3-4b7c-9b72-2afe08125102-kube-api-access-4h5nt\") pod \"iptables-alerter-z7jkb\" (UID: \"d7d6c54b-cee3-4b7c-9b72-2afe08125102\") " pod="openshift-network-operator/iptables-alerter-z7jkb" Jul 13 07:02:07.714698 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:07.714222 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3724c5e-1a2b-410b-a745-b0f90bf27358-original-pull-secret podName:c3724c5e-1a2b-410b-a745-b0f90bf27358 nodeName:}" failed. No retries permitted until 2026-07-13 07:02:08.214203471 +0000 UTC m=+3.049767299 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c3724c5e-1a2b-410b-a745-b0f90bf27358-original-pull-secret") pod "global-pull-secret-syncer-lb8th" (UID: "c3724c5e-1a2b-410b-a745-b0f90bf27358") : object "kube-system"/"original-pull-secret" not registered Jul 13 07:02:07.714698 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.714239 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c3724c5e-1a2b-410b-a745-b0f90bf27358-dbus\") pod \"global-pull-secret-syncer-lb8th\" (UID: \"c3724c5e-1a2b-410b-a745-b0f90bf27358\") " pod="kube-system/global-pull-secret-syncer-lb8th" Jul 13 07:02:07.714698 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.714263 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e1b48537-330d-49c0-80be-80fe72862d9d-etc-sysconfig\") pod \"tuned-6l8np\" (UID: \"e1b48537-330d-49c0-80be-80fe72862d9d\") " pod="openshift-cluster-node-tuning-operator/tuned-6l8np" Jul 13 07:02:07.714698 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.714305 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d7d6c54b-cee3-4b7c-9b72-2afe08125102-iptables-alerter-script\") pod \"iptables-alerter-z7jkb\" (UID: \"d7d6c54b-cee3-4b7c-9b72-2afe08125102\") " pod="openshift-network-operator/iptables-alerter-z7jkb" Jul 13 07:02:07.715449 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.714403 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e1b48537-330d-49c0-80be-80fe72862d9d-etc-sysconfig\") pod \"tuned-6l8np\" (UID: \"e1b48537-330d-49c0-80be-80fe72862d9d\") " pod="openshift-cluster-node-tuning-operator/tuned-6l8np" Jul 13 07:02:07.715449 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.714445 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhst6\" (UniqueName: \"kubernetes.io/projected/638764ba-e970-42f1-83ec-cf5c7075d61e-kube-api-access-zhst6\") pod \"network-metrics-daemon-znhkv\" (UID: \"638764ba-e970-42f1-83ec-cf5c7075d61e\") " pod="openshift-multus/network-metrics-daemon-znhkv" Jul 13 07:02:07.715449 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.714457 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c3724c5e-1a2b-410b-a745-b0f90bf27358-dbus\") pod \"global-pull-secret-syncer-lb8th\" (UID: \"c3724c5e-1a2b-410b-a745-b0f90bf27358\") " pod="kube-system/global-pull-secret-syncer-lb8th" Jul 13 07:02:07.715449 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.714492 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2d5a84b8-155e-4dc0-8526-19a67235d08f-cnibin\") pod \"multus-additional-cni-plugins-56rlp\" (UID: \"2d5a84b8-155e-4dc0-8526-19a67235d08f\") " pod="openshift-multus/multus-additional-cni-plugins-56rlp" Jul 13 07:02:07.715449 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:07.714530 2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 07:02:07.715449 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.714572 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2d5a84b8-155e-4dc0-8526-19a67235d08f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-56rlp\" (UID: \"2d5a84b8-155e-4dc0-8526-19a67235d08f\") " pod="openshift-multus/multus-additional-cni-plugins-56rlp" Jul 13 07:02:07.715449 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.714614 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2d5a84b8-155e-4dc0-8526-19a67235d08f-cnibin\") pod \"multus-additional-cni-plugins-56rlp\" (UID: \"2d5a84b8-155e-4dc0-8526-19a67235d08f\") " pod="openshift-multus/multus-additional-cni-plugins-56rlp" Jul 13 07:02:07.715449 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.714622 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4198fdf7-f7df-4dc0-99e0-92b0c4874587-sys-fs\") pod \"aws-ebs-csi-driver-node-m28rr\" (UID: \"4198fdf7-f7df-4dc0-99e0-92b0c4874587\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m28rr" Jul 13 07:02:07.715449 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:07.714664 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/638764ba-e970-42f1-83ec-cf5c7075d61e-metrics-certs podName:638764ba-e970-42f1-83ec-cf5c7075d61e nodeName:}" failed. No retries permitted until 2026-07-13 07:02:08.214617337 +0000 UTC m=+3.050181151 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/638764ba-e970-42f1-83ec-cf5c7075d61e-metrics-certs") pod "network-metrics-daemon-znhkv" (UID: "638764ba-e970-42f1-83ec-cf5c7075d61e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 07:02:07.715449 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.714574 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2d5a84b8-155e-4dc0-8526-19a67235d08f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-56rlp\" (UID: \"2d5a84b8-155e-4dc0-8526-19a67235d08f\") " pod="openshift-multus/multus-additional-cni-plugins-56rlp" Jul 13 07:02:07.715449 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.714725 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w99lf\" (UniqueName: \"kubernetes.io/projected/89973bd4-81f7-4536-841e-1571b0fd9506-kube-api-access-w99lf\") pod \"multus-g9z6n\" (UID: \"89973bd4-81f7-4536-841e-1571b0fd9506\") " pod="openshift-multus/multus-g9z6n" Jul 13 07:02:07.715449 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.714742 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4198fdf7-f7df-4dc0-99e0-92b0c4874587-registration-dir\") pod \"aws-ebs-csi-driver-node-m28rr\" (UID: \"4198fdf7-f7df-4dc0-99e0-92b0c4874587\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m28rr" Jul 13 07:02:07.715449 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.714752 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4198fdf7-f7df-4dc0-99e0-92b0c4874587-etc-selinux\") pod \"aws-ebs-csi-driver-node-m28rr\" (UID: \"4198fdf7-f7df-4dc0-99e0-92b0c4874587\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m28rr" Jul 13 07:02:07.715449 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.714784 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e1b48537-330d-49c0-80be-80fe72862d9d-tmp\") pod \"tuned-6l8np\" (UID: \"e1b48537-330d-49c0-80be-80fe72862d9d\") " pod="openshift-cluster-node-tuning-operator/tuned-6l8np" Jul 13 07:02:07.715449 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.714809 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/89973bd4-81f7-4536-841e-1571b0fd9506-multus-cni-dir\") pod \"multus-g9z6n\" (UID: \"89973bd4-81f7-4536-841e-1571b0fd9506\") " pod="openshift-multus/multus-g9z6n" Jul 13 07:02:07.715449 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.714832 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/89973bd4-81f7-4536-841e-1571b0fd9506-host-var-lib-cni-bin\") pod \"multus-g9z6n\" (UID: \"89973bd4-81f7-4536-841e-1571b0fd9506\") " pod="openshift-multus/multus-g9z6n" Jul 13 07:02:07.715449 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.714853 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4198fdf7-f7df-4dc0-99e0-92b0c4874587-etc-selinux\") pod \"aws-ebs-csi-driver-node-m28rr\" (UID: \"4198fdf7-f7df-4dc0-99e0-92b0c4874587\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m28rr" Jul 13 07:02:07.716250 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.714856 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/89973bd4-81f7-4536-841e-1571b0fd9506-etc-kubernetes\") pod \"multus-g9z6n\" (UID: \"89973bd4-81f7-4536-841e-1571b0fd9506\") " pod="openshift-multus/multus-g9z6n" Jul 13 07:02:07.716250 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.714895 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/75604003-f810-475e-b0db-af8180c49604-hosts-file\") pod \"node-resolver-g6kgx\" (UID: \"75604003-f810-475e-b0db-af8180c49604\") " pod="openshift-dns/node-resolver-g6kgx" Jul 13 07:02:07.716250 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.714921 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tns76\" (UniqueName: \"kubernetes.io/projected/75604003-f810-475e-b0db-af8180c49604-kube-api-access-tns76\") pod \"node-resolver-g6kgx\" (UID: \"75604003-f810-475e-b0db-af8180c49604\") " pod="openshift-dns/node-resolver-g6kgx" Jul 13 07:02:07.716250 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.714945 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3d71897a-4b2e-4e4e-9525-ccd89bd18458-host-run-netns\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.716250 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.714971 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2d5a84b8-155e-4dc0-8526-19a67235d08f-system-cni-dir\") pod \"multus-additional-cni-plugins-56rlp\" (UID: \"2d5a84b8-155e-4dc0-8526-19a67235d08f\") " pod="openshift-multus/multus-additional-cni-plugins-56rlp" Jul 13 07:02:07.716250 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.715006 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e1b48537-330d-49c0-80be-80fe72862d9d-etc-tuned\") pod \"tuned-6l8np\" (UID: \"e1b48537-330d-49c0-80be-80fe72862d9d\") " pod="openshift-cluster-node-tuning-operator/tuned-6l8np" Jul 13 07:02:07.716250 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.715029 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/be89589f-0a40-4e74-9ecf-aeac7e231126-serviceca\") pod \"node-ca-vxcrv\" (UID: \"be89589f-0a40-4e74-9ecf-aeac7e231126\") " pod="openshift-image-registry/node-ca-vxcrv" Jul 13 07:02:07.716250 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.715052 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75604003-f810-475e-b0db-af8180c49604-tmp-dir\") pod \"node-resolver-g6kgx\" (UID: \"75604003-f810-475e-b0db-af8180c49604\") " pod="openshift-dns/node-resolver-g6kgx" Jul 13 07:02:07.716250 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.715076 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3d71897a-4b2e-4e4e-9525-ccd89bd18458-etc-openvswitch\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.716250 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.715098 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3d71897a-4b2e-4e4e-9525-ccd89bd18458-log-socket\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.716250 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.715052 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2d5a84b8-155e-4dc0-8526-19a67235d08f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-56rlp\" (UID: \"2d5a84b8-155e-4dc0-8526-19a67235d08f\") " pod="openshift-multus/multus-additional-cni-plugins-56rlp" Jul 13 07:02:07.716250 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.715125 2586 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 13 07:02:07.716250 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.715149 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3d71897a-4b2e-4e4e-9525-ccd89bd18458-host-run-ovn-kubernetes\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.716250 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.715177 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2d5a84b8-155e-4dc0-8526-19a67235d08f-system-cni-dir\") pod \"multus-additional-cni-plugins-56rlp\" (UID: \"2d5a84b8-155e-4dc0-8526-19a67235d08f\") " pod="openshift-multus/multus-additional-cni-plugins-56rlp" Jul 13 07:02:07.716250 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.715185 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2abf1e96-1ba2-4cc1-a632-48e28b374830-agent-certs\") pod \"konnectivity-agent-xs7q5\" (UID: \"2abf1e96-1ba2-4cc1-a632-48e28b374830\") " pod="kube-system/konnectivity-agent-xs7q5" Jul 13 07:02:07.716250 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.715214 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2abf1e96-1ba2-4cc1-a632-48e28b374830-konnectivity-ca\") pod \"konnectivity-agent-xs7q5\" (UID: \"2abf1e96-1ba2-4cc1-a632-48e28b374830\") " pod="kube-system/konnectivity-agent-xs7q5" Jul 13 07:02:07.716250 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.715239 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d7d6c54b-cee3-4b7c-9b72-2afe08125102-host-slash\") pod \"iptables-alerter-z7jkb\" (UID: \"d7d6c54b-cee3-4b7c-9b72-2afe08125102\") " pod="openshift-network-operator/iptables-alerter-z7jkb" Jul 13 07:02:07.717051 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.715509 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9dfz\" (UniqueName: \"kubernetes.io/projected/e1b48537-330d-49c0-80be-80fe72862d9d-kube-api-access-k9dfz\") pod \"tuned-6l8np\" (UID: \"e1b48537-330d-49c0-80be-80fe72862d9d\") " pod="openshift-cluster-node-tuning-operator/tuned-6l8np" Jul 13 07:02:07.717051 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.715541 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnxn5\" (UniqueName: \"kubernetes.io/projected/be89589f-0a40-4e74-9ecf-aeac7e231126-kube-api-access-wnxn5\") pod \"node-ca-vxcrv\" (UID: \"be89589f-0a40-4e74-9ecf-aeac7e231126\") " pod="openshift-image-registry/node-ca-vxcrv" Jul 13 07:02:07.717051 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.715568 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/89973bd4-81f7-4536-841e-1571b0fd9506-multus-socket-dir-parent\") pod \"multus-g9z6n\" (UID: \"89973bd4-81f7-4536-841e-1571b0fd9506\") " pod="openshift-multus/multus-g9z6n" Jul 13 07:02:07.717051 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.715591 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/89973bd4-81f7-4536-841e-1571b0fd9506-host-var-lib-kubelet\") pod \"multus-g9z6n\" (UID: \"89973bd4-81f7-4536-841e-1571b0fd9506\") " pod="openshift-multus/multus-g9z6n" Jul 13 07:02:07.717051 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.715634 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98fzf\" (UniqueName: \"kubernetes.io/projected/2d5a84b8-155e-4dc0-8526-19a67235d08f-kube-api-access-98fzf\") pod \"multus-additional-cni-plugins-56rlp\" (UID: \"2d5a84b8-155e-4dc0-8526-19a67235d08f\") " pod="openshift-multus/multus-additional-cni-plugins-56rlp" Jul 13 07:02:07.717051 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.715658 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e1b48537-330d-49c0-80be-80fe72862d9d-etc-sysctl-d\") pod \"tuned-6l8np\" (UID: \"e1b48537-330d-49c0-80be-80fe72862d9d\") " pod="openshift-cluster-node-tuning-operator/tuned-6l8np" Jul 13 07:02:07.717051 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.715696 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e1b48537-330d-49c0-80be-80fe72862d9d-lib-modules\") pod \"tuned-6l8np\" (UID: \"e1b48537-330d-49c0-80be-80fe72862d9d\") " pod="openshift-cluster-node-tuning-operator/tuned-6l8np" Jul 13 07:02:07.717051 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.715722 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e1b48537-330d-49c0-80be-80fe72862d9d-var-lib-kubelet\") pod \"tuned-6l8np\" (UID: \"e1b48537-330d-49c0-80be-80fe72862d9d\") " pod="openshift-cluster-node-tuning-operator/tuned-6l8np" Jul 13 07:02:07.717051 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.715748 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3d71897a-4b2e-4e4e-9525-ccd89bd18458-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.717051 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.715741 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2abf1e96-1ba2-4cc1-a632-48e28b374830-konnectivity-ca\") pod \"konnectivity-agent-xs7q5\" (UID: \"2abf1e96-1ba2-4cc1-a632-48e28b374830\") " pod="kube-system/konnectivity-agent-xs7q5" Jul 13 07:02:07.717051 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.715778 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6hjd\" (UniqueName: \"kubernetes.io/projected/3d71897a-4b2e-4e4e-9525-ccd89bd18458-kube-api-access-c6hjd\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.717051 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.715820 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c3724c5e-1a2b-410b-a745-b0f90bf27358-kubelet-config\") pod \"global-pull-secret-syncer-lb8th\" (UID: \"c3724c5e-1a2b-410b-a745-b0f90bf27358\") " pod="kube-system/global-pull-secret-syncer-lb8th" Jul 13 07:02:07.717051 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.715845 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e1b48537-330d-49c0-80be-80fe72862d9d-etc-modprobe-d\") pod \"tuned-6l8np\" (UID: \"e1b48537-330d-49c0-80be-80fe72862d9d\") " pod="openshift-cluster-node-tuning-operator/tuned-6l8np" Jul 13 07:02:07.717051 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.715868 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e1b48537-330d-49c0-80be-80fe72862d9d-sys\") pod \"tuned-6l8np\" (UID: \"e1b48537-330d-49c0-80be-80fe72862d9d\") " pod="openshift-cluster-node-tuning-operator/tuned-6l8np" Jul 13 07:02:07.717051 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.715875 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d7d6c54b-cee3-4b7c-9b72-2afe08125102-host-slash\") pod \"iptables-alerter-z7jkb\" (UID: \"d7d6c54b-cee3-4b7c-9b72-2afe08125102\") " pod="openshift-network-operator/iptables-alerter-z7jkb" Jul 13 07:02:07.717051 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.715893 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/89973bd4-81f7-4536-841e-1571b0fd9506-cnibin\") pod \"multus-g9z6n\" (UID: \"89973bd4-81f7-4536-841e-1571b0fd9506\") " pod="openshift-multus/multus-g9z6n" Jul 13 07:02:07.717051 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.715929 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/89973bd4-81f7-4536-841e-1571b0fd9506-os-release\") pod \"multus-g9z6n\" (UID: \"89973bd4-81f7-4536-841e-1571b0fd9506\") " pod="openshift-multus/multus-g9z6n" Jul 13 07:02:07.717833 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.715955 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/89973bd4-81f7-4536-841e-1571b0fd9506-cni-binary-copy\") pod \"multus-g9z6n\" (UID: \"89973bd4-81f7-4536-841e-1571b0fd9506\") " pod="openshift-multus/multus-g9z6n" Jul 13 07:02:07.717833 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.715983 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3d71897a-4b2e-4e4e-9525-ccd89bd18458-run-ovn\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.717833 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.716010 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e1b48537-330d-49c0-80be-80fe72862d9d-host\") pod \"tuned-6l8np\" (UID: \"e1b48537-330d-49c0-80be-80fe72862d9d\") " pod="openshift-cluster-node-tuning-operator/tuned-6l8np" Jul 13 07:02:07.717833 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.716064 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4198fdf7-f7df-4dc0-99e0-92b0c4874587-device-dir\") pod \"aws-ebs-csi-driver-node-m28rr\" (UID: \"4198fdf7-f7df-4dc0-99e0-92b0c4874587\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m28rr" Jul 13 07:02:07.717833 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.716090 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2d5a84b8-155e-4dc0-8526-19a67235d08f-os-release\") pod \"multus-additional-cni-plugins-56rlp\" (UID: \"2d5a84b8-155e-4dc0-8526-19a67235d08f\") " pod="openshift-multus/multus-additional-cni-plugins-56rlp" Jul 13 07:02:07.717833 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.716117 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2d5a84b8-155e-4dc0-8526-19a67235d08f-cni-binary-copy\") pod \"multus-additional-cni-plugins-56rlp\" (UID: \"2d5a84b8-155e-4dc0-8526-19a67235d08f\") " pod="openshift-multus/multus-additional-cni-plugins-56rlp" Jul 13 07:02:07.717833 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.716135 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c3724c5e-1a2b-410b-a745-b0f90bf27358-kubelet-config\") pod \"global-pull-secret-syncer-lb8th\" (UID: \"c3724c5e-1a2b-410b-a745-b0f90bf27358\") " pod="kube-system/global-pull-secret-syncer-lb8th" Jul 13 07:02:07.717833 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.716151 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxcr7\" (UniqueName: \"kubernetes.io/projected/4198fdf7-f7df-4dc0-99e0-92b0c4874587-kube-api-access-rxcr7\") pod \"aws-ebs-csi-driver-node-m28rr\" (UID: \"4198fdf7-f7df-4dc0-99e0-92b0c4874587\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m28rr" Jul 13 07:02:07.717833 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.716232 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e1b48537-330d-49c0-80be-80fe72862d9d-etc-modprobe-d\") pod \"tuned-6l8np\" (UID: \"e1b48537-330d-49c0-80be-80fe72862d9d\") " pod="openshift-cluster-node-tuning-operator/tuned-6l8np" Jul 13 07:02:07.717833 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.716278 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e1b48537-330d-49c0-80be-80fe72862d9d-sys\") pod \"tuned-6l8np\" (UID: \"e1b48537-330d-49c0-80be-80fe72862d9d\") " pod="openshift-cluster-node-tuning-operator/tuned-6l8np" Jul 13 07:02:07.717833 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.716364 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e1b48537-330d-49c0-80be-80fe72862d9d-etc-sysctl-d\") pod \"tuned-6l8np\" (UID: \"e1b48537-330d-49c0-80be-80fe72862d9d\") " pod="openshift-cluster-node-tuning-operator/tuned-6l8np" Jul 13 07:02:07.717833 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.716403 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e1b48537-330d-49c0-80be-80fe72862d9d-host\") pod \"tuned-6l8np\" (UID: \"e1b48537-330d-49c0-80be-80fe72862d9d\") " pod="openshift-cluster-node-tuning-operator/tuned-6l8np" Jul 13 07:02:07.717833 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.716477 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e1b48537-330d-49c0-80be-80fe72862d9d-lib-modules\") pod \"tuned-6l8np\" (UID: \"e1b48537-330d-49c0-80be-80fe72862d9d\") " pod="openshift-cluster-node-tuning-operator/tuned-6l8np" Jul 13 07:02:07.717833 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.716528 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e1b48537-330d-49c0-80be-80fe72862d9d-var-lib-kubelet\") pod \"tuned-6l8np\" (UID: \"e1b48537-330d-49c0-80be-80fe72862d9d\") " pod="openshift-cluster-node-tuning-operator/tuned-6l8np" Jul 13 07:02:07.717833 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.716582 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4198fdf7-f7df-4dc0-99e0-92b0c4874587-device-dir\") pod \"aws-ebs-csi-driver-node-m28rr\" (UID: \"4198fdf7-f7df-4dc0-99e0-92b0c4874587\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m28rr" Jul 13 07:02:07.717833 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.716641 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2d5a84b8-155e-4dc0-8526-19a67235d08f-os-release\") pod \"multus-additional-cni-plugins-56rlp\" (UID: \"2d5a84b8-155e-4dc0-8526-19a67235d08f\") " pod="openshift-multus/multus-additional-cni-plugins-56rlp" Jul 13 07:02:07.717833 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.717179 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2d5a84b8-155e-4dc0-8526-19a67235d08f-cni-binary-copy\") pod \"multus-additional-cni-plugins-56rlp\" (UID: \"2d5a84b8-155e-4dc0-8526-19a67235d08f\") " pod="openshift-multus/multus-additional-cni-plugins-56rlp" Jul 13 07:02:07.720377 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:07.720355 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 07:02:07.720485 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:07.720382 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 07:02:07.720485 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:07.720396 2586 projected.go:196] Error preparing data for projected volume kube-api-access-ldwqr for pod openshift-network-diagnostics/network-check-target-6xdjl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 07:02:07.720485 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:07.720450 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ade3045-6b94-41a9-baeb-db7ce7a84c7a-kube-api-access-ldwqr podName:0ade3045-6b94-41a9-baeb-db7ce7a84c7a nodeName:}" failed. No retries permitted until 2026-07-13 07:02:08.220434014 +0000 UTC m=+3.055997828 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-ldwqr" (UniqueName: "kubernetes.io/projected/0ade3045-6b94-41a9-baeb-db7ce7a84c7a-kube-api-access-ldwqr") pod "network-check-target-6xdjl" (UID: "0ade3045-6b94-41a9-baeb-db7ce7a84c7a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 07:02:07.722127 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.721562 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e1b48537-330d-49c0-80be-80fe72862d9d-tmp\") pod \"tuned-6l8np\" (UID: \"e1b48537-330d-49c0-80be-80fe72862d9d\") " pod="openshift-cluster-node-tuning-operator/tuned-6l8np" Jul 13 07:02:07.722127 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.722087 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e1b48537-330d-49c0-80be-80fe72862d9d-etc-tuned\") pod \"tuned-6l8np\" (UID: \"e1b48537-330d-49c0-80be-80fe72862d9d\") " pod="openshift-cluster-node-tuning-operator/tuned-6l8np" Jul 13 07:02:07.723322 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.723299 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2abf1e96-1ba2-4cc1-a632-48e28b374830-agent-certs\") pod \"konnectivity-agent-xs7q5\" (UID: \"2abf1e96-1ba2-4cc1-a632-48e28b374830\") " pod="kube-system/konnectivity-agent-xs7q5" Jul 13 07:02:07.723423 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.723321 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4h5nt\" (UniqueName: \"kubernetes.io/projected/d7d6c54b-cee3-4b7c-9b72-2afe08125102-kube-api-access-4h5nt\") pod \"iptables-alerter-z7jkb\" (UID: \"d7d6c54b-cee3-4b7c-9b72-2afe08125102\") " pod="openshift-network-operator/iptables-alerter-z7jkb" Jul 13 07:02:07.724423 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.724390 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhst6\" (UniqueName: \"kubernetes.io/projected/638764ba-e970-42f1-83ec-cf5c7075d61e-kube-api-access-zhst6\") pod \"network-metrics-daemon-znhkv\" (UID: \"638764ba-e970-42f1-83ec-cf5c7075d61e\") " pod="openshift-multus/network-metrics-daemon-znhkv" Jul 13 07:02:07.725486 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.725465 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxcr7\" (UniqueName: \"kubernetes.io/projected/4198fdf7-f7df-4dc0-99e0-92b0c4874587-kube-api-access-rxcr7\") pod \"aws-ebs-csi-driver-node-m28rr\" (UID: \"4198fdf7-f7df-4dc0-99e0-92b0c4874587\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m28rr" Jul 13 07:02:07.725572 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.725503 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98fzf\" (UniqueName: \"kubernetes.io/projected/2d5a84b8-155e-4dc0-8526-19a67235d08f-kube-api-access-98fzf\") pod \"multus-additional-cni-plugins-56rlp\" (UID: \"2d5a84b8-155e-4dc0-8526-19a67235d08f\") " pod="openshift-multus/multus-additional-cni-plugins-56rlp" Jul 13 07:02:07.726864 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.726844 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9dfz\" (UniqueName: \"kubernetes.io/projected/e1b48537-330d-49c0-80be-80fe72862d9d-kube-api-access-k9dfz\") pod \"tuned-6l8np\" (UID: \"e1b48537-330d-49c0-80be-80fe72862d9d\") " pod="openshift-cluster-node-tuning-operator/tuned-6l8np" Jul 13 07:02:07.810300 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.810242 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-37.ec2.internal" event={"ID":"0de86516d8a93e213ba2d06529343779","Type":"ContainerStarted","Data":"5c4889cbfb8955532686a7795b2b6c12a4ac792ddfe083c2c8fbc6f72de2ebe6"} Jul 13 07:02:07.811943 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.811911 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-37.ec2.internal" event={"ID":"6df327e3e80f10552c93b38d2fd449c4","Type":"ContainerStarted","Data":"69c8665a96052afd3cbb43be0af397f30bac4da0b8db2423f07c75ae464ec37c"} Jul 13 07:02:07.817322 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.817297 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/89973bd4-81f7-4536-841e-1571b0fd9506-multus-daemon-config\") pod \"multus-g9z6n\" (UID: \"89973bd4-81f7-4536-841e-1571b0fd9506\") " pod="openshift-multus/multus-g9z6n" Jul 13 07:02:07.817449 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.817336 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/be89589f-0a40-4e74-9ecf-aeac7e231126-host\") pod \"node-ca-vxcrv\" (UID: \"be89589f-0a40-4e74-9ecf-aeac7e231126\") " pod="openshift-image-registry/node-ca-vxcrv" Jul 13 07:02:07.817449 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.817363 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/89973bd4-81f7-4536-841e-1571b0fd9506-host-run-netns\") pod \"multus-g9z6n\" (UID: \"89973bd4-81f7-4536-841e-1571b0fd9506\") " pod="openshift-multus/multus-g9z6n" Jul 13 07:02:07.817449 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.817387 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3d71897a-4b2e-4e4e-9525-ccd89bd18458-var-lib-openvswitch\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.817449 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.817409 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3d71897a-4b2e-4e4e-9525-ccd89bd18458-host-slash\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.817449 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.817431 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/89973bd4-81f7-4536-841e-1571b0fd9506-host-var-lib-cni-multus\") pod \"multus-g9z6n\" (UID: \"89973bd4-81f7-4536-841e-1571b0fd9506\") " pod="openshift-multus/multus-g9z6n" Jul 13 07:02:07.817716 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.817453 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/89973bd4-81f7-4536-841e-1571b0fd9506-hostroot\") pod \"multus-g9z6n\" (UID: \"89973bd4-81f7-4536-841e-1571b0fd9506\") " pod="openshift-multus/multus-g9z6n" Jul 13 07:02:07.817716 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.817476 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3d71897a-4b2e-4e4e-9525-ccd89bd18458-run-systemd\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.817716 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.817495 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3d71897a-4b2e-4e4e-9525-ccd89bd18458-node-log\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.817716 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.817511 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3d71897a-4b2e-4e4e-9525-ccd89bd18458-ovnkube-script-lib\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.817716 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.817552 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/89973bd4-81f7-4536-841e-1571b0fd9506-system-cni-dir\") pod \"multus-g9z6n\" (UID: \"89973bd4-81f7-4536-841e-1571b0fd9506\") " pod="openshift-multus/multus-g9z6n" Jul 13 07:02:07.817716 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.817569 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/89973bd4-81f7-4536-841e-1571b0fd9506-host-run-multus-certs\") pod \"multus-g9z6n\" (UID: \"89973bd4-81f7-4536-841e-1571b0fd9506\") " pod="openshift-multus/multus-g9z6n" Jul 13 07:02:07.817716 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.817592 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3d71897a-4b2e-4e4e-9525-ccd89bd18458-ovnkube-config\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.817716 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.817615 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3d71897a-4b2e-4e4e-9525-ccd89bd18458-ovn-node-metrics-cert\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.817716 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.817659 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3d71897a-4b2e-4e4e-9525-ccd89bd18458-host-kubelet\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.817716 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.817697 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3d71897a-4b2e-4e4e-9525-ccd89bd18458-systemd-units\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.818177 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.817724 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3d71897a-4b2e-4e4e-9525-ccd89bd18458-host-cni-netd\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.818177 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.817748 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/89973bd4-81f7-4536-841e-1571b0fd9506-host-run-k8s-cni-cncf-io\") pod \"multus-g9z6n\" (UID: \"89973bd4-81f7-4536-841e-1571b0fd9506\") " pod="openshift-multus/multus-g9z6n" Jul 13 07:02:07.818177 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.817771 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/89973bd4-81f7-4536-841e-1571b0fd9506-multus-conf-dir\") pod \"multus-g9z6n\" (UID: \"89973bd4-81f7-4536-841e-1571b0fd9506\") " pod="openshift-multus/multus-g9z6n" Jul 13 07:02:07.818177 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.817794 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3d71897a-4b2e-4e4e-9525-ccd89bd18458-run-openvswitch\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.818177 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.817817 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3d71897a-4b2e-4e4e-9525-ccd89bd18458-host-cni-bin\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.818177 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.817842 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3d71897a-4b2e-4e4e-9525-ccd89bd18458-env-overrides\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.818177 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.817875 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w99lf\" (UniqueName: \"kubernetes.io/projected/89973bd4-81f7-4536-841e-1571b0fd9506-kube-api-access-w99lf\") pod \"multus-g9z6n\" (UID: \"89973bd4-81f7-4536-841e-1571b0fd9506\") " pod="openshift-multus/multus-g9z6n" Jul 13 07:02:07.818177 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.817901 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/89973bd4-81f7-4536-841e-1571b0fd9506-multus-cni-dir\") pod \"multus-g9z6n\" (UID: \"89973bd4-81f7-4536-841e-1571b0fd9506\") " pod="openshift-multus/multus-g9z6n" Jul 13 07:02:07.818177 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.817923 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/89973bd4-81f7-4536-841e-1571b0fd9506-host-var-lib-cni-bin\") pod \"multus-g9z6n\" (UID: \"89973bd4-81f7-4536-841e-1571b0fd9506\") " pod="openshift-multus/multus-g9z6n" Jul 13 07:02:07.818177 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.817935 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/89973bd4-81f7-4536-841e-1571b0fd9506-system-cni-dir\") pod \"multus-g9z6n\" (UID: \"89973bd4-81f7-4536-841e-1571b0fd9506\") " pod="openshift-multus/multus-g9z6n" Jul 13 07:02:07.818177 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.817948 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/89973bd4-81f7-4536-841e-1571b0fd9506-etc-kubernetes\") pod \"multus-g9z6n\" (UID: \"89973bd4-81f7-4536-841e-1571b0fd9506\") " pod="openshift-multus/multus-g9z6n" Jul 13 07:02:07.818177 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.817979 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/75604003-f810-475e-b0db-af8180c49604-hosts-file\") pod \"node-resolver-g6kgx\" (UID: \"75604003-f810-475e-b0db-af8180c49604\") " pod="openshift-dns/node-resolver-g6kgx" Jul 13 07:02:07.818177 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.817993 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/be89589f-0a40-4e74-9ecf-aeac7e231126-host\") pod \"node-ca-vxcrv\" (UID: \"be89589f-0a40-4e74-9ecf-aeac7e231126\") " pod="openshift-image-registry/node-ca-vxcrv" Jul 13 07:02:07.818177 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.818002 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tns76\" (UniqueName: \"kubernetes.io/projected/75604003-f810-475e-b0db-af8180c49604-kube-api-access-tns76\") pod \"node-resolver-g6kgx\" (UID: \"75604003-f810-475e-b0db-af8180c49604\") " pod="openshift-dns/node-resolver-g6kgx" Jul 13 07:02:07.818177 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.818027 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3d71897a-4b2e-4e4e-9525-ccd89bd18458-host-run-netns\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.818177 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.818032 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/89973bd4-81f7-4536-841e-1571b0fd9506-host-run-netns\") pod \"multus-g9z6n\" (UID: \"89973bd4-81f7-4536-841e-1571b0fd9506\") " pod="openshift-multus/multus-g9z6n" Jul 13 07:02:07.818177 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.818073 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3d71897a-4b2e-4e4e-9525-ccd89bd18458-var-lib-openvswitch\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.818177 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.818090 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3d71897a-4b2e-4e4e-9525-ccd89bd18458-host-run-netns\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.819069 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.818102 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/be89589f-0a40-4e74-9ecf-aeac7e231126-serviceca\") pod \"node-ca-vxcrv\" (UID: \"be89589f-0a40-4e74-9ecf-aeac7e231126\") " pod="openshift-image-registry/node-ca-vxcrv" Jul 13 07:02:07.819069 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.818130 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75604003-f810-475e-b0db-af8180c49604-tmp-dir\") pod \"node-resolver-g6kgx\" (UID: \"75604003-f810-475e-b0db-af8180c49604\") " pod="openshift-dns/node-resolver-g6kgx" Jul 13 07:02:07.819069 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.818138 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/89973bd4-81f7-4536-841e-1571b0fd9506-host-run-multus-certs\") pod \"multus-g9z6n\" (UID: \"89973bd4-81f7-4536-841e-1571b0fd9506\") " pod="openshift-multus/multus-g9z6n" Jul 13 07:02:07.819069 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.818154 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3d71897a-4b2e-4e4e-9525-ccd89bd18458-etc-openvswitch\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.819069 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.818178 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3d71897a-4b2e-4e4e-9525-ccd89bd18458-log-socket\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.819069 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.818204 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3d71897a-4b2e-4e4e-9525-ccd89bd18458-host-run-ovn-kubernetes\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.819069 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.818236 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnxn5\" (UniqueName: \"kubernetes.io/projected/be89589f-0a40-4e74-9ecf-aeac7e231126-kube-api-access-wnxn5\") pod \"node-ca-vxcrv\" (UID: \"be89589f-0a40-4e74-9ecf-aeac7e231126\") " pod="openshift-image-registry/node-ca-vxcrv" Jul 13 07:02:07.819069 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.818264 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/89973bd4-81f7-4536-841e-1571b0fd9506-multus-socket-dir-parent\") pod \"multus-g9z6n\" (UID: \"89973bd4-81f7-4536-841e-1571b0fd9506\") " pod="openshift-multus/multus-g9z6n" Jul 13 07:02:07.819069 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.818287 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/89973bd4-81f7-4536-841e-1571b0fd9506-host-var-lib-kubelet\") pod \"multus-g9z6n\" (UID: \"89973bd4-81f7-4536-841e-1571b0fd9506\") " pod="openshift-multus/multus-g9z6n" Jul 13 07:02:07.819069 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.818317 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3d71897a-4b2e-4e4e-9525-ccd89bd18458-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.819069 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.818343 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6hjd\" (UniqueName: \"kubernetes.io/projected/3d71897a-4b2e-4e4e-9525-ccd89bd18458-kube-api-access-c6hjd\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.819069 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.818662 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3d71897a-4b2e-4e4e-9525-ccd89bd18458-ovnkube-config\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.819069 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.818774 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/be89589f-0a40-4e74-9ecf-aeac7e231126-serviceca\") pod \"node-ca-vxcrv\" (UID: \"be89589f-0a40-4e74-9ecf-aeac7e231126\") " pod="openshift-image-registry/node-ca-vxcrv" Jul 13 07:02:07.819069 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.818819 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/89973bd4-81f7-4536-841e-1571b0fd9506-cnibin\") pod \"multus-g9z6n\" (UID: \"89973bd4-81f7-4536-841e-1571b0fd9506\") " pod="openshift-multus/multus-g9z6n" Jul 13 07:02:07.819069 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.818847 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/89973bd4-81f7-4536-841e-1571b0fd9506-os-release\") pod \"multus-g9z6n\" (UID: \"89973bd4-81f7-4536-841e-1571b0fd9506\") " pod="openshift-multus/multus-g9z6n" Jul 13 07:02:07.819069 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.818866 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75604003-f810-475e-b0db-af8180c49604-tmp-dir\") pod \"node-resolver-g6kgx\" (UID: \"75604003-f810-475e-b0db-af8180c49604\") " pod="openshift-dns/node-resolver-g6kgx" Jul 13 07:02:07.819069 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.818874 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/89973bd4-81f7-4536-841e-1571b0fd9506-cni-binary-copy\") pod \"multus-g9z6n\" (UID: \"89973bd4-81f7-4536-841e-1571b0fd9506\") " pod="openshift-multus/multus-g9z6n" Jul 13 07:02:07.819069 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.818901 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3d71897a-4b2e-4e4e-9525-ccd89bd18458-run-ovn\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.819871 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.818918 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3d71897a-4b2e-4e4e-9525-ccd89bd18458-log-socket\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.819871 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.818978 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3d71897a-4b2e-4e4e-9525-ccd89bd18458-host-slash\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.819871 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.818975 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3d71897a-4b2e-4e4e-9525-ccd89bd18458-host-cni-bin\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.819871 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.819014 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3d71897a-4b2e-4e4e-9525-ccd89bd18458-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.819871 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.819024 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/89973bd4-81f7-4536-841e-1571b0fd9506-host-var-lib-cni-multus\") pod \"multus-g9z6n\" (UID: \"89973bd4-81f7-4536-841e-1571b0fd9506\") " pod="openshift-multus/multus-g9z6n" Jul 13 07:02:07.819871 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.819060 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/89973bd4-81f7-4536-841e-1571b0fd9506-hostroot\") pod \"multus-g9z6n\" (UID: \"89973bd4-81f7-4536-841e-1571b0fd9506\") " pod="openshift-multus/multus-g9z6n" Jul 13 07:02:07.819871 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.819065 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3d71897a-4b2e-4e4e-9525-ccd89bd18458-systemd-units\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.819871 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.819093 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3d71897a-4b2e-4e4e-9525-ccd89bd18458-host-kubelet\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.819871 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.819107 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3d71897a-4b2e-4e4e-9525-ccd89bd18458-host-cni-netd\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.819871 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.819127 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3d71897a-4b2e-4e4e-9525-ccd89bd18458-run-systemd\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.819871 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.819144 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/89973bd4-81f7-4536-841e-1571b0fd9506-host-run-k8s-cni-cncf-io\") pod \"multus-g9z6n\" (UID: \"89973bd4-81f7-4536-841e-1571b0fd9506\") " pod="openshift-multus/multus-g9z6n" Jul 13 07:02:07.819871 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.819161 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3d71897a-4b2e-4e4e-9525-ccd89bd18458-node-log\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.819871 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.819193 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3d71897a-4b2e-4e4e-9525-ccd89bd18458-etc-openvswitch\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.819871 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.819196 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3d71897a-4b2e-4e4e-9525-ccd89bd18458-run-ovn\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.819871 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.819223 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3d71897a-4b2e-4e4e-9525-ccd89bd18458-host-run-ovn-kubernetes\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.819871 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.819443 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3d71897a-4b2e-4e4e-9525-ccd89bd18458-env-overrides\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.819871 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.819456 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/89973bd4-81f7-4536-841e-1571b0fd9506-multus-socket-dir-parent\") pod \"multus-g9z6n\" (UID: \"89973bd4-81f7-4536-841e-1571b0fd9506\") " pod="openshift-multus/multus-g9z6n" Jul 13 07:02:07.819871 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.819494 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/89973bd4-81f7-4536-841e-1571b0fd9506-host-var-lib-kubelet\") pod \"multus-g9z6n\" (UID: \"89973bd4-81f7-4536-841e-1571b0fd9506\") " pod="openshift-multus/multus-g9z6n" Jul 13 07:02:07.820739 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.819659 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/89973bd4-81f7-4536-841e-1571b0fd9506-multus-conf-dir\") pod \"multus-g9z6n\" (UID: \"89973bd4-81f7-4536-841e-1571b0fd9506\") " pod="openshift-multus/multus-g9z6n" Jul 13 07:02:07.820739 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.819660 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3d71897a-4b2e-4e4e-9525-ccd89bd18458-ovnkube-script-lib\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.820739 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.819708 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/89973bd4-81f7-4536-841e-1571b0fd9506-etc-kubernetes\") pod \"multus-g9z6n\" (UID: \"89973bd4-81f7-4536-841e-1571b0fd9506\") " pod="openshift-multus/multus-g9z6n" Jul 13 07:02:07.820739 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.819752 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/75604003-f810-475e-b0db-af8180c49604-hosts-file\") pod \"node-resolver-g6kgx\" (UID: \"75604003-f810-475e-b0db-af8180c49604\") " pod="openshift-dns/node-resolver-g6kgx" Jul 13 07:02:07.820739 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.819802 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/89973bd4-81f7-4536-841e-1571b0fd9506-multus-cni-dir\") pod \"multus-g9z6n\" (UID: \"89973bd4-81f7-4536-841e-1571b0fd9506\") " pod="openshift-multus/multus-g9z6n" Jul 13 07:02:07.820739 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.819836 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/89973bd4-81f7-4536-841e-1571b0fd9506-host-var-lib-cni-bin\") pod \"multus-g9z6n\" (UID: \"89973bd4-81f7-4536-841e-1571b0fd9506\") " pod="openshift-multus/multus-g9z6n" Jul 13 07:02:07.820739 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.819866 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3d71897a-4b2e-4e4e-9525-ccd89bd18458-run-openvswitch\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.820739 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.819907 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/89973bd4-81f7-4536-841e-1571b0fd9506-cnibin\") pod \"multus-g9z6n\" (UID: \"89973bd4-81f7-4536-841e-1571b0fd9506\") " pod="openshift-multus/multus-g9z6n" Jul 13 07:02:07.820739 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.819916 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/89973bd4-81f7-4536-841e-1571b0fd9506-os-release\") pod \"multus-g9z6n\" (UID: \"89973bd4-81f7-4536-841e-1571b0fd9506\") " pod="openshift-multus/multus-g9z6n" Jul 13 07:02:07.820739 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.820065 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/89973bd4-81f7-4536-841e-1571b0fd9506-cni-binary-copy\") pod \"multus-g9z6n\" (UID: \"89973bd4-81f7-4536-841e-1571b0fd9506\") " pod="openshift-multus/multus-g9z6n" Jul 13 07:02:07.820739 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.820103 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/89973bd4-81f7-4536-841e-1571b0fd9506-multus-daemon-config\") pod \"multus-g9z6n\" (UID: \"89973bd4-81f7-4536-841e-1571b0fd9506\") " pod="openshift-multus/multus-g9z6n" Jul 13 07:02:07.822299 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.822230 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3d71897a-4b2e-4e4e-9525-ccd89bd18458-ovn-node-metrics-cert\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.828636 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.828583 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnxn5\" (UniqueName: \"kubernetes.io/projected/be89589f-0a40-4e74-9ecf-aeac7e231126-kube-api-access-wnxn5\") pod \"node-ca-vxcrv\" (UID: \"be89589f-0a40-4e74-9ecf-aeac7e231126\") " pod="openshift-image-registry/node-ca-vxcrv" Jul 13 07:02:07.830481 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.830458 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6hjd\" (UniqueName: \"kubernetes.io/projected/3d71897a-4b2e-4e4e-9525-ccd89bd18458-kube-api-access-c6hjd\") pod \"ovnkube-node-7b964\" (UID: \"3d71897a-4b2e-4e4e-9525-ccd89bd18458\") " pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.831115 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.831092 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tns76\" (UniqueName: \"kubernetes.io/projected/75604003-f810-475e-b0db-af8180c49604-kube-api-access-tns76\") pod \"node-resolver-g6kgx\" (UID: \"75604003-f810-475e-b0db-af8180c49604\") " pod="openshift-dns/node-resolver-g6kgx" Jul 13 07:02:07.831194 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.831172 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w99lf\" (UniqueName: \"kubernetes.io/projected/89973bd4-81f7-4536-841e-1571b0fd9506-kube-api-access-w99lf\") pod \"multus-g9z6n\" (UID: \"89973bd4-81f7-4536-841e-1571b0fd9506\") " pod="openshift-multus/multus-g9z6n" Jul 13 07:02:07.902075 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.902043 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-z7jkb" Jul 13 07:02:07.911181 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:07.911155 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7d6c54b_cee3_4b7c_9b72_2afe08125102.slice/crio-ffb7cd0c87d86cebd4b1f98a1bd3d1daacd88be31f719fe1e3e8d0ce3669a1c6 WatchSource:0}: Error finding container ffb7cd0c87d86cebd4b1f98a1bd3d1daacd88be31f719fe1e3e8d0ce3669a1c6: Status 404 returned error can't find the container with id ffb7cd0c87d86cebd4b1f98a1bd3d1daacd88be31f719fe1e3e8d0ce3669a1c6 Jul 13 07:02:07.912189 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.912167 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m28rr" Jul 13 07:02:07.921050 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:07.921030 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4198fdf7_f7df_4dc0_99e0_92b0c4874587.slice/crio-a1e1a849e04f468ee189243170159671be6816f81656711610cfec43dc9be4e8 WatchSource:0}: Error finding container a1e1a849e04f468ee189243170159671be6816f81656711610cfec43dc9be4e8: Status 404 returned error can't find the container with id a1e1a849e04f468ee189243170159671be6816f81656711610cfec43dc9be4e8 Jul 13 07:02:07.928747 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.928729 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xs7q5" Jul 13 07:02:07.936622 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.936600 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6l8np" Jul 13 07:02:07.937371 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:07.937345 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2abf1e96_1ba2_4cc1_a632_48e28b374830.slice/crio-50022e16a80096efb722e90e750de348d8775803dacf0f8522737dc64ff65980 WatchSource:0}: Error finding container 50022e16a80096efb722e90e750de348d8775803dacf0f8522737dc64ff65980: Status 404 returned error can't find the container with id 50022e16a80096efb722e90e750de348d8775803dacf0f8522737dc64ff65980 Jul 13 07:02:07.942992 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.942966 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-56rlp" Jul 13 07:02:07.947038 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:07.947016 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1b48537_330d_49c0_80be_80fe72862d9d.slice/crio-49baee8d938afb8270c504a7c7e2ce7850aef283891fd70bbcfb8a2d3122fb01 WatchSource:0}: Error finding container 49baee8d938afb8270c504a7c7e2ce7850aef283891fd70bbcfb8a2d3122fb01: Status 404 returned error can't find the container with id 49baee8d938afb8270c504a7c7e2ce7850aef283891fd70bbcfb8a2d3122fb01 Jul 13 07:02:07.952486 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:07.952464 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d5a84b8_155e_4dc0_8526_19a67235d08f.slice/crio-3f2775af546e217a2b6e305f794751c27083f34636094cd1142e04b1202e4f7a WatchSource:0}: Error finding container 3f2775af546e217a2b6e305f794751c27083f34636094cd1142e04b1202e4f7a: Status 404 returned error can't find the container with id 3f2775af546e217a2b6e305f794751c27083f34636094cd1142e04b1202e4f7a Jul 13 07:02:07.952586 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.952521 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vxcrv" Jul 13 07:02:07.962017 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.961995 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-g9z6n" Jul 13 07:02:07.963066 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:07.963041 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe89589f_0a40_4e74_9ecf_aeac7e231126.slice/crio-01a8919bc727918761045a248788cd9245fb1c70430e9094d5e0a97840225662 WatchSource:0}: Error finding container 01a8919bc727918761045a248788cd9245fb1c70430e9094d5e0a97840225662: Status 404 returned error can't find the container with id 01a8919bc727918761045a248788cd9245fb1c70430e9094d5e0a97840225662 Jul 13 07:02:07.968472 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.968430 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-g6kgx" Jul 13 07:02:07.971571 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:07.971548 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89973bd4_81f7_4536_841e_1571b0fd9506.slice/crio-d99593ada88ffca738a777cfe060f51235426e981cc97bc8195d61af5c87ff6f WatchSource:0}: Error finding container d99593ada88ffca738a777cfe060f51235426e981cc97bc8195d61af5c87ff6f: Status 404 returned error can't find the container with id d99593ada88ffca738a777cfe060f51235426e981cc97bc8195d61af5c87ff6f Jul 13 07:02:07.972740 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:07.972721 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:07.983357 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:07.983331 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75604003_f810_475e_b0db_af8180c49604.slice/crio-7c3f12c4d9ef2967fa150eac3bbd4cce74aa9259f818c93843afef079ffafdd3 WatchSource:0}: Error finding container 7c3f12c4d9ef2967fa150eac3bbd4cce74aa9259f818c93843afef079ffafdd3: Status 404 returned error can't find the container with id 7c3f12c4d9ef2967fa150eac3bbd4cce74aa9259f818c93843afef079ffafdd3 Jul 13 07:02:07.984235 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:07.983957 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d71897a_4b2e_4e4e_9525_ccd89bd18458.slice/crio-d6c64b4730f8dfe606ee24e9cae03e7faf36e06a85a5aeacd1dc74da1cafd48a WatchSource:0}: Error finding container d6c64b4730f8dfe606ee24e9cae03e7faf36e06a85a5aeacd1dc74da1cafd48a: Status 404 returned error can't find the container with id d6c64b4730f8dfe606ee24e9cae03e7faf36e06a85a5aeacd1dc74da1cafd48a Jul 13 07:02:08.073893 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:08.073861 2586 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 07:02:08.222232 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:08.222197 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldwqr\" (UniqueName: \"kubernetes.io/projected/0ade3045-6b94-41a9-baeb-db7ce7a84c7a-kube-api-access-ldwqr\") pod \"network-check-target-6xdjl\" (UID: \"0ade3045-6b94-41a9-baeb-db7ce7a84c7a\") " pod="openshift-network-diagnostics/network-check-target-6xdjl" Jul 13 07:02:08.222412 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:08.222246 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/638764ba-e970-42f1-83ec-cf5c7075d61e-metrics-certs\") pod \"network-metrics-daemon-znhkv\" (UID: \"638764ba-e970-42f1-83ec-cf5c7075d61e\") " pod="openshift-multus/network-metrics-daemon-znhkv" Jul 13 07:02:08.222412 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:08.222278 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c3724c5e-1a2b-410b-a745-b0f90bf27358-original-pull-secret\") pod \"global-pull-secret-syncer-lb8th\" (UID: \"c3724c5e-1a2b-410b-a745-b0f90bf27358\") " pod="kube-system/global-pull-secret-syncer-lb8th" Jul 13 07:02:08.222412 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:08.222363 2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 07:02:08.222412 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:08.222387 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 07:02:08.222412 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:08.222408 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 07:02:08.222636 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:08.222424 2586 projected.go:196] Error preparing data for projected volume kube-api-access-ldwqr for pod openshift-network-diagnostics/network-check-target-6xdjl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 07:02:08.222636 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:08.222437 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/638764ba-e970-42f1-83ec-cf5c7075d61e-metrics-certs podName:638764ba-e970-42f1-83ec-cf5c7075d61e nodeName:}" failed. No retries permitted until 2026-07-13 07:02:09.222421586 +0000 UTC m=+4.057985394 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/638764ba-e970-42f1-83ec-cf5c7075d61e-metrics-certs") pod "network-metrics-daemon-znhkv" (UID: "638764ba-e970-42f1-83ec-cf5c7075d61e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 07:02:08.222636 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:08.222367 2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 07:02:08.222636 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:08.222474 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ade3045-6b94-41a9-baeb-db7ce7a84c7a-kube-api-access-ldwqr podName:0ade3045-6b94-41a9-baeb-db7ce7a84c7a nodeName:}" failed. No retries permitted until 2026-07-13 07:02:09.222458643 +0000 UTC m=+4.058022466 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-ldwqr" (UniqueName: "kubernetes.io/projected/0ade3045-6b94-41a9-baeb-db7ce7a84c7a-kube-api-access-ldwqr") pod "network-check-target-6xdjl" (UID: "0ade3045-6b94-41a9-baeb-db7ce7a84c7a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 07:02:08.222636 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:08.222542 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3724c5e-1a2b-410b-a745-b0f90bf27358-original-pull-secret podName:c3724c5e-1a2b-410b-a745-b0f90bf27358 nodeName:}" failed. No retries permitted until 2026-07-13 07:02:09.222521696 +0000 UTC m=+4.058085520 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c3724c5e-1a2b-410b-a745-b0f90bf27358-original-pull-secret") pod "global-pull-secret-syncer-lb8th" (UID: "c3724c5e-1a2b-410b-a745-b0f90bf27358") : object "kube-system"/"original-pull-secret" not registered Jul 13 07:02:08.648733 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:08.648631 2586 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 06:57:07 +0000 UTC" deadline="2028-03-07 15:10:01.407160991 +0000 UTC" Jul 13 07:02:08.648733 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:08.648671 2586 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14480h7m52.758494475s" Jul 13 07:02:08.799391 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:08.799352 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lb8th" Jul 13 07:02:08.799634 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:08.799481 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lb8th" podUID="c3724c5e-1a2b-410b-a745-b0f90bf27358" Jul 13 07:02:08.799900 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:08.799881 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-znhkv" Jul 13 07:02:08.800035 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:08.799985 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-znhkv" podUID="638764ba-e970-42f1-83ec-cf5c7075d61e" Jul 13 07:02:08.800097 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:08.800088 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6xdjl" Jul 13 07:02:08.800194 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:08.800160 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6xdjl" podUID="0ade3045-6b94-41a9-baeb-db7ce7a84c7a" Jul 13 07:02:08.817822 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:08.817788 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vxcrv" event={"ID":"be89589f-0a40-4e74-9ecf-aeac7e231126","Type":"ContainerStarted","Data":"01a8919bc727918761045a248788cd9245fb1c70430e9094d5e0a97840225662"} Jul 13 07:02:08.819342 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:08.819313 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-56rlp" event={"ID":"2d5a84b8-155e-4dc0-8526-19a67235d08f","Type":"ContainerStarted","Data":"3f2775af546e217a2b6e305f794751c27083f34636094cd1142e04b1202e4f7a"} Jul 13 07:02:08.821980 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:08.821955 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-z7jkb" event={"ID":"d7d6c54b-cee3-4b7c-9b72-2afe08125102","Type":"ContainerStarted","Data":"ffb7cd0c87d86cebd4b1f98a1bd3d1daacd88be31f719fe1e3e8d0ce3669a1c6"} Jul 13 07:02:08.823577 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:08.823553 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7b964" event={"ID":"3d71897a-4b2e-4e4e-9525-ccd89bd18458","Type":"ContainerStarted","Data":"d6c64b4730f8dfe606ee24e9cae03e7faf36e06a85a5aeacd1dc74da1cafd48a"} Jul 13 07:02:08.824939 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:08.824915 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-g6kgx" event={"ID":"75604003-f810-475e-b0db-af8180c49604","Type":"ContainerStarted","Data":"7c3f12c4d9ef2967fa150eac3bbd4cce74aa9259f818c93843afef079ffafdd3"} Jul 13 07:02:08.826757 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:08.826735 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-g9z6n" event={"ID":"89973bd4-81f7-4536-841e-1571b0fd9506","Type":"ContainerStarted","Data":"d99593ada88ffca738a777cfe060f51235426e981cc97bc8195d61af5c87ff6f"} Jul 13 07:02:08.828101 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:08.828076 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6l8np" event={"ID":"e1b48537-330d-49c0-80be-80fe72862d9d","Type":"ContainerStarted","Data":"49baee8d938afb8270c504a7c7e2ce7850aef283891fd70bbcfb8a2d3122fb01"} Jul 13 07:02:08.830262 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:08.830239 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xs7q5" event={"ID":"2abf1e96-1ba2-4cc1-a632-48e28b374830","Type":"ContainerStarted","Data":"50022e16a80096efb722e90e750de348d8775803dacf0f8522737dc64ff65980"} Jul 13 07:02:08.837754 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:08.837674 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m28rr" event={"ID":"4198fdf7-f7df-4dc0-99e0-92b0c4874587","Type":"ContainerStarted","Data":"a1e1a849e04f468ee189243170159671be6816f81656711610cfec43dc9be4e8"} Jul 13 07:02:09.228663 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:09.228604 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldwqr\" (UniqueName: \"kubernetes.io/projected/0ade3045-6b94-41a9-baeb-db7ce7a84c7a-kube-api-access-ldwqr\") pod \"network-check-target-6xdjl\" (UID: \"0ade3045-6b94-41a9-baeb-db7ce7a84c7a\") " pod="openshift-network-diagnostics/network-check-target-6xdjl" Jul 13 07:02:09.228886 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:09.228749 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/638764ba-e970-42f1-83ec-cf5c7075d61e-metrics-certs\") pod \"network-metrics-daemon-znhkv\" (UID: \"638764ba-e970-42f1-83ec-cf5c7075d61e\") " pod="openshift-multus/network-metrics-daemon-znhkv" Jul 13 07:02:09.228886 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:09.228783 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c3724c5e-1a2b-410b-a745-b0f90bf27358-original-pull-secret\") pod \"global-pull-secret-syncer-lb8th\" (UID: \"c3724c5e-1a2b-410b-a745-b0f90bf27358\") " pod="kube-system/global-pull-secret-syncer-lb8th" Jul 13 07:02:09.229003 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:09.228943 2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 07:02:09.229056 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:09.229006 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3724c5e-1a2b-410b-a745-b0f90bf27358-original-pull-secret podName:c3724c5e-1a2b-410b-a745-b0f90bf27358 nodeName:}" failed. No retries permitted until 2026-07-13 07:02:11.228986443 +0000 UTC m=+6.064550278 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c3724c5e-1a2b-410b-a745-b0f90bf27358-original-pull-secret") pod "global-pull-secret-syncer-lb8th" (UID: "c3724c5e-1a2b-410b-a745-b0f90bf27358") : object "kube-system"/"original-pull-secret" not registered Jul 13 07:02:09.229732 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:09.229433 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 07:02:09.229732 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:09.229454 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 07:02:09.229732 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:09.229466 2586 projected.go:196] Error preparing data for projected volume kube-api-access-ldwqr for pod openshift-network-diagnostics/network-check-target-6xdjl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 07:02:09.229732 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:09.229513 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ade3045-6b94-41a9-baeb-db7ce7a84c7a-kube-api-access-ldwqr podName:0ade3045-6b94-41a9-baeb-db7ce7a84c7a nodeName:}" failed. No retries permitted until 2026-07-13 07:02:11.229497665 +0000 UTC m=+6.065061490 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-ldwqr" (UniqueName: "kubernetes.io/projected/0ade3045-6b94-41a9-baeb-db7ce7a84c7a-kube-api-access-ldwqr") pod "network-check-target-6xdjl" (UID: "0ade3045-6b94-41a9-baeb-db7ce7a84c7a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 07:02:09.229732 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:09.229577 2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 07:02:09.229732 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:09.229610 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/638764ba-e970-42f1-83ec-cf5c7075d61e-metrics-certs podName:638764ba-e970-42f1-83ec-cf5c7075d61e nodeName:}" failed. No retries permitted until 2026-07-13 07:02:11.229599783 +0000 UTC m=+6.065163607 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/638764ba-e970-42f1-83ec-cf5c7075d61e-metrics-certs") pod "network-metrics-daemon-znhkv" (UID: "638764ba-e970-42f1-83ec-cf5c7075d61e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 07:02:09.649093 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:09.648965 2586 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 06:57:07 +0000 UTC" deadline="2028-01-30 22:55:21.191494557 +0000 UTC" Jul 13 07:02:09.649093 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:09.649037 2586 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13599h53m11.542461711s" Jul 13 07:02:10.799436 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:10.799164 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-znhkv" Jul 13 07:02:10.799916 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:10.799563 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-znhkv" podUID="638764ba-e970-42f1-83ec-cf5c7075d61e" Jul 13 07:02:10.799916 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:10.799402 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lb8th" Jul 13 07:02:10.799916 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:10.799660 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lb8th" podUID="c3724c5e-1a2b-410b-a745-b0f90bf27358" Jul 13 07:02:10.799916 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:10.799356 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6xdjl" Jul 13 07:02:10.799916 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:10.799748 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6xdjl" podUID="0ade3045-6b94-41a9-baeb-db7ce7a84c7a" Jul 13 07:02:10.854137 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:10.853291 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-37.ec2.internal" event={"ID":"6df327e3e80f10552c93b38d2fd449c4","Type":"ContainerStarted","Data":"6627d28b1788b5198196f35e1da2e6392500d9c3101fe314c56626acad8fb7e9"} Jul 13 07:02:10.873503 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:10.872860 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-37.ec2.internal" event={"ID":"0de86516d8a93e213ba2d06529343779","Type":"ContainerStarted","Data":"41352c23a750792ade24bcbfb7bd3a08c5adf1f19cab7bca98ca09d5bf1f1059"} Jul 13 07:02:10.888876 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:10.888231 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-37.ec2.internal" podStartSLOduration=4.888206382 podStartE2EDuration="4.888206382s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 07:02:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:02:10.869444306 +0000 UTC m=+5.705008138" watchObservedRunningTime="2026-07-13 07:02:10.888206382 +0000 UTC m=+5.723770213" Jul 13 07:02:11.247660 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:11.247621 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/638764ba-e970-42f1-83ec-cf5c7075d61e-metrics-certs\") pod \"network-metrics-daemon-znhkv\" (UID: \"638764ba-e970-42f1-83ec-cf5c7075d61e\") " pod="openshift-multus/network-metrics-daemon-znhkv" Jul 13 07:02:11.247851 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:11.247675 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c3724c5e-1a2b-410b-a745-b0f90bf27358-original-pull-secret\") pod \"global-pull-secret-syncer-lb8th\" (UID: \"c3724c5e-1a2b-410b-a745-b0f90bf27358\") " pod="kube-system/global-pull-secret-syncer-lb8th" Jul 13 07:02:11.247851 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:11.247762 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldwqr\" (UniqueName: \"kubernetes.io/projected/0ade3045-6b94-41a9-baeb-db7ce7a84c7a-kube-api-access-ldwqr\") pod \"network-check-target-6xdjl\" (UID: \"0ade3045-6b94-41a9-baeb-db7ce7a84c7a\") " pod="openshift-network-diagnostics/network-check-target-6xdjl" Jul 13 07:02:11.247851 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:11.247819 2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 07:02:11.248024 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:11.247852 2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 07:02:11.248024 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:11.247875 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 07:02:11.248024 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:11.247896 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 07:02:11.248024 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:11.247903 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/638764ba-e970-42f1-83ec-cf5c7075d61e-metrics-certs podName:638764ba-e970-42f1-83ec-cf5c7075d61e nodeName:}" failed. No retries permitted until 2026-07-13 07:02:15.247881157 +0000 UTC m=+10.083444980 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/638764ba-e970-42f1-83ec-cf5c7075d61e-metrics-certs") pod "network-metrics-daemon-znhkv" (UID: "638764ba-e970-42f1-83ec-cf5c7075d61e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 07:02:11.248024 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:11.247909 2586 projected.go:196] Error preparing data for projected volume kube-api-access-ldwqr for pod openshift-network-diagnostics/network-check-target-6xdjl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 07:02:11.248024 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:11.247926 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3724c5e-1a2b-410b-a745-b0f90bf27358-original-pull-secret podName:c3724c5e-1a2b-410b-a745-b0f90bf27358 nodeName:}" failed. No retries permitted until 2026-07-13 07:02:15.24791369 +0000 UTC m=+10.083477502 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c3724c5e-1a2b-410b-a745-b0f90bf27358-original-pull-secret") pod "global-pull-secret-syncer-lb8th" (UID: "c3724c5e-1a2b-410b-a745-b0f90bf27358") : object "kube-system"/"original-pull-secret" not registered Jul 13 07:02:11.248024 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:11.247952 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ade3045-6b94-41a9-baeb-db7ce7a84c7a-kube-api-access-ldwqr podName:0ade3045-6b94-41a9-baeb-db7ce7a84c7a nodeName:}" failed. No retries permitted until 2026-07-13 07:02:15.247937174 +0000 UTC m=+10.083500998 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-ldwqr" (UniqueName: "kubernetes.io/projected/0ade3045-6b94-41a9-baeb-db7ce7a84c7a-kube-api-access-ldwqr") pod "network-check-target-6xdjl" (UID: "0ade3045-6b94-41a9-baeb-db7ce7a84c7a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 07:02:11.877311 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:11.876850 2586 generic.go:356] "Generic (PLEG): container finished" podID="0de86516d8a93e213ba2d06529343779" containerID="41352c23a750792ade24bcbfb7bd3a08c5adf1f19cab7bca98ca09d5bf1f1059" exitCode=0 Jul 13 07:02:11.877311 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:11.876945 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-37.ec2.internal" event={"ID":"0de86516d8a93e213ba2d06529343779","Type":"ContainerDied","Data":"41352c23a750792ade24bcbfb7bd3a08c5adf1f19cab7bca98ca09d5bf1f1059"} Jul 13 07:02:12.799280 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:12.799240 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lb8th" Jul 13 07:02:12.799452 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:12.799371 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-znhkv" Jul 13 07:02:12.799452 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:12.799377 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lb8th" podUID="c3724c5e-1a2b-410b-a745-b0f90bf27358" Jul 13 07:02:12.799547 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:12.799483 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-znhkv" podUID="638764ba-e970-42f1-83ec-cf5c7075d61e" Jul 13 07:02:12.799547 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:12.799376 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6xdjl" Jul 13 07:02:12.799647 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:12.799574 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6xdjl" podUID="0ade3045-6b94-41a9-baeb-db7ce7a84c7a" Jul 13 07:02:14.799265 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:14.799183 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lb8th" Jul 13 07:02:14.799692 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:14.799327 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lb8th" podUID="c3724c5e-1a2b-410b-a745-b0f90bf27358" Jul 13 07:02:14.799766 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:14.799744 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-znhkv" Jul 13 07:02:14.799886 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:14.799863 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-znhkv" podUID="638764ba-e970-42f1-83ec-cf5c7075d61e" Jul 13 07:02:14.799937 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:14.799923 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6xdjl" Jul 13 07:02:14.800007 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:14.799990 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6xdjl" podUID="0ade3045-6b94-41a9-baeb-db7ce7a84c7a" Jul 13 07:02:15.282341 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:15.282299 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldwqr\" (UniqueName: \"kubernetes.io/projected/0ade3045-6b94-41a9-baeb-db7ce7a84c7a-kube-api-access-ldwqr\") pod \"network-check-target-6xdjl\" (UID: \"0ade3045-6b94-41a9-baeb-db7ce7a84c7a\") " pod="openshift-network-diagnostics/network-check-target-6xdjl" Jul 13 07:02:15.282554 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:15.282354 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/638764ba-e970-42f1-83ec-cf5c7075d61e-metrics-certs\") pod \"network-metrics-daemon-znhkv\" (UID: \"638764ba-e970-42f1-83ec-cf5c7075d61e\") " pod="openshift-multus/network-metrics-daemon-znhkv" Jul 13 07:02:15.282554 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:15.282384 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c3724c5e-1a2b-410b-a745-b0f90bf27358-original-pull-secret\") pod \"global-pull-secret-syncer-lb8th\" (UID: \"c3724c5e-1a2b-410b-a745-b0f90bf27358\") " pod="kube-system/global-pull-secret-syncer-lb8th" Jul 13 07:02:15.282554 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:15.282488 2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 07:02:15.282554 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:15.282499 2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 07:02:15.282554 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:15.282487 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 07:02:15.282554 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:15.282548 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3724c5e-1a2b-410b-a745-b0f90bf27358-original-pull-secret podName:c3724c5e-1a2b-410b-a745-b0f90bf27358 nodeName:}" failed. No retries permitted until 2026-07-13 07:02:23.282530207 +0000 UTC m=+18.118094022 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c3724c5e-1a2b-410b-a745-b0f90bf27358-original-pull-secret") pod "global-pull-secret-syncer-lb8th" (UID: "c3724c5e-1a2b-410b-a745-b0f90bf27358") : object "kube-system"/"original-pull-secret" not registered Jul 13 07:02:15.282554 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:15.282555 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 07:02:15.282932 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:15.282567 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/638764ba-e970-42f1-83ec-cf5c7075d61e-metrics-certs podName:638764ba-e970-42f1-83ec-cf5c7075d61e nodeName:}" failed. No retries permitted until 2026-07-13 07:02:23.282556906 +0000 UTC m=+18.118120716 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/638764ba-e970-42f1-83ec-cf5c7075d61e-metrics-certs") pod "network-metrics-daemon-znhkv" (UID: "638764ba-e970-42f1-83ec-cf5c7075d61e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 07:02:15.282932 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:15.282571 2586 projected.go:196] Error preparing data for projected volume kube-api-access-ldwqr for pod openshift-network-diagnostics/network-check-target-6xdjl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 07:02:15.282932 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:15.282618 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ade3045-6b94-41a9-baeb-db7ce7a84c7a-kube-api-access-ldwqr podName:0ade3045-6b94-41a9-baeb-db7ce7a84c7a nodeName:}" failed. No retries permitted until 2026-07-13 07:02:23.282601982 +0000 UTC m=+18.118165804 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-ldwqr" (UniqueName: "kubernetes.io/projected/0ade3045-6b94-41a9-baeb-db7ce7a84c7a-kube-api-access-ldwqr") pod "network-check-target-6xdjl" (UID: "0ade3045-6b94-41a9-baeb-db7ce7a84c7a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 07:02:16.799493 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:16.799453 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-znhkv" Jul 13 07:02:16.800107 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:16.799461 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lb8th" Jul 13 07:02:16.800107 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:16.799587 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-znhkv" podUID="638764ba-e970-42f1-83ec-cf5c7075d61e" Jul 13 07:02:16.800107 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:16.799598 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6xdjl" Jul 13 07:02:16.800107 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:16.799661 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lb8th" podUID="c3724c5e-1a2b-410b-a745-b0f90bf27358" Jul 13 07:02:16.800107 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:16.799774 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6xdjl" podUID="0ade3045-6b94-41a9-baeb-db7ce7a84c7a" Jul 13 07:02:18.799177 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:18.798876 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lb8th" Jul 13 07:02:18.799588 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:18.798931 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6xdjl" Jul 13 07:02:18.799588 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:18.799228 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lb8th" podUID="c3724c5e-1a2b-410b-a745-b0f90bf27358" Jul 13 07:02:18.799588 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:18.799303 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6xdjl" podUID="0ade3045-6b94-41a9-baeb-db7ce7a84c7a" Jul 13 07:02:18.799588 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:18.798934 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-znhkv" Jul 13 07:02:18.799588 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:18.799402 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-znhkv" podUID="638764ba-e970-42f1-83ec-cf5c7075d61e" Jul 13 07:02:18.894750 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:18.894715 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-56rlp" event={"ID":"2d5a84b8-155e-4dc0-8526-19a67235d08f","Type":"ContainerStarted","Data":"8cb59becb785eefdb87aa76c03e77c964022b0aadb2dd5892cc5402361fae003"} Jul 13 07:02:18.896564 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:18.896534 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-g6kgx" event={"ID":"75604003-f810-475e-b0db-af8180c49604","Type":"ContainerStarted","Data":"9b1a626b76a437fbed324fd1c27296528a1d9bb36b806a7aa55e15d15dc5c0f5"} Jul 13 07:02:18.898657 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:18.898632 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6l8np" event={"ID":"e1b48537-330d-49c0-80be-80fe72862d9d","Type":"ContainerStarted","Data":"5487d7b215642c48dcaf2943ee661c8253cd70a4d9048778ef88d723de140440"} Jul 13 07:02:18.900470 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:18.900342 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xs7q5" event={"ID":"2abf1e96-1ba2-4cc1-a632-48e28b374830","Type":"ContainerStarted","Data":"1233d959370dba214b3a004d1a2d01440574dcd207ac8f9974c883bab52ba9c6"} Jul 13 07:02:18.902088 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:18.902066 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m28rr" event={"ID":"4198fdf7-f7df-4dc0-99e0-92b0c4874587","Type":"ContainerStarted","Data":"488196a7aa270bdb69d2e18e41b386191ea6ab6ec851ced600ee44e601583e2f"} Jul 13 07:02:18.906348 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:18.904927 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-37.ec2.internal" event={"ID":"0de86516d8a93e213ba2d06529343779","Type":"ContainerStarted","Data":"dd43678ed7a65fa181b567b96e22814bb1ec939fb9359e5eb3b3fdc85db99724"} Jul 13 07:02:18.907133 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:18.907109 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vxcrv" event={"ID":"be89589f-0a40-4e74-9ecf-aeac7e231126","Type":"ContainerStarted","Data":"3991a12b145d541abe4a918edd70ba75ec898935a908cf241fb895e53bc9a023"} Jul 13 07:02:18.926865 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:18.926745 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-g6kgx" podStartSLOduration=2.686666765 podStartE2EDuration="12.926729424s" totalImagesPullingTime="10.240062659s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 07:02:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:02:18.926565876 +0000 UTC m=+13.762129707" watchObservedRunningTime="2026-07-13 07:02:18.926729424 +0000 UTC m=+13.762293257" Jul 13 07:02:18.940255 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:18.940208 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-37.ec2.internal" podStartSLOduration=12.940192682 podStartE2EDuration="12.940192682s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 07:02:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:02:18.939420989 +0000 UTC m=+13.774984823" watchObservedRunningTime="2026-07-13 07:02:18.940192682 +0000 UTC m=+13.775756514" Jul 13 07:02:18.951815 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:18.951773 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-vxcrv" podStartSLOduration=2.6887132400000002 podStartE2EDuration="12.951757792s" totalImagesPullingTime="10.263044552s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 07:02:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:02:18.951435996 +0000 UTC m=+13.786999830" watchObservedRunningTime="2026-07-13 07:02:18.951757792 +0000 UTC m=+13.787321625" Jul 13 07:02:18.965044 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:18.965002 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-6l8np" podStartSLOduration=3.435866825 podStartE2EDuration="13.964991071s" totalImagesPullingTime="10.529124246s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 07:02:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:02:18.964750522 +0000 UTC m=+13.800314357" watchObservedRunningTime="2026-07-13 07:02:18.964991071 +0000 UTC m=+13.800554904" Jul 13 07:02:18.985186 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:18.985142 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-xs7q5" podStartSLOduration=2.690383137 podStartE2EDuration="12.985126129s" totalImagesPullingTime="10.294742992s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:02:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:02:18.985050564 +0000 UTC m=+13.820614396" watchObservedRunningTime="2026-07-13 07:02:18.985126129 +0000 UTC m=+13.820689961" Jul 13 07:02:18.991209 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:18.991189 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-xs7q5" Jul 13 07:02:18.991986 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:18.991970 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-xs7q5" Jul 13 07:02:19.911342 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:19.911246 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-z7jkb" event={"ID":"d7d6c54b-cee3-4b7c-9b72-2afe08125102","Type":"ContainerStarted","Data":"f6de6ac5778378cff9299199c829fd4b4f5313127f62b1b5a078356aec4aecf1"} Jul 13 07:02:19.925713 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:19.925645 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-z7jkb" podStartSLOduration=4.655643252 podStartE2EDuration="14.925630451s" totalImagesPullingTime="10.269987199s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 07:02:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:02:19.925175706 +0000 UTC m=+14.760739540" watchObservedRunningTime="2026-07-13 07:02:19.925630451 +0000 UTC m=+14.761194282" Jul 13 07:02:20.798884 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:20.798852 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-znhkv" Jul 13 07:02:20.799060 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:20.798853 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lb8th" Jul 13 07:02:20.799060 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:20.798990 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-znhkv" podUID="638764ba-e970-42f1-83ec-cf5c7075d61e" Jul 13 07:02:20.799060 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:20.798853 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6xdjl" Jul 13 07:02:20.799060 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:20.799044 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lb8th" podUID="c3724c5e-1a2b-410b-a745-b0f90bf27358" Jul 13 07:02:20.799255 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:20.799134 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6xdjl" podUID="0ade3045-6b94-41a9-baeb-db7ce7a84c7a" Jul 13 07:02:20.913326 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:20.913297 2586 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 13 07:02:22.799370 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:22.799331 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-znhkv" Jul 13 07:02:22.800126 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:22.799331 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lb8th" Jul 13 07:02:22.800126 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:22.799468 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-znhkv" podUID="638764ba-e970-42f1-83ec-cf5c7075d61e" Jul 13 07:02:22.800126 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:22.799339 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6xdjl" Jul 13 07:02:22.800126 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:22.799559 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lb8th" podUID="c3724c5e-1a2b-410b-a745-b0f90bf27358" Jul 13 07:02:22.800126 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:22.799601 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6xdjl" podUID="0ade3045-6b94-41a9-baeb-db7ce7a84c7a" Jul 13 07:02:22.919676 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:22.919588 2586 generic.go:356] "Generic (PLEG): container finished" podID="2d5a84b8-155e-4dc0-8526-19a67235d08f" containerID="8cb59becb785eefdb87aa76c03e77c964022b0aadb2dd5892cc5402361fae003" exitCode=0 Jul 13 07:02:22.919676 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:22.919633 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-56rlp" event={"ID":"2d5a84b8-155e-4dc0-8526-19a67235d08f","Type":"ContainerDied","Data":"8cb59becb785eefdb87aa76c03e77c964022b0aadb2dd5892cc5402361fae003"} Jul 13 07:02:23.343425 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:23.343377 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldwqr\" (UniqueName: \"kubernetes.io/projected/0ade3045-6b94-41a9-baeb-db7ce7a84c7a-kube-api-access-ldwqr\") pod \"network-check-target-6xdjl\" (UID: \"0ade3045-6b94-41a9-baeb-db7ce7a84c7a\") " pod="openshift-network-diagnostics/network-check-target-6xdjl" Jul 13 07:02:23.343614 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:23.343435 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/638764ba-e970-42f1-83ec-cf5c7075d61e-metrics-certs\") pod \"network-metrics-daemon-znhkv\" (UID: \"638764ba-e970-42f1-83ec-cf5c7075d61e\") " pod="openshift-multus/network-metrics-daemon-znhkv" Jul 13 07:02:23.343614 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:23.343461 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c3724c5e-1a2b-410b-a745-b0f90bf27358-original-pull-secret\") pod \"global-pull-secret-syncer-lb8th\" (UID: \"c3724c5e-1a2b-410b-a745-b0f90bf27358\") " pod="kube-system/global-pull-secret-syncer-lb8th" Jul 13 07:02:23.343614 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:23.343556 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 07:02:23.343614 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:23.343586 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 07:02:23.343614 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:23.343598 2586 projected.go:196] Error preparing data for projected volume kube-api-access-ldwqr for pod openshift-network-diagnostics/network-check-target-6xdjl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 07:02:23.343852 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:23.343623 2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 07:02:23.343852 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:23.343561 2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 07:02:23.343852 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:23.343648 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ade3045-6b94-41a9-baeb-db7ce7a84c7a-kube-api-access-ldwqr podName:0ade3045-6b94-41a9-baeb-db7ce7a84c7a nodeName:}" failed. No retries permitted until 2026-07-13 07:02:39.343630175 +0000 UTC m=+34.179193987 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-ldwqr" (UniqueName: "kubernetes.io/projected/0ade3045-6b94-41a9-baeb-db7ce7a84c7a-kube-api-access-ldwqr") pod "network-check-target-6xdjl" (UID: "0ade3045-6b94-41a9-baeb-db7ce7a84c7a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 07:02:23.343852 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:23.343703 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3724c5e-1a2b-410b-a745-b0f90bf27358-original-pull-secret podName:c3724c5e-1a2b-410b-a745-b0f90bf27358 nodeName:}" failed. No retries permitted until 2026-07-13 07:02:39.343676037 +0000 UTC m=+34.179239847 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c3724c5e-1a2b-410b-a745-b0f90bf27358-original-pull-secret") pod "global-pull-secret-syncer-lb8th" (UID: "c3724c5e-1a2b-410b-a745-b0f90bf27358") : object "kube-system"/"original-pull-secret" not registered Jul 13 07:02:23.343852 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:23.343721 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/638764ba-e970-42f1-83ec-cf5c7075d61e-metrics-certs podName:638764ba-e970-42f1-83ec-cf5c7075d61e nodeName:}" failed. No retries permitted until 2026-07-13 07:02:39.343712531 +0000 UTC m=+34.179276339 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/638764ba-e970-42f1-83ec-cf5c7075d61e-metrics-certs") pod "network-metrics-daemon-znhkv" (UID: "638764ba-e970-42f1-83ec-cf5c7075d61e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 07:02:24.799400 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:24.799361 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lb8th" Jul 13 07:02:24.799981 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:24.799403 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6xdjl" Jul 13 07:02:24.799981 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:24.799407 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-znhkv" Jul 13 07:02:24.799981 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:24.799490 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lb8th" podUID="c3724c5e-1a2b-410b-a745-b0f90bf27358" Jul 13 07:02:24.799981 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:24.799596 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6xdjl" podUID="0ade3045-6b94-41a9-baeb-db7ce7a84c7a" Jul 13 07:02:24.799981 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:24.799700 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-znhkv" podUID="638764ba-e970-42f1-83ec-cf5c7075d61e" Jul 13 07:02:26.799128 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:26.799080 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6xdjl" Jul 13 07:02:26.799536 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:26.799147 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-znhkv" Jul 13 07:02:26.799536 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:26.799101 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lb8th" Jul 13 07:02:26.799536 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:26.799210 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6xdjl" podUID="0ade3045-6b94-41a9-baeb-db7ce7a84c7a" Jul 13 07:02:26.799536 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:26.799272 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-znhkv" podUID="638764ba-e970-42f1-83ec-cf5c7075d61e" Jul 13 07:02:26.799536 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:26.799359 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lb8th" podUID="c3724c5e-1a2b-410b-a745-b0f90bf27358" Jul 13 07:02:27.227147 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:27.227117 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-xs7q5" Jul 13 07:02:27.227324 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:27.227255 2586 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 13 07:02:27.228319 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:27.228050 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-xs7q5" Jul 13 07:02:27.442558 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:27.442375 2586 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 13 07:02:27.704766 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:27.704459 2586 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-13T07:02:27.442395392Z","UUID":"d7546ac5-e962-4f9d-a6ba-b689b7f5b17f","Handler":null,"Name":"","Endpoint":""} Jul 13 07:02:27.706161 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:27.706140 2586 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 13 07:02:27.706263 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:27.706177 2586 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 13 07:02:27.933317 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:27.933288 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7b964_3d71897a-4b2e-4e4e-9525-ccd89bd18458/ovn-acl-logging/0.log" Jul 13 07:02:27.933773 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:27.933670 2586 generic.go:356] "Generic (PLEG): container finished" podID="3d71897a-4b2e-4e4e-9525-ccd89bd18458" containerID="1a91cc4a7d34794fb9dcd8425771076065c6575380273e0ae4f746b58d28c928" exitCode=1 Jul 13 07:02:27.933773 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:27.933752 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7b964" event={"ID":"3d71897a-4b2e-4e4e-9525-ccd89bd18458","Type":"ContainerStarted","Data":"54bd035530bac3ca10d8c1cadc13d0d56498fc7d2dd2aa955e9445e78b59d0be"} Jul 13 07:02:27.933901 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:27.933877 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7b964" event={"ID":"3d71897a-4b2e-4e4e-9525-ccd89bd18458","Type":"ContainerStarted","Data":"88ec892439a87249278f3c6d3ae876dee4ffc2b5a80cefd9219050abb7d39ab5"} Jul 13 07:02:27.933939 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:27.933931 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7b964" event={"ID":"3d71897a-4b2e-4e4e-9525-ccd89bd18458","Type":"ContainerStarted","Data":"952121eddf39cd1cb5919de587c61fad80a0e18a464fb579298e93e25e6dc620"} Jul 13 07:02:27.933993 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:27.933979 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7b964" event={"ID":"3d71897a-4b2e-4e4e-9525-ccd89bd18458","Type":"ContainerDied","Data":"1a91cc4a7d34794fb9dcd8425771076065c6575380273e0ae4f746b58d28c928"} Jul 13 07:02:27.934047 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:27.934036 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7b964" event={"ID":"3d71897a-4b2e-4e4e-9525-ccd89bd18458","Type":"ContainerStarted","Data":"ce8135d3193fbc43b746be09551b188ebf4aa46761575eb1d0178b1e82d8bf2f"} Jul 13 07:02:27.935473 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:27.935450 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-g9z6n" event={"ID":"89973bd4-81f7-4536-841e-1571b0fd9506","Type":"ContainerStarted","Data":"14b285149701634a3afcb3e93e5726cafe0ba66f60353ce0660a80b5eae2bc28"} Jul 13 07:02:27.937722 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:27.937695 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m28rr" event={"ID":"4198fdf7-f7df-4dc0-99e0-92b0c4874587","Type":"ContainerStarted","Data":"e40e634950ed628a1d48bc7c7af46fd2ff6f08538ede8323d8fee08b82abea62"} Jul 13 07:02:27.950893 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:27.950844 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-g9z6n" podStartSLOduration=2.619039273 podStartE2EDuration="21.950828388s" totalImagesPullingTime="19.331789115s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 07:02:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:02:27.950042189 +0000 UTC m=+22.785606027" watchObservedRunningTime="2026-07-13 07:02:27.950828388 +0000 UTC m=+22.786392218" Jul 13 07:02:28.799484 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:28.799449 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6xdjl" Jul 13 07:02:28.799484 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:28.799484 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lb8th" Jul 13 07:02:28.799945 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:28.799448 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-znhkv" Jul 13 07:02:28.799945 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:28.799575 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6xdjl" podUID="0ade3045-6b94-41a9-baeb-db7ce7a84c7a" Jul 13 07:02:28.799945 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:28.799711 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-znhkv" podUID="638764ba-e970-42f1-83ec-cf5c7075d61e" Jul 13 07:02:28.799945 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:28.799799 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lb8th" podUID="c3724c5e-1a2b-410b-a745-b0f90bf27358" Jul 13 07:02:28.942349 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:28.942311 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m28rr" event={"ID":"4198fdf7-f7df-4dc0-99e0-92b0c4874587","Type":"ContainerStarted","Data":"2e40eb9e07abba36ab1dca6743aa0a4e21fc1101e150a584a783011a0f32e70e"} Jul 13 07:02:28.945604 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:28.945573 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7b964_3d71897a-4b2e-4e4e-9525-ccd89bd18458/ovn-acl-logging/0.log" Jul 13 07:02:28.946035 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:28.946016 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7b964" event={"ID":"3d71897a-4b2e-4e4e-9525-ccd89bd18458","Type":"ContainerStarted","Data":"3a5209f4219fdecfcaa7a52553e00b518f657af873541660267ae33a7f642ed9"} Jul 13 07:02:28.957999 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:28.957961 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m28rr" podStartSLOduration=3.354811023 podStartE2EDuration="23.957948331s" totalImagesPullingTime="20.603137308s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 07:02:05 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:02:28.957080626 +0000 UTC m=+23.792644482" watchObservedRunningTime="2026-07-13 07:02:28.957948331 +0000 UTC m=+23.793512162" Jul 13 07:02:30.799022 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:30.798980 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6xdjl" Jul 13 07:02:30.799022 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:30.799012 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lb8th" Jul 13 07:02:30.799555 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:30.799015 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-znhkv" Jul 13 07:02:30.799555 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:30.799101 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6xdjl" podUID="0ade3045-6b94-41a9-baeb-db7ce7a84c7a" Jul 13 07:02:30.799555 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:30.799217 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lb8th" podUID="c3724c5e-1a2b-410b-a745-b0f90bf27358" Jul 13 07:02:30.799555 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:30.799311 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-znhkv" podUID="638764ba-e970-42f1-83ec-cf5c7075d61e" Jul 13 07:02:30.953592 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:30.953559 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7b964_3d71897a-4b2e-4e4e-9525-ccd89bd18458/ovn-acl-logging/0.log" Jul 13 07:02:30.954000 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:30.953966 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7b964" event={"ID":"3d71897a-4b2e-4e4e-9525-ccd89bd18458","Type":"ContainerStarted","Data":"4dfb1422578409cf007c3f63aff6fe3a26975ac4ec438db046f4ae408aa3abc0"} Jul 13 07:02:32.799751 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:32.799549 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lb8th" Jul 13 07:02:32.800303 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:32.799555 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-znhkv" Jul 13 07:02:32.800303 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:32.799832 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lb8th" podUID="c3724c5e-1a2b-410b-a745-b0f90bf27358" Jul 13 07:02:32.800303 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:32.799593 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6xdjl" Jul 13 07:02:32.800303 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:32.799899 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-znhkv" podUID="638764ba-e970-42f1-83ec-cf5c7075d61e" Jul 13 07:02:32.800303 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:32.799970 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6xdjl" podUID="0ade3045-6b94-41a9-baeb-db7ce7a84c7a" Jul 13 07:02:32.960064 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:32.960032 2586 generic.go:356] "Generic (PLEG): container finished" podID="2d5a84b8-155e-4dc0-8526-19a67235d08f" containerID="268d13a36d92a29284a9cb9ceaa7a35e8e065cbff6db08aeb738c0e2194bf1fb" exitCode=0 Jul 13 07:02:32.960247 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:32.960106 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-56rlp" event={"ID":"2d5a84b8-155e-4dc0-8526-19a67235d08f","Type":"ContainerDied","Data":"268d13a36d92a29284a9cb9ceaa7a35e8e065cbff6db08aeb738c0e2194bf1fb"} Jul 13 07:02:32.964021 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:32.964001 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7b964_3d71897a-4b2e-4e4e-9525-ccd89bd18458/ovn-acl-logging/0.log" Jul 13 07:02:32.964373 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:32.964354 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7b964" event={"ID":"3d71897a-4b2e-4e4e-9525-ccd89bd18458","Type":"ContainerStarted","Data":"5cee095a94ce2f9213c9c6bf1feb1845f9035b366b528174cdf0a0bbe61abee3"} Jul 13 07:02:32.964609 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:32.964586 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:32.964692 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:32.964633 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:32.964793 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:32.964773 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:32.964908 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:32.964830 2586 scope.go:117] "RemoveContainer" containerID="1a91cc4a7d34794fb9dcd8425771076065c6575380273e0ae4f746b58d28c928" Jul 13 07:02:32.980020 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:32.979993 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:32.981184 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:32.981164 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:02:33.970891 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:33.970863 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7b964_3d71897a-4b2e-4e4e-9525-ccd89bd18458/ovn-acl-logging/0.log" Jul 13 07:02:33.971341 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:33.971256 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7b964" event={"ID":"3d71897a-4b2e-4e4e-9525-ccd89bd18458","Type":"ContainerStarted","Data":"602bcdb49a19033cdea986dafbf15564e5896c5b4142f354be6c08a46926161b"} Jul 13 07:02:33.997242 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:33.997193 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-7b964" podStartSLOduration=8.616471848 podStartE2EDuration="27.997175131s" totalImagesPullingTime="19.380703283s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 07:02:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:02:33.995853888 +0000 UTC m=+28.831417734" watchObservedRunningTime="2026-07-13 07:02:33.997175131 +0000 UTC m=+28.832738963" Jul 13 07:02:34.610242 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:34.610143 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lb8th"] Jul 13 07:02:34.610377 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:34.610291 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lb8th" Jul 13 07:02:34.610435 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:34.610390 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lb8th" podUID="c3724c5e-1a2b-410b-a745-b0f90bf27358" Jul 13 07:02:34.613372 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:34.613346 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-znhkv"] Jul 13 07:02:34.613500 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:34.613486 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-znhkv" Jul 13 07:02:34.613615 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:34.613594 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-znhkv" podUID="638764ba-e970-42f1-83ec-cf5c7075d61e" Jul 13 07:02:34.625320 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:34.625302 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6xdjl"] Jul 13 07:02:34.625417 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:34.625379 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6xdjl" Jul 13 07:02:34.625473 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:34.625445 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6xdjl" podUID="0ade3045-6b94-41a9-baeb-db7ce7a84c7a" Jul 13 07:02:34.975672 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:34.975643 2586 generic.go:356] "Generic (PLEG): container finished" podID="2d5a84b8-155e-4dc0-8526-19a67235d08f" containerID="8d96682fdfb59a83c5f910ec07c937fa1d0a00b55d9f6f95926acbf1115b4c18" exitCode=0 Jul 13 07:02:34.976182 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:34.975714 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-56rlp" event={"ID":"2d5a84b8-155e-4dc0-8526-19a67235d08f","Type":"ContainerDied","Data":"8d96682fdfb59a83c5f910ec07c937fa1d0a00b55d9f6f95926acbf1115b4c18"} Jul 13 07:02:36.799550 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:36.799323 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lb8th" Jul 13 07:02:36.799892 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:36.799341 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-znhkv" Jul 13 07:02:36.799892 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:36.799591 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lb8th" podUID="c3724c5e-1a2b-410b-a745-b0f90bf27358" Jul 13 07:02:36.799892 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:36.799379 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6xdjl" Jul 13 07:02:36.799892 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:36.799664 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-znhkv" podUID="638764ba-e970-42f1-83ec-cf5c7075d61e" Jul 13 07:02:36.799892 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:36.799749 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6xdjl" podUID="0ade3045-6b94-41a9-baeb-db7ce7a84c7a" Jul 13 07:02:36.982566 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:36.982535 2586 generic.go:356] "Generic (PLEG): container finished" podID="2d5a84b8-155e-4dc0-8526-19a67235d08f" containerID="1c9ccfa65c73b01e4ab7c9ac31520fd9ac7b74cd32f97feb49aede9592e8bb85" exitCode=0 Jul 13 07:02:36.982733 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:36.982598 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-56rlp" event={"ID":"2d5a84b8-155e-4dc0-8526-19a67235d08f","Type":"ContainerDied","Data":"1c9ccfa65c73b01e4ab7c9ac31520fd9ac7b74cd32f97feb49aede9592e8bb85"} Jul 13 07:02:38.799201 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:38.799168 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-znhkv" Jul 13 07:02:38.799806 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:38.799175 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lb8th" Jul 13 07:02:38.799806 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:38.799302 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6xdjl" Jul 13 07:02:38.799806 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:38.799279 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-znhkv" podUID="638764ba-e970-42f1-83ec-cf5c7075d61e" Jul 13 07:02:38.799806 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:38.799369 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lb8th" podUID="c3724c5e-1a2b-410b-a745-b0f90bf27358" Jul 13 07:02:38.799806 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:38.799471 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6xdjl" podUID="0ade3045-6b94-41a9-baeb-db7ce7a84c7a" Jul 13 07:02:39.360317 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:39.360223 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldwqr\" (UniqueName: \"kubernetes.io/projected/0ade3045-6b94-41a9-baeb-db7ce7a84c7a-kube-api-access-ldwqr\") pod \"network-check-target-6xdjl\" (UID: \"0ade3045-6b94-41a9-baeb-db7ce7a84c7a\") " pod="openshift-network-diagnostics/network-check-target-6xdjl" Jul 13 07:02:39.360317 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:39.360280 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/638764ba-e970-42f1-83ec-cf5c7075d61e-metrics-certs\") pod \"network-metrics-daemon-znhkv\" (UID: \"638764ba-e970-42f1-83ec-cf5c7075d61e\") " pod="openshift-multus/network-metrics-daemon-znhkv" Jul 13 07:02:39.360317 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:39.360307 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c3724c5e-1a2b-410b-a745-b0f90bf27358-original-pull-secret\") pod \"global-pull-secret-syncer-lb8th\" (UID: \"c3724c5e-1a2b-410b-a745-b0f90bf27358\") " pod="kube-system/global-pull-secret-syncer-lb8th" Jul 13 07:02:39.360573 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:39.360405 2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 07:02:39.360573 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:39.360480 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/638764ba-e970-42f1-83ec-cf5c7075d61e-metrics-certs podName:638764ba-e970-42f1-83ec-cf5c7075d61e nodeName:}" failed. No retries permitted until 2026-07-13 07:03:11.360460791 +0000 UTC m=+66.196024616 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/638764ba-e970-42f1-83ec-cf5c7075d61e-metrics-certs") pod "network-metrics-daemon-znhkv" (UID: "638764ba-e970-42f1-83ec-cf5c7075d61e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 07:02:39.360573 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:39.360415 2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 07:02:39.360573 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:39.360551 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3724c5e-1a2b-410b-a745-b0f90bf27358-original-pull-secret podName:c3724c5e-1a2b-410b-a745-b0f90bf27358 nodeName:}" failed. No retries permitted until 2026-07-13 07:03:11.360534492 +0000 UTC m=+66.196098301 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c3724c5e-1a2b-410b-a745-b0f90bf27358-original-pull-secret") pod "global-pull-secret-syncer-lb8th" (UID: "c3724c5e-1a2b-410b-a745-b0f90bf27358") : object "kube-system"/"original-pull-secret" not registered Jul 13 07:02:39.360573 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:39.360419 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 07:02:39.360573 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:39.360574 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 07:02:39.360886 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:39.360584 2586 projected.go:196] Error preparing data for projected volume kube-api-access-ldwqr for pod openshift-network-diagnostics/network-check-target-6xdjl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 07:02:39.360886 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:39.360612 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ade3045-6b94-41a9-baeb-db7ce7a84c7a-kube-api-access-ldwqr podName:0ade3045-6b94-41a9-baeb-db7ce7a84c7a nodeName:}" failed. No retries permitted until 2026-07-13 07:03:11.360605687 +0000 UTC m=+66.196169496 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-ldwqr" (UniqueName: "kubernetes.io/projected/0ade3045-6b94-41a9-baeb-db7ce7a84c7a-kube-api-access-ldwqr") pod "network-check-target-6xdjl" (UID: "0ade3045-6b94-41a9-baeb-db7ce7a84c7a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 07:02:40.799063 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:40.799020 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6xdjl" Jul 13 07:02:40.799628 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:40.799022 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-znhkv" Jul 13 07:02:40.799628 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:40.799162 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6xdjl" podUID="0ade3045-6b94-41a9-baeb-db7ce7a84c7a" Jul 13 07:02:40.799628 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:40.799041 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lb8th" Jul 13 07:02:40.799628 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:40.799229 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-znhkv" podUID="638764ba-e970-42f1-83ec-cf5c7075d61e" Jul 13 07:02:40.799628 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:40.799292 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lb8th" podUID="c3724c5e-1a2b-410b-a745-b0f90bf27358" Jul 13 07:02:40.946475 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:40.946447 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-37.ec2.internal" event="NodeReady" Jul 13 07:02:40.946652 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:40.946593 2586 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 13 07:02:40.985114 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:40.985019 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-69599bff85-z8nf7"] Jul 13 07:02:40.987491 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:40.987463 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-drnth"] Jul 13 07:02:40.987651 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:40.987603 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69599bff85-z8nf7" Jul 13 07:02:40.989459 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:40.989428 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-drnth" Jul 13 07:02:40.990338 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:40.990304 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 13 07:02:40.991048 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:40.990775 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 13 07:02:40.991048 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:40.990808 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 13 07:02:40.991048 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:40.990833 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-tn7lg\"" Jul 13 07:02:40.991296 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:40.991279 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 13 07:02:40.991711 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:40.991607 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 13 07:02:40.991711 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:40.991702 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-9pnfc\"" Jul 13 07:02:40.996912 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:40.996889 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 13 07:02:40.999248 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:40.999224 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-r878q"] Jul 13 07:02:41.002342 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.001109 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-r878q" Jul 13 07:02:41.002342 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.001979 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-z7x8m"] Jul 13 07:02:41.006096 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.006070 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 13 07:02:41.007263 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.007237 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-q8c2x\"" Jul 13 07:02:41.007520 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.007489 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 13 07:02:41.009203 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.009181 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 13 07:02:41.010786 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.010756 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 13 07:02:41.011625 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.011535 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-m7g8b"] Jul 13 07:02:41.011738 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.011664 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-z7x8m" Jul 13 07:02:41.013363 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.013341 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-lhs5q"] Jul 13 07:02:41.013933 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.013554 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-m7g8b" Jul 13 07:02:41.014477 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.014456 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 13 07:02:41.014561 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.014536 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 13 07:02:41.014754 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.014736 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 13 07:02:41.015094 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.015074 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 13 07:02:41.015225 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.015209 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 13 07:02:41.015322 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.015281 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-8jcdn\"" Jul 13 07:02:41.015603 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.015583 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-74xdm"] Jul 13 07:02:41.015753 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.015736 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-lhs5q" Jul 13 07:02:41.016693 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.016656 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 13 07:02:41.016829 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.016809 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 13 07:02:41.017102 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.017083 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-nxgnr\"" Jul 13 07:02:41.017201 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.017129 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 13 07:02:41.017521 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.017492 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7f7c7b8964-kxkhw"] Jul 13 07:02:41.017622 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.017607 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-74xdm" Jul 13 07:02:41.021615 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.019282 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-8grn6"] Jul 13 07:02:41.021751 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.021028 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 13 07:02:41.021888 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.021563 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7f7c7b8964-kxkhw" Jul 13 07:02:41.024696 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.024654 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-bh4jx"] Jul 13 07:02:41.025091 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.025041 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 13 07:02:41.025581 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.025563 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 13 07:02:41.025765 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.025746 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 13 07:02:41.027164 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.027141 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 13 07:02:41.027958 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.027936 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-69599bff85-z8nf7"] Jul 13 07:02:41.028055 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.027978 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-wg6mf"] Jul 13 07:02:41.028055 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.028009 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-cj6rs\"" Jul 13 07:02:41.028157 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.028075 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-8grn6" Jul 13 07:02:41.029110 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.029094 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-bh4jx" Jul 13 07:02:41.029231 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.029210 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 13 07:02:41.029532 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.029130 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 13 07:02:41.029597 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.029176 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 13 07:02:41.029883 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.029861 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-ps2kk"] Jul 13 07:02:41.030617 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.030532 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 13 07:02:41.031024 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.030811 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 13 07:02:41.031024 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.030857 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wg6mf" Jul 13 07:02:41.031024 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.031002 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-xdpjm\"" Jul 13 07:02:41.031227 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.031209 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 13 07:02:41.031553 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.031535 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 13 07:02:41.031947 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.031904 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 13 07:02:41.032553 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.032457 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 13 07:02:41.032841 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.032553 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 13 07:02:41.037425 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.036387 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-2rcmc\"" Jul 13 07:02:41.038099 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.038054 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 13 07:02:41.038547 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.038423 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 13 07:02:41.038716 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.038645 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 13 07:02:41.042170 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.038888 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-fgjdm\"" Jul 13 07:02:41.042170 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.038773 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 13 07:02:41.042170 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.041154 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 13 07:02:41.042400 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.042226 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 13 07:02:41.042540 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.042524 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-hr5g7\"" Jul 13 07:02:41.044044 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.043997 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-r878q"] Jul 13 07:02:41.044044 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.044046 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-m7g8b"] Jul 13 07:02:41.044206 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.044071 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-lhs5q"] Jul 13 07:02:41.044206 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.044173 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ps2kk" Jul 13 07:02:41.044303 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.044215 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-bh4jx"] Jul 13 07:02:41.044303 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.044288 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-drnth"] Jul 13 07:02:41.044394 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.044319 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-8grn6"] Jul 13 07:02:41.044394 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.044362 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wg6mf"] Jul 13 07:02:41.044491 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.044395 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-z7x8m"] Jul 13 07:02:41.044562 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.044543 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ts2w7\"" Jul 13 07:02:41.045270 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.044889 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 13 07:02:41.045814 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.045795 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 13 07:02:41.045911 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.045850 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7f7c7b8964-kxkhw"] Jul 13 07:02:41.050139 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.050105 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-wcqht\"" Jul 13 07:02:41.050730 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.050710 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 13 07:02:41.050863 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.050716 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 13 07:02:41.050985 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.050966 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 13 07:02:41.053270 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.053208 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ps2kk"] Jul 13 07:02:41.058362 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.058337 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-74xdm"] Jul 13 07:02:41.071939 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.071914 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-image-registry-private-configuration\") pod \"image-registry-69599bff85-z8nf7\" (UID: \"16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d\") " pod="openshift-image-registry/image-registry-69599bff85-z8nf7" Jul 13 07:02:41.072062 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.071949 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-installation-pull-secrets\") pod \"image-registry-69599bff85-z8nf7\" (UID: \"16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d\") " pod="openshift-image-registry/image-registry-69599bff85-z8nf7" Jul 13 07:02:41.072062 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.071976 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdqlp\" (UniqueName: \"kubernetes.io/projected/c4a2167f-7cee-4066-914f-d9c9e4d3bff7-kube-api-access-rdqlp\") pod \"volume-data-source-validator-78495597f6-drnth\" (UID: \"c4a2167f-7cee-4066-914f-d9c9e4d3bff7\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-drnth" Jul 13 07:02:41.072062 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.072029 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-registry-tls\") pod \"image-registry-69599bff85-z8nf7\" (UID: \"16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d\") " pod="openshift-image-registry/image-registry-69599bff85-z8nf7" Jul 13 07:02:41.072062 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.072054 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-ca-trust-extracted\") pod \"image-registry-69599bff85-z8nf7\" (UID: \"16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d\") " pod="openshift-image-registry/image-registry-69599bff85-z8nf7" Jul 13 07:02:41.072263 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.072109 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5psx\" (UniqueName: \"kubernetes.io/projected/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-kube-api-access-h5psx\") pod \"image-registry-69599bff85-z8nf7\" (UID: \"16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d\") " pod="openshift-image-registry/image-registry-69599bff85-z8nf7" Jul 13 07:02:41.072263 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.072139 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-bound-sa-token\") pod \"image-registry-69599bff85-z8nf7\" (UID: \"16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d\") " pod="openshift-image-registry/image-registry-69599bff85-z8nf7" Jul 13 07:02:41.072263 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.072162 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/293edc99-2691-4915-ba03-7407d88b89b4-serving-cert\") pod \"console-operator-7866497494-r878q\" (UID: \"293edc99-2691-4915-ba03-7407d88b89b4\") " pod="openshift-console-operator/console-operator-7866497494-r878q" Jul 13 07:02:41.072263 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.072188 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-trusted-ca\") pod \"image-registry-69599bff85-z8nf7\" (UID: \"16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d\") " pod="openshift-image-registry/image-registry-69599bff85-z8nf7" Jul 13 07:02:41.072263 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.072205 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgkcb\" (UniqueName: \"kubernetes.io/projected/293edc99-2691-4915-ba03-7407d88b89b4-kube-api-access-cgkcb\") pod \"console-operator-7866497494-r878q\" (UID: \"293edc99-2691-4915-ba03-7407d88b89b4\") " pod="openshift-console-operator/console-operator-7866497494-r878q" Jul 13 07:02:41.072263 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.072221 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/293edc99-2691-4915-ba03-7407d88b89b4-config\") pod \"console-operator-7866497494-r878q\" (UID: \"293edc99-2691-4915-ba03-7407d88b89b4\") " pod="openshift-console-operator/console-operator-7866497494-r878q" Jul 13 07:02:41.072468 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.072286 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-registry-certificates\") pod \"image-registry-69599bff85-z8nf7\" (UID: \"16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d\") " pod="openshift-image-registry/image-registry-69599bff85-z8nf7" Jul 13 07:02:41.072468 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.072324 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/293edc99-2691-4915-ba03-7407d88b89b4-trusted-ca\") pod \"console-operator-7866497494-r878q\" (UID: \"293edc99-2691-4915-ba03-7407d88b89b4\") " pod="openshift-console-operator/console-operator-7866497494-r878q" Jul 13 07:02:41.173236 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.173199 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/293edc99-2691-4915-ba03-7407d88b89b4-config\") pod \"console-operator-7866497494-r878q\" (UID: \"293edc99-2691-4915-ba03-7407d88b89b4\") " pod="openshift-console-operator/console-operator-7866497494-r878q" Jul 13 07:02:41.173428 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.173246 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fz7m\" (UniqueName: \"kubernetes.io/projected/8d761ca6-7ceb-4a5f-be27-3e668d545e84-kube-api-access-5fz7m\") pod \"network-check-source-869dcfcdcd-bh4jx\" (UID: \"8d761ca6-7ceb-4a5f-be27-3e668d545e84\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-bh4jx" Jul 13 07:02:41.173428 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.173292 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gc2qz\" (UniqueName: \"kubernetes.io/projected/30689818-dbf9-4665-86dc-b81d96b6bed1-kube-api-access-gc2qz\") pod \"ingress-canary-ps2kk\" (UID: \"30689818-dbf9-4665-86dc-b81d96b6bed1\") " pod="openshift-ingress-canary/ingress-canary-ps2kk" Jul 13 07:02:41.173428 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.173319 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-registry-certificates\") pod \"image-registry-69599bff85-z8nf7\" (UID: \"16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d\") " pod="openshift-image-registry/image-registry-69599bff85-z8nf7" Jul 13 07:02:41.173653 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.173621 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vn7jz\" (UniqueName: \"kubernetes.io/projected/05bf1881-84d9-42e6-a7ed-1a348839015d-kube-api-access-vn7jz\") pod \"insights-operator-68d9dd94cf-z7x8m\" (UID: \"05bf1881-84d9-42e6-a7ed-1a348839015d\") " pod="openshift-insights/insights-operator-68d9dd94cf-z7x8m" Jul 13 07:02:41.173798 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.173713 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/293edc99-2691-4915-ba03-7407d88b89b4-trusted-ca\") pod \"console-operator-7866497494-r878q\" (UID: \"293edc99-2691-4915-ba03-7407d88b89b4\") " pod="openshift-console-operator/console-operator-7866497494-r878q" Jul 13 07:02:41.174012 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.173991 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-registry-certificates\") pod \"image-registry-69599bff85-z8nf7\" (UID: \"16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d\") " pod="openshift-image-registry/image-registry-69599bff85-z8nf7" Jul 13 07:02:41.174088 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.173745 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c28d91ed-a056-47d9-ba7c-f789f42dd6cc-service-ca-bundle\") pod \"router-default-7f7c7b8964-kxkhw\" (UID: \"c28d91ed-a056-47d9-ba7c-f789f42dd6cc\") " pod="openshift-ingress/router-default-7f7c7b8964-kxkhw" Jul 13 07:02:41.174088 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.174056 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/293edc99-2691-4915-ba03-7407d88b89b4-config\") pod \"console-operator-7866497494-r878q\" (UID: \"293edc99-2691-4915-ba03-7407d88b89b4\") " pod="openshift-console-operator/console-operator-7866497494-r878q" Jul 13 07:02:41.174178 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.174092 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e38c842e-619b-4849-9a60-d65e577ba45e-serving-cert\") pod \"service-ca-operator-7695fc6c8c-lhs5q\" (UID: \"e38c842e-619b-4849-9a60-d65e577ba45e\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-lhs5q" Jul 13 07:02:41.174178 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.174119 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/05bf1881-84d9-42e6-a7ed-1a348839015d-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-z7x8m\" (UID: \"05bf1881-84d9-42e6-a7ed-1a348839015d\") " pod="openshift-insights/insights-operator-68d9dd94cf-z7x8m" Jul 13 07:02:41.174178 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.174149 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/396e6c4f-4cbc-40ed-b6e8-babbbbc48d39-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-m7g8b\" (UID: \"396e6c4f-4cbc-40ed-b6e8-babbbbc48d39\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-m7g8b" Jul 13 07:02:41.174330 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.174214 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c28d91ed-a056-47d9-ba7c-f789f42dd6cc-metrics-certs\") pod \"router-default-7f7c7b8964-kxkhw\" (UID: \"c28d91ed-a056-47d9-ba7c-f789f42dd6cc\") " pod="openshift-ingress/router-default-7f7c7b8964-kxkhw" Jul 13 07:02:41.174330 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.174274 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-image-registry-private-configuration\") pod \"image-registry-69599bff85-z8nf7\" (UID: \"16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d\") " pod="openshift-image-registry/image-registry-69599bff85-z8nf7" Jul 13 07:02:41.174424 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.174334 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-installation-pull-secrets\") pod \"image-registry-69599bff85-z8nf7\" (UID: \"16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d\") " pod="openshift-image-registry/image-registry-69599bff85-z8nf7" Jul 13 07:02:41.174424 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.174383 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a7c70271-ef81-4393-9206-d7957e5c86e1-metrics-tls\") pod \"dns-default-wg6mf\" (UID: \"a7c70271-ef81-4393-9206-d7957e5c86e1\") " pod="openshift-dns/dns-default-wg6mf" Jul 13 07:02:41.174513 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.174456 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/293edc99-2691-4915-ba03-7407d88b89b4-trusted-ca\") pod \"console-operator-7866497494-r878q\" (UID: \"293edc99-2691-4915-ba03-7407d88b89b4\") " pod="openshift-console-operator/console-operator-7866497494-r878q" Jul 13 07:02:41.174513 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.174478 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtpfg\" (UniqueName: \"kubernetes.io/projected/a7c70271-ef81-4393-9206-d7957e5c86e1-kube-api-access-wtpfg\") pod \"dns-default-wg6mf\" (UID: \"a7c70271-ef81-4393-9206-d7957e5c86e1\") " pod="openshift-dns/dns-default-wg6mf" Jul 13 07:02:41.174513 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.174508 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/05bf1881-84d9-42e6-a7ed-1a348839015d-snapshots\") pod \"insights-operator-68d9dd94cf-z7x8m\" (UID: \"05bf1881-84d9-42e6-a7ed-1a348839015d\") " pod="openshift-insights/insights-operator-68d9dd94cf-z7x8m" Jul 13 07:02:41.174674 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.174532 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/30689818-dbf9-4665-86dc-b81d96b6bed1-cert\") pod \"ingress-canary-ps2kk\" (UID: \"30689818-dbf9-4665-86dc-b81d96b6bed1\") " pod="openshift-ingress-canary/ingress-canary-ps2kk" Jul 13 07:02:41.174674 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.174563 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdqlp\" (UniqueName: \"kubernetes.io/projected/c4a2167f-7cee-4066-914f-d9c9e4d3bff7-kube-api-access-rdqlp\") pod \"volume-data-source-validator-78495597f6-drnth\" (UID: \"c4a2167f-7cee-4066-914f-d9c9e4d3bff7\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-drnth" Jul 13 07:02:41.174674 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.174617 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-registry-tls\") pod \"image-registry-69599bff85-z8nf7\" (UID: \"16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d\") " pod="openshift-image-registry/image-registry-69599bff85-z8nf7" Jul 13 07:02:41.174674 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.174643 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ddf4f6fd-5259-4c72-94df-245960dc7642-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-8grn6\" (UID: \"ddf4f6fd-5259-4c72-94df-245960dc7642\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-8grn6" Jul 13 07:02:41.174889 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.174675 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-ca-trust-extracted\") pod \"image-registry-69599bff85-z8nf7\" (UID: \"16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d\") " pod="openshift-image-registry/image-registry-69599bff85-z8nf7" Jul 13 07:02:41.174889 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.174718 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/5b0bbb6f-a4f2-428a-8c0b-8561d031061c-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-74xdm\" (UID: \"5b0bbb6f-a4f2-428a-8c0b-8561d031061c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-74xdm" Jul 13 07:02:41.174889 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:41.174727 2586 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 13 07:02:41.174889 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:41.174743 2586 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69599bff85-z8nf7: secret "image-registry-tls" not found Jul 13 07:02:41.174889 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.174743 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xg64\" (UniqueName: \"kubernetes.io/projected/396e6c4f-4cbc-40ed-b6e8-babbbbc48d39-kube-api-access-6xg64\") pod \"kube-storage-version-migrator-operator-799cb7f958-m7g8b\" (UID: \"396e6c4f-4cbc-40ed-b6e8-babbbbc48d39\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-m7g8b" Jul 13 07:02:41.174889 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.174773 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgkcb\" (UniqueName: \"kubernetes.io/projected/293edc99-2691-4915-ba03-7407d88b89b4-kube-api-access-cgkcb\") pod \"console-operator-7866497494-r878q\" (UID: \"293edc99-2691-4915-ba03-7407d88b89b4\") " pod="openshift-console-operator/console-operator-7866497494-r878q" Jul 13 07:02:41.174889 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:41.174809 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-registry-tls podName:16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d nodeName:}" failed. No retries permitted until 2026-07-13 07:02:41.674791273 +0000 UTC m=+36.510355085 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-registry-tls") pod "image-registry-69599bff85-z8nf7" (UID: "16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d") : secret "image-registry-tls" not found Jul 13 07:02:41.174889 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.174832 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c28d91ed-a056-47d9-ba7c-f789f42dd6cc-default-certificate\") pod \"router-default-7f7c7b8964-kxkhw\" (UID: \"c28d91ed-a056-47d9-ba7c-f789f42dd6cc\") " pod="openshift-ingress/router-default-7f7c7b8964-kxkhw" Jul 13 07:02:41.174889 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.174863 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttlcx\" (UniqueName: \"kubernetes.io/projected/e38c842e-619b-4849-9a60-d65e577ba45e-kube-api-access-ttlcx\") pod \"service-ca-operator-7695fc6c8c-lhs5q\" (UID: \"e38c842e-619b-4849-9a60-d65e577ba45e\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-lhs5q" Jul 13 07:02:41.175326 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.174911 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7c70271-ef81-4393-9206-d7957e5c86e1-tmp-dir\") pod \"dns-default-wg6mf\" (UID: \"a7c70271-ef81-4393-9206-d7957e5c86e1\") " pod="openshift-dns/dns-default-wg6mf" Jul 13 07:02:41.175326 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.174937 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxqrw\" (UniqueName: \"kubernetes.io/projected/c28d91ed-a056-47d9-ba7c-f789f42dd6cc-kube-api-access-dxqrw\") pod \"router-default-7f7c7b8964-kxkhw\" (UID: \"c28d91ed-a056-47d9-ba7c-f789f42dd6cc\") " pod="openshift-ingress/router-default-7f7c7b8964-kxkhw" Jul 13 07:02:41.175326 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.174982 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/05bf1881-84d9-42e6-a7ed-1a348839015d-serving-cert\") pod \"insights-operator-68d9dd94cf-z7x8m\" (UID: \"05bf1881-84d9-42e6-a7ed-1a348839015d\") " pod="openshift-insights/insights-operator-68d9dd94cf-z7x8m" Jul 13 07:02:41.175326 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.175033 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a7c70271-ef81-4393-9206-d7957e5c86e1-config-volume\") pod \"dns-default-wg6mf\" (UID: \"a7c70271-ef81-4393-9206-d7957e5c86e1\") " pod="openshift-dns/dns-default-wg6mf" Jul 13 07:02:41.175326 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.175066 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c28d91ed-a056-47d9-ba7c-f789f42dd6cc-stats-auth\") pod \"router-default-7f7c7b8964-kxkhw\" (UID: \"c28d91ed-a056-47d9-ba7c-f789f42dd6cc\") " pod="openshift-ingress/router-default-7f7c7b8964-kxkhw" Jul 13 07:02:41.175326 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.175095 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5b0bbb6f-a4f2-428a-8c0b-8561d031061c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-74xdm\" (UID: \"5b0bbb6f-a4f2-428a-8c0b-8561d031061c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-74xdm" Jul 13 07:02:41.175326 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.175124 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/05bf1881-84d9-42e6-a7ed-1a348839015d-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-z7x8m\" (UID: \"05bf1881-84d9-42e6-a7ed-1a348839015d\") " pod="openshift-insights/insights-operator-68d9dd94cf-z7x8m" Jul 13 07:02:41.175326 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.175165 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-ca-trust-extracted\") pod \"image-registry-69599bff85-z8nf7\" (UID: \"16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d\") " pod="openshift-image-registry/image-registry-69599bff85-z8nf7" Jul 13 07:02:41.175326 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.175175 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e38c842e-619b-4849-9a60-d65e577ba45e-config\") pod \"service-ca-operator-7695fc6c8c-lhs5q\" (UID: \"e38c842e-619b-4849-9a60-d65e577ba45e\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-lhs5q" Jul 13 07:02:41.175326 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.175211 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzqqc\" (UniqueName: \"kubernetes.io/projected/ddf4f6fd-5259-4c72-94df-245960dc7642-kube-api-access-zzqqc\") pod \"cluster-samples-operator-65db858894-8grn6\" (UID: \"ddf4f6fd-5259-4c72-94df-245960dc7642\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-8grn6" Jul 13 07:02:41.175326 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.175255 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/05bf1881-84d9-42e6-a7ed-1a348839015d-tmp\") pod \"insights-operator-68d9dd94cf-z7x8m\" (UID: \"05bf1881-84d9-42e6-a7ed-1a348839015d\") " pod="openshift-insights/insights-operator-68d9dd94cf-z7x8m" Jul 13 07:02:41.175326 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.175291 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/396e6c4f-4cbc-40ed-b6e8-babbbbc48d39-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-m7g8b\" (UID: \"396e6c4f-4cbc-40ed-b6e8-babbbbc48d39\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-m7g8b" Jul 13 07:02:41.175326 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.175321 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5psx\" (UniqueName: \"kubernetes.io/projected/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-kube-api-access-h5psx\") pod \"image-registry-69599bff85-z8nf7\" (UID: \"16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d\") " pod="openshift-image-registry/image-registry-69599bff85-z8nf7" Jul 13 07:02:41.175955 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.175350 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-bound-sa-token\") pod \"image-registry-69599bff85-z8nf7\" (UID: \"16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d\") " pod="openshift-image-registry/image-registry-69599bff85-z8nf7" Jul 13 07:02:41.175955 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.175375 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/293edc99-2691-4915-ba03-7407d88b89b4-serving-cert\") pod \"console-operator-7866497494-r878q\" (UID: \"293edc99-2691-4915-ba03-7407d88b89b4\") " pod="openshift-console-operator/console-operator-7866497494-r878q" Jul 13 07:02:41.175955 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.175406 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-trusted-ca\") pod \"image-registry-69599bff85-z8nf7\" (UID: \"16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d\") " pod="openshift-image-registry/image-registry-69599bff85-z8nf7" Jul 13 07:02:41.175955 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.175431 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8ssd\" (UniqueName: \"kubernetes.io/projected/5b0bbb6f-a4f2-428a-8c0b-8561d031061c-kube-api-access-p8ssd\") pod \"cluster-monitoring-operator-5f7f494dd4-74xdm\" (UID: \"5b0bbb6f-a4f2-428a-8c0b-8561d031061c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-74xdm" Jul 13 07:02:41.176364 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.176341 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-trusted-ca\") pod \"image-registry-69599bff85-z8nf7\" (UID: \"16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d\") " pod="openshift-image-registry/image-registry-69599bff85-z8nf7" Jul 13 07:02:41.179023 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.178977 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/293edc99-2691-4915-ba03-7407d88b89b4-serving-cert\") pod \"console-operator-7866497494-r878q\" (UID: \"293edc99-2691-4915-ba03-7407d88b89b4\") " pod="openshift-console-operator/console-operator-7866497494-r878q" Jul 13 07:02:41.179124 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.179025 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-image-registry-private-configuration\") pod \"image-registry-69599bff85-z8nf7\" (UID: \"16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d\") " pod="openshift-image-registry/image-registry-69599bff85-z8nf7" Jul 13 07:02:41.179124 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.179107 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-installation-pull-secrets\") pod \"image-registry-69599bff85-z8nf7\" (UID: \"16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d\") " pod="openshift-image-registry/image-registry-69599bff85-z8nf7" Jul 13 07:02:41.189477 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.189439 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdqlp\" (UniqueName: \"kubernetes.io/projected/c4a2167f-7cee-4066-914f-d9c9e4d3bff7-kube-api-access-rdqlp\") pod \"volume-data-source-validator-78495597f6-drnth\" (UID: \"c4a2167f-7cee-4066-914f-d9c9e4d3bff7\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-drnth" Jul 13 07:02:41.190023 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.190000 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgkcb\" (UniqueName: \"kubernetes.io/projected/293edc99-2691-4915-ba03-7407d88b89b4-kube-api-access-cgkcb\") pod \"console-operator-7866497494-r878q\" (UID: \"293edc99-2691-4915-ba03-7407d88b89b4\") " pod="openshift-console-operator/console-operator-7866497494-r878q" Jul 13 07:02:41.190131 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.190074 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5psx\" (UniqueName: \"kubernetes.io/projected/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-kube-api-access-h5psx\") pod \"image-registry-69599bff85-z8nf7\" (UID: \"16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d\") " pod="openshift-image-registry/image-registry-69599bff85-z8nf7" Jul 13 07:02:41.190311 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.190284 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-bound-sa-token\") pod \"image-registry-69599bff85-z8nf7\" (UID: \"16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d\") " pod="openshift-image-registry/image-registry-69599bff85-z8nf7" Jul 13 07:02:41.240546 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.240460 2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xs7q5_2abf1e96-1ba2-4cc1-a632-48e28b374830/konnectivity-agent/0.log" limit=1024 Jul 13 07:02:41.276289 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.276254 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c28d91ed-a056-47d9-ba7c-f789f42dd6cc-default-certificate\") pod \"router-default-7f7c7b8964-kxkhw\" (UID: \"c28d91ed-a056-47d9-ba7c-f789f42dd6cc\") " pod="openshift-ingress/router-default-7f7c7b8964-kxkhw" Jul 13 07:02:41.276488 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.276303 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttlcx\" (UniqueName: \"kubernetes.io/projected/e38c842e-619b-4849-9a60-d65e577ba45e-kube-api-access-ttlcx\") pod \"service-ca-operator-7695fc6c8c-lhs5q\" (UID: \"e38c842e-619b-4849-9a60-d65e577ba45e\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-lhs5q" Jul 13 07:02:41.276488 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.276339 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7c70271-ef81-4393-9206-d7957e5c86e1-tmp-dir\") pod \"dns-default-wg6mf\" (UID: \"a7c70271-ef81-4393-9206-d7957e5c86e1\") " pod="openshift-dns/dns-default-wg6mf" Jul 13 07:02:41.276488 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.276403 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxqrw\" (UniqueName: \"kubernetes.io/projected/c28d91ed-a056-47d9-ba7c-f789f42dd6cc-kube-api-access-dxqrw\") pod \"router-default-7f7c7b8964-kxkhw\" (UID: \"c28d91ed-a056-47d9-ba7c-f789f42dd6cc\") " pod="openshift-ingress/router-default-7f7c7b8964-kxkhw" Jul 13 07:02:41.276488 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.276429 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/05bf1881-84d9-42e6-a7ed-1a348839015d-serving-cert\") pod \"insights-operator-68d9dd94cf-z7x8m\" (UID: \"05bf1881-84d9-42e6-a7ed-1a348839015d\") " pod="openshift-insights/insights-operator-68d9dd94cf-z7x8m" Jul 13 07:02:41.276488 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.276454 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a7c70271-ef81-4393-9206-d7957e5c86e1-config-volume\") pod \"dns-default-wg6mf\" (UID: \"a7c70271-ef81-4393-9206-d7957e5c86e1\") " pod="openshift-dns/dns-default-wg6mf" Jul 13 07:02:41.276770 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.276479 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c28d91ed-a056-47d9-ba7c-f789f42dd6cc-stats-auth\") pod \"router-default-7f7c7b8964-kxkhw\" (UID: \"c28d91ed-a056-47d9-ba7c-f789f42dd6cc\") " pod="openshift-ingress/router-default-7f7c7b8964-kxkhw" Jul 13 07:02:41.276770 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.276517 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5b0bbb6f-a4f2-428a-8c0b-8561d031061c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-74xdm\" (UID: \"5b0bbb6f-a4f2-428a-8c0b-8561d031061c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-74xdm" Jul 13 07:02:41.276770 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.276544 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/05bf1881-84d9-42e6-a7ed-1a348839015d-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-z7x8m\" (UID: \"05bf1881-84d9-42e6-a7ed-1a348839015d\") " pod="openshift-insights/insights-operator-68d9dd94cf-z7x8m" Jul 13 07:02:41.276770 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.276583 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e38c842e-619b-4849-9a60-d65e577ba45e-config\") pod \"service-ca-operator-7695fc6c8c-lhs5q\" (UID: \"e38c842e-619b-4849-9a60-d65e577ba45e\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-lhs5q" Jul 13 07:02:41.276770 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.276606 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzqqc\" (UniqueName: \"kubernetes.io/projected/ddf4f6fd-5259-4c72-94df-245960dc7642-kube-api-access-zzqqc\") pod \"cluster-samples-operator-65db858894-8grn6\" (UID: \"ddf4f6fd-5259-4c72-94df-245960dc7642\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-8grn6" Jul 13 07:02:41.276770 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.276626 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/05bf1881-84d9-42e6-a7ed-1a348839015d-tmp\") pod \"insights-operator-68d9dd94cf-z7x8m\" (UID: \"05bf1881-84d9-42e6-a7ed-1a348839015d\") " pod="openshift-insights/insights-operator-68d9dd94cf-z7x8m" Jul 13 07:02:41.276770 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.276648 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/396e6c4f-4cbc-40ed-b6e8-babbbbc48d39-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-m7g8b\" (UID: \"396e6c4f-4cbc-40ed-b6e8-babbbbc48d39\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-m7g8b" Jul 13 07:02:41.276770 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.276728 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7c70271-ef81-4393-9206-d7957e5c86e1-tmp-dir\") pod \"dns-default-wg6mf\" (UID: \"a7c70271-ef81-4393-9206-d7957e5c86e1\") " pod="openshift-dns/dns-default-wg6mf" Jul 13 07:02:41.276770 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.276743 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8ssd\" (UniqueName: \"kubernetes.io/projected/5b0bbb6f-a4f2-428a-8c0b-8561d031061c-kube-api-access-p8ssd\") pod \"cluster-monitoring-operator-5f7f494dd4-74xdm\" (UID: \"5b0bbb6f-a4f2-428a-8c0b-8561d031061c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-74xdm" Jul 13 07:02:41.277169 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:41.276834 2586 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 13 07:02:41.277169 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:41.276898 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b0bbb6f-a4f2-428a-8c0b-8561d031061c-cluster-monitoring-operator-tls podName:5b0bbb6f-a4f2-428a-8c0b-8561d031061c nodeName:}" failed. No retries permitted until 2026-07-13 07:02:41.776876826 +0000 UTC m=+36.612440652 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5b0bbb6f-a4f2-428a-8c0b-8561d031061c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-74xdm" (UID: "5b0bbb6f-a4f2-428a-8c0b-8561d031061c") : secret "cluster-monitoring-operator-tls" not found Jul 13 07:02:41.277293 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.277274 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/05bf1881-84d9-42e6-a7ed-1a348839015d-tmp\") pod \"insights-operator-68d9dd94cf-z7x8m\" (UID: \"05bf1881-84d9-42e6-a7ed-1a348839015d\") " pod="openshift-insights/insights-operator-68d9dd94cf-z7x8m" Jul 13 07:02:41.277415 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.277386 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fz7m\" (UniqueName: \"kubernetes.io/projected/8d761ca6-7ceb-4a5f-be27-3e668d545e84-kube-api-access-5fz7m\") pod \"network-check-source-869dcfcdcd-bh4jx\" (UID: \"8d761ca6-7ceb-4a5f-be27-3e668d545e84\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-bh4jx" Jul 13 07:02:41.277503 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.277432 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gc2qz\" (UniqueName: \"kubernetes.io/projected/30689818-dbf9-4665-86dc-b81d96b6bed1-kube-api-access-gc2qz\") pod \"ingress-canary-ps2kk\" (UID: \"30689818-dbf9-4665-86dc-b81d96b6bed1\") " pod="openshift-ingress-canary/ingress-canary-ps2kk" Jul 13 07:02:41.277503 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.277434 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/05bf1881-84d9-42e6-a7ed-1a348839015d-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-z7x8m\" (UID: \"05bf1881-84d9-42e6-a7ed-1a348839015d\") " pod="openshift-insights/insights-operator-68d9dd94cf-z7x8m" Jul 13 07:02:41.277503 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.277465 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vn7jz\" (UniqueName: \"kubernetes.io/projected/05bf1881-84d9-42e6-a7ed-1a348839015d-kube-api-access-vn7jz\") pod \"insights-operator-68d9dd94cf-z7x8m\" (UID: \"05bf1881-84d9-42e6-a7ed-1a348839015d\") " pod="openshift-insights/insights-operator-68d9dd94cf-z7x8m" Jul 13 07:02:41.277503 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.277496 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c28d91ed-a056-47d9-ba7c-f789f42dd6cc-service-ca-bundle\") pod \"router-default-7f7c7b8964-kxkhw\" (UID: \"c28d91ed-a056-47d9-ba7c-f789f42dd6cc\") " pod="openshift-ingress/router-default-7f7c7b8964-kxkhw" Jul 13 07:02:41.277770 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.277507 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a7c70271-ef81-4393-9206-d7957e5c86e1-config-volume\") pod \"dns-default-wg6mf\" (UID: \"a7c70271-ef81-4393-9206-d7957e5c86e1\") " pod="openshift-dns/dns-default-wg6mf" Jul 13 07:02:41.277770 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.277529 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e38c842e-619b-4849-9a60-d65e577ba45e-serving-cert\") pod \"service-ca-operator-7695fc6c8c-lhs5q\" (UID: \"e38c842e-619b-4849-9a60-d65e577ba45e\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-lhs5q" Jul 13 07:02:41.277770 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.277554 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/05bf1881-84d9-42e6-a7ed-1a348839015d-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-z7x8m\" (UID: \"05bf1881-84d9-42e6-a7ed-1a348839015d\") " pod="openshift-insights/insights-operator-68d9dd94cf-z7x8m" Jul 13 07:02:41.277770 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.277583 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/396e6c4f-4cbc-40ed-b6e8-babbbbc48d39-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-m7g8b\" (UID: \"396e6c4f-4cbc-40ed-b6e8-babbbbc48d39\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-m7g8b" Jul 13 07:02:41.277770 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.277616 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c28d91ed-a056-47d9-ba7c-f789f42dd6cc-metrics-certs\") pod \"router-default-7f7c7b8964-kxkhw\" (UID: \"c28d91ed-a056-47d9-ba7c-f789f42dd6cc\") " pod="openshift-ingress/router-default-7f7c7b8964-kxkhw" Jul 13 07:02:41.277770 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.277642 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a7c70271-ef81-4393-9206-d7957e5c86e1-metrics-tls\") pod \"dns-default-wg6mf\" (UID: \"a7c70271-ef81-4393-9206-d7957e5c86e1\") " pod="openshift-dns/dns-default-wg6mf" Jul 13 07:02:41.277770 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.277699 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtpfg\" (UniqueName: \"kubernetes.io/projected/a7c70271-ef81-4393-9206-d7957e5c86e1-kube-api-access-wtpfg\") pod \"dns-default-wg6mf\" (UID: \"a7c70271-ef81-4393-9206-d7957e5c86e1\") " pod="openshift-dns/dns-default-wg6mf" Jul 13 07:02:41.277770 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.277727 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/05bf1881-84d9-42e6-a7ed-1a348839015d-snapshots\") pod \"insights-operator-68d9dd94cf-z7x8m\" (UID: \"05bf1881-84d9-42e6-a7ed-1a348839015d\") " pod="openshift-insights/insights-operator-68d9dd94cf-z7x8m" Jul 13 07:02:41.277770 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.277760 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/30689818-dbf9-4665-86dc-b81d96b6bed1-cert\") pod \"ingress-canary-ps2kk\" (UID: \"30689818-dbf9-4665-86dc-b81d96b6bed1\") " pod="openshift-ingress-canary/ingress-canary-ps2kk" Jul 13 07:02:41.278323 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.277796 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e38c842e-619b-4849-9a60-d65e577ba45e-config\") pod \"service-ca-operator-7695fc6c8c-lhs5q\" (UID: \"e38c842e-619b-4849-9a60-d65e577ba45e\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-lhs5q" Jul 13 07:02:41.278323 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:41.277921 2586 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 07:02:41.278323 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:41.277929 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c28d91ed-a056-47d9-ba7c-f789f42dd6cc-service-ca-bundle podName:c28d91ed-a056-47d9-ba7c-f789f42dd6cc nodeName:}" failed. No retries permitted until 2026-07-13 07:02:41.777913167 +0000 UTC m=+36.613476991 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c28d91ed-a056-47d9-ba7c-f789f42dd6cc-service-ca-bundle") pod "router-default-7f7c7b8964-kxkhw" (UID: "c28d91ed-a056-47d9-ba7c-f789f42dd6cc") : configmap references non-existent config key: service-ca.crt Jul 13 07:02:41.278323 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:41.278011 2586 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 13 07:02:41.278323 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:41.278054 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c28d91ed-a056-47d9-ba7c-f789f42dd6cc-metrics-certs podName:c28d91ed-a056-47d9-ba7c-f789f42dd6cc nodeName:}" failed. No retries permitted until 2026-07-13 07:02:41.778043137 +0000 UTC m=+36.613606957 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c28d91ed-a056-47d9-ba7c-f789f42dd6cc-metrics-certs") pod "router-default-7f7c7b8964-kxkhw" (UID: "c28d91ed-a056-47d9-ba7c-f789f42dd6cc") : secret "router-metrics-certs-default" not found Jul 13 07:02:41.278323 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:41.278105 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7c70271-ef81-4393-9206-d7957e5c86e1-metrics-tls podName:a7c70271-ef81-4393-9206-d7957e5c86e1 nodeName:}" failed. No retries permitted until 2026-07-13 07:02:41.778074954 +0000 UTC m=+36.613638766 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a7c70271-ef81-4393-9206-d7957e5c86e1-metrics-tls") pod "dns-default-wg6mf" (UID: "a7c70271-ef81-4393-9206-d7957e5c86e1") : secret "dns-default-metrics-tls" not found Jul 13 07:02:41.278323 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.278275 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/396e6c4f-4cbc-40ed-b6e8-babbbbc48d39-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-m7g8b\" (UID: \"396e6c4f-4cbc-40ed-b6e8-babbbbc48d39\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-m7g8b" Jul 13 07:02:41.278732 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.278342 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ddf4f6fd-5259-4c72-94df-245960dc7642-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-8grn6\" (UID: \"ddf4f6fd-5259-4c72-94df-245960dc7642\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-8grn6" Jul 13 07:02:41.278732 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.278384 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/5b0bbb6f-a4f2-428a-8c0b-8561d031061c-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-74xdm\" (UID: \"5b0bbb6f-a4f2-428a-8c0b-8561d031061c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-74xdm" Jul 13 07:02:41.278732 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.278414 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xg64\" (UniqueName: \"kubernetes.io/projected/396e6c4f-4cbc-40ed-b6e8-babbbbc48d39-kube-api-access-6xg64\") pod \"kube-storage-version-migrator-operator-799cb7f958-m7g8b\" (UID: \"396e6c4f-4cbc-40ed-b6e8-babbbbc48d39\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-m7g8b" Jul 13 07:02:41.278894 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:41.278787 2586 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 13 07:02:41.278894 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.278816 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/05bf1881-84d9-42e6-a7ed-1a348839015d-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-z7x8m\" (UID: \"05bf1881-84d9-42e6-a7ed-1a348839015d\") " pod="openshift-insights/insights-operator-68d9dd94cf-z7x8m" Jul 13 07:02:41.278894 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:41.278836 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddf4f6fd-5259-4c72-94df-245960dc7642-samples-operator-tls podName:ddf4f6fd-5259-4c72-94df-245960dc7642 nodeName:}" failed. No retries permitted until 2026-07-13 07:02:41.778822871 +0000 UTC m=+36.614386688 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ddf4f6fd-5259-4c72-94df-245960dc7642-samples-operator-tls") pod "cluster-samples-operator-65db858894-8grn6" (UID: "ddf4f6fd-5259-4c72-94df-245960dc7642") : secret "samples-operator-tls" not found Jul 13 07:02:41.279474 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.279454 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/5b0bbb6f-a4f2-428a-8c0b-8561d031061c-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-74xdm\" (UID: \"5b0bbb6f-a4f2-428a-8c0b-8561d031061c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-74xdm" Jul 13 07:02:41.279596 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.279574 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/05bf1881-84d9-42e6-a7ed-1a348839015d-snapshots\") pod \"insights-operator-68d9dd94cf-z7x8m\" (UID: \"05bf1881-84d9-42e6-a7ed-1a348839015d\") " pod="openshift-insights/insights-operator-68d9dd94cf-z7x8m" Jul 13 07:02:41.280057 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.280035 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/396e6c4f-4cbc-40ed-b6e8-babbbbc48d39-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-m7g8b\" (UID: \"396e6c4f-4cbc-40ed-b6e8-babbbbc48d39\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-m7g8b" Jul 13 07:02:41.280184 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.280167 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c28d91ed-a056-47d9-ba7c-f789f42dd6cc-default-certificate\") pod \"router-default-7f7c7b8964-kxkhw\" (UID: \"c28d91ed-a056-47d9-ba7c-f789f42dd6cc\") " pod="openshift-ingress/router-default-7f7c7b8964-kxkhw" Jul 13 07:02:41.280454 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.280417 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/05bf1881-84d9-42e6-a7ed-1a348839015d-serving-cert\") pod \"insights-operator-68d9dd94cf-z7x8m\" (UID: \"05bf1881-84d9-42e6-a7ed-1a348839015d\") " pod="openshift-insights/insights-operator-68d9dd94cf-z7x8m" Jul 13 07:02:41.280543 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.280472 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/30689818-dbf9-4665-86dc-b81d96b6bed1-cert\") pod \"ingress-canary-ps2kk\" (UID: \"30689818-dbf9-4665-86dc-b81d96b6bed1\") " pod="openshift-ingress-canary/ingress-canary-ps2kk" Jul 13 07:02:41.280597 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.280553 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e38c842e-619b-4849-9a60-d65e577ba45e-serving-cert\") pod \"service-ca-operator-7695fc6c8c-lhs5q\" (UID: \"e38c842e-619b-4849-9a60-d65e577ba45e\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-lhs5q" Jul 13 07:02:41.282191 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.282169 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c28d91ed-a056-47d9-ba7c-f789f42dd6cc-stats-auth\") pod \"router-default-7f7c7b8964-kxkhw\" (UID: \"c28d91ed-a056-47d9-ba7c-f789f42dd6cc\") " pod="openshift-ingress/router-default-7f7c7b8964-kxkhw" Jul 13 07:02:41.289027 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.288706 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxqrw\" (UniqueName: \"kubernetes.io/projected/c28d91ed-a056-47d9-ba7c-f789f42dd6cc-kube-api-access-dxqrw\") pod \"router-default-7f7c7b8964-kxkhw\" (UID: \"c28d91ed-a056-47d9-ba7c-f789f42dd6cc\") " pod="openshift-ingress/router-default-7f7c7b8964-kxkhw" Jul 13 07:02:41.289027 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.288977 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8ssd\" (UniqueName: \"kubernetes.io/projected/5b0bbb6f-a4f2-428a-8c0b-8561d031061c-kube-api-access-p8ssd\") pod \"cluster-monitoring-operator-5f7f494dd4-74xdm\" (UID: \"5b0bbb6f-a4f2-428a-8c0b-8561d031061c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-74xdm" Jul 13 07:02:41.290278 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.290072 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttlcx\" (UniqueName: \"kubernetes.io/projected/e38c842e-619b-4849-9a60-d65e577ba45e-kube-api-access-ttlcx\") pod \"service-ca-operator-7695fc6c8c-lhs5q\" (UID: \"e38c842e-619b-4849-9a60-d65e577ba45e\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-lhs5q" Jul 13 07:02:41.290629 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.290577 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzqqc\" (UniqueName: \"kubernetes.io/projected/ddf4f6fd-5259-4c72-94df-245960dc7642-kube-api-access-zzqqc\") pod \"cluster-samples-operator-65db858894-8grn6\" (UID: \"ddf4f6fd-5259-4c72-94df-245960dc7642\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-8grn6" Jul 13 07:02:41.294801 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.291969 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gc2qz\" (UniqueName: \"kubernetes.io/projected/30689818-dbf9-4665-86dc-b81d96b6bed1-kube-api-access-gc2qz\") pod \"ingress-canary-ps2kk\" (UID: \"30689818-dbf9-4665-86dc-b81d96b6bed1\") " pod="openshift-ingress-canary/ingress-canary-ps2kk" Jul 13 07:02:41.294801 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.291975 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xg64\" (UniqueName: \"kubernetes.io/projected/396e6c4f-4cbc-40ed-b6e8-babbbbc48d39-kube-api-access-6xg64\") pod \"kube-storage-version-migrator-operator-799cb7f958-m7g8b\" (UID: \"396e6c4f-4cbc-40ed-b6e8-babbbbc48d39\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-m7g8b" Jul 13 07:02:41.294801 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.292007 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fz7m\" (UniqueName: \"kubernetes.io/projected/8d761ca6-7ceb-4a5f-be27-3e668d545e84-kube-api-access-5fz7m\") pod \"network-check-source-869dcfcdcd-bh4jx\" (UID: \"8d761ca6-7ceb-4a5f-be27-3e668d545e84\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-bh4jx" Jul 13 07:02:41.294801 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.292140 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vn7jz\" (UniqueName: \"kubernetes.io/projected/05bf1881-84d9-42e6-a7ed-1a348839015d-kube-api-access-vn7jz\") pod \"insights-operator-68d9dd94cf-z7x8m\" (UID: \"05bf1881-84d9-42e6-a7ed-1a348839015d\") " pod="openshift-insights/insights-operator-68d9dd94cf-z7x8m" Jul 13 07:02:41.294801 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.293938 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-6fw29"] Jul 13 07:02:41.297155 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.297134 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtpfg\" (UniqueName: \"kubernetes.io/projected/a7c70271-ef81-4393-9206-d7957e5c86e1-kube-api-access-wtpfg\") pod \"dns-default-wg6mf\" (UID: \"a7c70271-ef81-4393-9206-d7957e5c86e1\") " pod="openshift-dns/dns-default-wg6mf" Jul 13 07:02:41.297812 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.297791 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-6fw29" Jul 13 07:02:41.302259 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.302207 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 13 07:02:41.302259 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.302232 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-d87kp\"" Jul 13 07:02:41.302423 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.302260 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 13 07:02:41.307433 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.307409 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-6fw29"] Jul 13 07:02:41.317773 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.317743 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-drnth" Jul 13 07:02:41.329710 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.329667 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-r878q" Jul 13 07:02:41.336542 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.336517 2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xs7q5_2abf1e96-1ba2-4cc1-a632-48e28b374830/konnectivity-agent/0.log" limit=1024 Jul 13 07:02:41.348379 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.348353 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-z7x8m" Jul 13 07:02:41.359229 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.359202 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-m7g8b" Jul 13 07:02:41.366062 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.366036 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-lhs5q" Jul 13 07:02:41.379017 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.378981 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ec04b338-54ca-471f-b138-eabbf7009550-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-6fw29\" (UID: \"ec04b338-54ca-471f-b138-eabbf7009550\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-6fw29" Jul 13 07:02:41.379172 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.379060 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ec04b338-54ca-471f-b138-eabbf7009550-nginx-conf\") pod \"networking-console-plugin-6cf856758c-6fw29\" (UID: \"ec04b338-54ca-471f-b138-eabbf7009550\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-6fw29" Jul 13 07:02:41.399062 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.399024 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-bh4jx" Jul 13 07:02:41.413144 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.413106 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ps2kk" Jul 13 07:02:41.416822 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.416762 2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xs7q5_2abf1e96-1ba2-4cc1-a632-48e28b374830/konnectivity-agent/0.log" limit=1024 Jul 13 07:02:41.479934 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.479899 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ec04b338-54ca-471f-b138-eabbf7009550-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-6fw29\" (UID: \"ec04b338-54ca-471f-b138-eabbf7009550\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-6fw29" Jul 13 07:02:41.480123 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.479982 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ec04b338-54ca-471f-b138-eabbf7009550-nginx-conf\") pod \"networking-console-plugin-6cf856758c-6fw29\" (UID: \"ec04b338-54ca-471f-b138-eabbf7009550\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-6fw29" Jul 13 07:02:41.480123 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:41.480070 2586 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 13 07:02:41.480232 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:41.480152 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec04b338-54ca-471f-b138-eabbf7009550-networking-console-plugin-cert podName:ec04b338-54ca-471f-b138-eabbf7009550 nodeName:}" failed. No retries permitted until 2026-07-13 07:02:41.980133837 +0000 UTC m=+36.815697655 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ec04b338-54ca-471f-b138-eabbf7009550-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-6fw29" (UID: "ec04b338-54ca-471f-b138-eabbf7009550") : secret "networking-console-plugin-cert" not found Jul 13 07:02:41.480776 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.480753 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ec04b338-54ca-471f-b138-eabbf7009550-nginx-conf\") pod \"networking-console-plugin-6cf856758c-6fw29\" (UID: \"ec04b338-54ca-471f-b138-eabbf7009550\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-6fw29" Jul 13 07:02:41.516912 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.516826 2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xs7q5_2abf1e96-1ba2-4cc1-a632-48e28b374830/konnectivity-agent/0.log" limit=1024 Jul 13 07:02:41.618404 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.618369 2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xs7q5_2abf1e96-1ba2-4cc1-a632-48e28b374830/konnectivity-agent/0.log" limit=1024 Jul 13 07:02:41.670949 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.670909 2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xs7q5_2abf1e96-1ba2-4cc1-a632-48e28b374830/konnectivity-agent/0.log" limit=1024 Jul 13 07:02:41.682214 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.682176 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-registry-tls\") pod \"image-registry-69599bff85-z8nf7\" (UID: \"16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d\") " pod="openshift-image-registry/image-registry-69599bff85-z8nf7" Jul 13 07:02:41.682396 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:41.682354 2586 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 13 07:02:41.682396 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:41.682371 2586 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69599bff85-z8nf7: secret "image-registry-tls" not found Jul 13 07:02:41.682483 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:41.682431 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-registry-tls podName:16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d nodeName:}" failed. No retries permitted until 2026-07-13 07:02:42.682411042 +0000 UTC m=+37.517974862 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-registry-tls") pod "image-registry-69599bff85-z8nf7" (UID: "16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d") : secret "image-registry-tls" not found Jul 13 07:02:41.783486 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.783401 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a7c70271-ef81-4393-9206-d7957e5c86e1-metrics-tls\") pod \"dns-default-wg6mf\" (UID: \"a7c70271-ef81-4393-9206-d7957e5c86e1\") " pod="openshift-dns/dns-default-wg6mf" Jul 13 07:02:41.783486 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.783474 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ddf4f6fd-5259-4c72-94df-245960dc7642-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-8grn6\" (UID: \"ddf4f6fd-5259-4c72-94df-245960dc7642\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-8grn6" Jul 13 07:02:41.783732 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.783551 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5b0bbb6f-a4f2-428a-8c0b-8561d031061c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-74xdm\" (UID: \"5b0bbb6f-a4f2-428a-8c0b-8561d031061c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-74xdm" Jul 13 07:02:41.783732 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:41.783575 2586 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 07:02:41.783732 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.783628 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c28d91ed-a056-47d9-ba7c-f789f42dd6cc-service-ca-bundle\") pod \"router-default-7f7c7b8964-kxkhw\" (UID: \"c28d91ed-a056-47d9-ba7c-f789f42dd6cc\") " pod="openshift-ingress/router-default-7f7c7b8964-kxkhw" Jul 13 07:02:41.783732 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:41.783647 2586 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 13 07:02:41.783732 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:41.783651 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7c70271-ef81-4393-9206-d7957e5c86e1-metrics-tls podName:a7c70271-ef81-4393-9206-d7957e5c86e1 nodeName:}" failed. No retries permitted until 2026-07-13 07:02:42.783632315 +0000 UTC m=+37.619196135 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a7c70271-ef81-4393-9206-d7957e5c86e1-metrics-tls") pod "dns-default-wg6mf" (UID: "a7c70271-ef81-4393-9206-d7957e5c86e1") : secret "dns-default-metrics-tls" not found Jul 13 07:02:41.783732 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:41.783647 2586 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 13 07:02:41.784023 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:41.783774 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c28d91ed-a056-47d9-ba7c-f789f42dd6cc-service-ca-bundle podName:c28d91ed-a056-47d9-ba7c-f789f42dd6cc nodeName:}" failed. No retries permitted until 2026-07-13 07:02:42.783755 +0000 UTC m=+37.619318813 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c28d91ed-a056-47d9-ba7c-f789f42dd6cc-service-ca-bundle") pod "router-default-7f7c7b8964-kxkhw" (UID: "c28d91ed-a056-47d9-ba7c-f789f42dd6cc") : configmap references non-existent config key: service-ca.crt Jul 13 07:02:41.784023 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:41.783802 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b0bbb6f-a4f2-428a-8c0b-8561d031061c-cluster-monitoring-operator-tls podName:5b0bbb6f-a4f2-428a-8c0b-8561d031061c nodeName:}" failed. No retries permitted until 2026-07-13 07:02:42.783791118 +0000 UTC m=+37.619354927 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5b0bbb6f-a4f2-428a-8c0b-8561d031061c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-74xdm" (UID: "5b0bbb6f-a4f2-428a-8c0b-8561d031061c") : secret "cluster-monitoring-operator-tls" not found Jul 13 07:02:41.784023 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:41.783839 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddf4f6fd-5259-4c72-94df-245960dc7642-samples-operator-tls podName:ddf4f6fd-5259-4c72-94df-245960dc7642 nodeName:}" failed. No retries permitted until 2026-07-13 07:02:42.783816056 +0000 UTC m=+37.619379866 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ddf4f6fd-5259-4c72-94df-245960dc7642-samples-operator-tls") pod "cluster-samples-operator-65db858894-8grn6" (UID: "ddf4f6fd-5259-4c72-94df-245960dc7642") : secret "samples-operator-tls" not found Jul 13 07:02:41.784023 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.783888 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c28d91ed-a056-47d9-ba7c-f789f42dd6cc-metrics-certs\") pod \"router-default-7f7c7b8964-kxkhw\" (UID: \"c28d91ed-a056-47d9-ba7c-f789f42dd6cc\") " pod="openshift-ingress/router-default-7f7c7b8964-kxkhw" Jul 13 07:02:41.784023 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:41.783995 2586 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 13 07:02:41.784203 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:41.784054 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c28d91ed-a056-47d9-ba7c-f789f42dd6cc-metrics-certs podName:c28d91ed-a056-47d9-ba7c-f789f42dd6cc nodeName:}" failed. No retries permitted until 2026-07-13 07:02:42.784038455 +0000 UTC m=+37.619602267 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c28d91ed-a056-47d9-ba7c-f789f42dd6cc-metrics-certs") pod "router-default-7f7c7b8964-kxkhw" (UID: "c28d91ed-a056-47d9-ba7c-f789f42dd6cc") : secret "router-metrics-certs-default" not found Jul 13 07:02:41.985718 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:41.985184 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ec04b338-54ca-471f-b138-eabbf7009550-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-6fw29\" (UID: \"ec04b338-54ca-471f-b138-eabbf7009550\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-6fw29" Jul 13 07:02:41.985718 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:41.985591 2586 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 13 07:02:41.985718 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:41.985659 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec04b338-54ca-471f-b138-eabbf7009550-networking-console-plugin-cert podName:ec04b338-54ca-471f-b138-eabbf7009550 nodeName:}" failed. No retries permitted until 2026-07-13 07:02:42.985638493 +0000 UTC m=+37.821202315 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ec04b338-54ca-471f-b138-eabbf7009550-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-6fw29" (UID: "ec04b338-54ca-471f-b138-eabbf7009550") : secret "networking-console-plugin-cert" not found Jul 13 07:02:42.206115 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:42.205763 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-lhs5q"] Jul 13 07:02:42.222605 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:42.222531 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ps2kk"] Jul 13 07:02:42.227245 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:42.227206 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-r878q"] Jul 13 07:02:42.231154 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:42.231128 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-drnth"] Jul 13 07:02:42.234602 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:42.234533 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-z7x8m"] Jul 13 07:02:42.238760 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:42.238733 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-m7g8b"] Jul 13 07:02:42.245227 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:42.245101 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-bh4jx"] Jul 13 07:02:42.248428 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:42.248398 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30689818_dbf9_4665_86dc_b81d96b6bed1.slice/crio-67af393f7077c88a321b75e50386c95cfc9bd2d3289add5276284db333a8b06d WatchSource:0}: Error finding container 67af393f7077c88a321b75e50386c95cfc9bd2d3289add5276284db333a8b06d: Status 404 returned error can't find the container with id 67af393f7077c88a321b75e50386c95cfc9bd2d3289add5276284db333a8b06d Jul 13 07:02:42.249658 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:42.249609 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod396e6c4f_4cbc_40ed_b6e8_babbbbc48d39.slice/crio-39c08ca21881342e8d441c68a1ccc309ab715e07e125abbba456de82b3b26794 WatchSource:0}: Error finding container 39c08ca21881342e8d441c68a1ccc309ab715e07e125abbba456de82b3b26794: Status 404 returned error can't find the container with id 39c08ca21881342e8d441c68a1ccc309ab715e07e125abbba456de82b3b26794 Jul 13 07:02:42.249997 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:42.249917 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod293edc99_2691_4915_ba03_7407d88b89b4.slice/crio-00f53a8412b66e25c5b7baa803a14e31d02cd071039be2bd9da42423e4f06398 WatchSource:0}: Error finding container 00f53a8412b66e25c5b7baa803a14e31d02cd071039be2bd9da42423e4f06398: Status 404 returned error can't find the container with id 00f53a8412b66e25c5b7baa803a14e31d02cd071039be2bd9da42423e4f06398 Jul 13 07:02:42.252502 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:42.251718 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4a2167f_7cee_4066_914f_d9c9e4d3bff7.slice/crio-03a2906762f4eb61087df519d382f0f2777d7b84cea688748c0c3331e82cebf4 WatchSource:0}: Error finding container 03a2906762f4eb61087df519d382f0f2777d7b84cea688748c0c3331e82cebf4: Status 404 returned error can't find the container with id 03a2906762f4eb61087df519d382f0f2777d7b84cea688748c0c3331e82cebf4 Jul 13 07:02:42.252617 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:42.252569 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05bf1881_84d9_42e6_a7ed_1a348839015d.slice/crio-8a5f06245cdb0df843d03a9aeb9dae150660706d4949456786caecea97cfcd89 WatchSource:0}: Error finding container 8a5f06245cdb0df843d03a9aeb9dae150660706d4949456786caecea97cfcd89: Status 404 returned error can't find the container with id 8a5f06245cdb0df843d03a9aeb9dae150660706d4949456786caecea97cfcd89 Jul 13 07:02:42.253825 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:02:42.253444 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d761ca6_7ceb_4a5f_be27_3e668d545e84.slice/crio-0ee0c674c882255c4d5125ed289ee47b60492cfe3cc39d246be3605e77eb36f9 WatchSource:0}: Error finding container 0ee0c674c882255c4d5125ed289ee47b60492cfe3cc39d246be3605e77eb36f9: Status 404 returned error can't find the container with id 0ee0c674c882255c4d5125ed289ee47b60492cfe3cc39d246be3605e77eb36f9 Jul 13 07:02:42.696264 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:42.696231 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-registry-tls\") pod \"image-registry-69599bff85-z8nf7\" (UID: \"16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d\") " pod="openshift-image-registry/image-registry-69599bff85-z8nf7" Jul 13 07:02:42.696424 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:42.696391 2586 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 13 07:02:42.696424 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:42.696409 2586 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69599bff85-z8nf7: secret "image-registry-tls" not found Jul 13 07:02:42.696495 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:42.696464 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-registry-tls podName:16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d nodeName:}" failed. No retries permitted until 2026-07-13 07:02:44.696449881 +0000 UTC m=+39.532013695 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-registry-tls") pod "image-registry-69599bff85-z8nf7" (UID: "16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d") : secret "image-registry-tls" not found Jul 13 07:02:42.797463 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:42.797410 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ddf4f6fd-5259-4c72-94df-245960dc7642-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-8grn6\" (UID: \"ddf4f6fd-5259-4c72-94df-245960dc7642\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-8grn6" Jul 13 07:02:42.797663 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:42.797506 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5b0bbb6f-a4f2-428a-8c0b-8561d031061c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-74xdm\" (UID: \"5b0bbb6f-a4f2-428a-8c0b-8561d031061c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-74xdm" Jul 13 07:02:42.797663 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:42.797551 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c28d91ed-a056-47d9-ba7c-f789f42dd6cc-service-ca-bundle\") pod \"router-default-7f7c7b8964-kxkhw\" (UID: \"c28d91ed-a056-47d9-ba7c-f789f42dd6cc\") " pod="openshift-ingress/router-default-7f7c7b8964-kxkhw" Jul 13 07:02:42.797663 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:42.797574 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c28d91ed-a056-47d9-ba7c-f789f42dd6cc-metrics-certs\") pod \"router-default-7f7c7b8964-kxkhw\" (UID: \"c28d91ed-a056-47d9-ba7c-f789f42dd6cc\") " pod="openshift-ingress/router-default-7f7c7b8964-kxkhw" Jul 13 07:02:42.797663 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:42.797585 2586 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 13 07:02:42.797663 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:42.797648 2586 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 13 07:02:42.797663 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:42.797654 2586 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 13 07:02:42.797954 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:42.797652 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddf4f6fd-5259-4c72-94df-245960dc7642-samples-operator-tls podName:ddf4f6fd-5259-4c72-94df-245960dc7642 nodeName:}" failed. No retries permitted until 2026-07-13 07:02:44.797637276 +0000 UTC m=+39.633201086 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ddf4f6fd-5259-4c72-94df-245960dc7642-samples-operator-tls") pod "cluster-samples-operator-65db858894-8grn6" (UID: "ddf4f6fd-5259-4c72-94df-245960dc7642") : secret "samples-operator-tls" not found Jul 13 07:02:42.797954 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:42.797708 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c28d91ed-a056-47d9-ba7c-f789f42dd6cc-service-ca-bundle podName:c28d91ed-a056-47d9-ba7c-f789f42dd6cc nodeName:}" failed. No retries permitted until 2026-07-13 07:02:44.797697415 +0000 UTC m=+39.633261228 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c28d91ed-a056-47d9-ba7c-f789f42dd6cc-service-ca-bundle") pod "router-default-7f7c7b8964-kxkhw" (UID: "c28d91ed-a056-47d9-ba7c-f789f42dd6cc") : configmap references non-existent config key: service-ca.crt Jul 13 07:02:42.797954 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:42.797590 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a7c70271-ef81-4393-9206-d7957e5c86e1-metrics-tls\") pod \"dns-default-wg6mf\" (UID: \"a7c70271-ef81-4393-9206-d7957e5c86e1\") " pod="openshift-dns/dns-default-wg6mf" Jul 13 07:02:42.797954 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:42.797717 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c28d91ed-a056-47d9-ba7c-f789f42dd6cc-metrics-certs podName:c28d91ed-a056-47d9-ba7c-f789f42dd6cc nodeName:}" failed. No retries permitted until 2026-07-13 07:02:44.797712367 +0000 UTC m=+39.633276176 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c28d91ed-a056-47d9-ba7c-f789f42dd6cc-metrics-certs") pod "router-default-7f7c7b8964-kxkhw" (UID: "c28d91ed-a056-47d9-ba7c-f789f42dd6cc") : secret "router-metrics-certs-default" not found Jul 13 07:02:42.797954 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:42.797726 2586 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 07:02:42.797954 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:42.797731 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b0bbb6f-a4f2-428a-8c0b-8561d031061c-cluster-monitoring-operator-tls podName:5b0bbb6f-a4f2-428a-8c0b-8561d031061c nodeName:}" failed. No retries permitted until 2026-07-13 07:02:44.797726321 +0000 UTC m=+39.633290130 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5b0bbb6f-a4f2-428a-8c0b-8561d031061c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-74xdm" (UID: "5b0bbb6f-a4f2-428a-8c0b-8561d031061c") : secret "cluster-monitoring-operator-tls" not found Jul 13 07:02:42.797954 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:42.797779 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7c70271-ef81-4393-9206-d7957e5c86e1-metrics-tls podName:a7c70271-ef81-4393-9206-d7957e5c86e1 nodeName:}" failed. No retries permitted until 2026-07-13 07:02:44.79776864 +0000 UTC m=+39.633332452 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a7c70271-ef81-4393-9206-d7957e5c86e1-metrics-tls") pod "dns-default-wg6mf" (UID: "a7c70271-ef81-4393-9206-d7957e5c86e1") : secret "dns-default-metrics-tls" not found Jul 13 07:02:42.799441 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:42.799417 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lb8th" Jul 13 07:02:42.799534 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:42.799417 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6xdjl" Jul 13 07:02:42.799640 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:42.799624 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-znhkv" Jul 13 07:02:42.801497 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:42.801477 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 13 07:02:42.802007 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:42.801987 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 13 07:02:42.802007 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:42.801999 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-z8tdw\"" Jul 13 07:02:42.802145 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:42.801987 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fh46b\"" Jul 13 07:02:43.000747 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:43.000638 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ec04b338-54ca-471f-b138-eabbf7009550-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-6fw29\" (UID: \"ec04b338-54ca-471f-b138-eabbf7009550\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-6fw29" Jul 13 07:02:43.001267 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:43.000868 2586 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 13 07:02:43.001267 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:43.000944 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec04b338-54ca-471f-b138-eabbf7009550-networking-console-plugin-cert podName:ec04b338-54ca-471f-b138-eabbf7009550 nodeName:}" failed. No retries permitted until 2026-07-13 07:02:45.000923678 +0000 UTC m=+39.836487489 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ec04b338-54ca-471f-b138-eabbf7009550-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-6fw29" (UID: "ec04b338-54ca-471f-b138-eabbf7009550") : secret "networking-console-plugin-cert" not found Jul 13 07:02:43.009864 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:43.008994 2586 generic.go:356] "Generic (PLEG): container finished" podID="2d5a84b8-155e-4dc0-8526-19a67235d08f" containerID="f36d7d036625ac297ca2bf83163f0eb7618dc78b31367a4f9184b414319b9afc" exitCode=0 Jul 13 07:02:43.009864 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:43.009088 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-56rlp" event={"ID":"2d5a84b8-155e-4dc0-8526-19a67235d08f","Type":"ContainerDied","Data":"f36d7d036625ac297ca2bf83163f0eb7618dc78b31367a4f9184b414319b9afc"} Jul 13 07:02:43.012099 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:43.012020 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-drnth" event={"ID":"c4a2167f-7cee-4066-914f-d9c9e4d3bff7","Type":"ContainerStarted","Data":"03a2906762f4eb61087df519d382f0f2777d7b84cea688748c0c3331e82cebf4"} Jul 13 07:02:43.014285 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:43.014254 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-r878q" event={"ID":"293edc99-2691-4915-ba03-7407d88b89b4","Type":"ContainerStarted","Data":"00f53a8412b66e25c5b7baa803a14e31d02cd071039be2bd9da42423e4f06398"} Jul 13 07:02:43.017032 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:43.017007 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-bh4jx" event={"ID":"8d761ca6-7ceb-4a5f-be27-3e668d545e84","Type":"ContainerStarted","Data":"0ee0c674c882255c4d5125ed289ee47b60492cfe3cc39d246be3605e77eb36f9"} Jul 13 07:02:43.018697 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:43.018642 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-z7x8m" event={"ID":"05bf1881-84d9-42e6-a7ed-1a348839015d","Type":"ContainerStarted","Data":"8a5f06245cdb0df843d03a9aeb9dae150660706d4949456786caecea97cfcd89"} Jul 13 07:02:43.021027 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:43.020990 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-m7g8b" event={"ID":"396e6c4f-4cbc-40ed-b6e8-babbbbc48d39","Type":"ContainerStarted","Data":"39c08ca21881342e8d441c68a1ccc309ab715e07e125abbba456de82b3b26794"} Jul 13 07:02:43.022767 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:43.022741 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-lhs5q" event={"ID":"e38c842e-619b-4849-9a60-d65e577ba45e","Type":"ContainerStarted","Data":"1bac50bb59ff52e567aeeef75c4b48a8e16229b532f75ebf3e065ab97c3ffdb2"} Jul 13 07:02:43.025326 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:43.025266 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ps2kk" event={"ID":"30689818-dbf9-4665-86dc-b81d96b6bed1","Type":"ContainerStarted","Data":"67af393f7077c88a321b75e50386c95cfc9bd2d3289add5276284db333a8b06d"} Jul 13 07:02:43.960322 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:43.959845 2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xs7q5_2abf1e96-1ba2-4cc1-a632-48e28b374830/konnectivity-agent/0.log" limit=1024 Jul 13 07:02:44.014579 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:44.014444 2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xs7q5_2abf1e96-1ba2-4cc1-a632-48e28b374830/konnectivity-agent/0.log" limit=1024 Jul 13 07:02:44.037756 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:44.037225 2586 generic.go:356] "Generic (PLEG): container finished" podID="2d5a84b8-155e-4dc0-8526-19a67235d08f" containerID="8bd79d30e9a9a37bf92f3ebbb77242b98cd00bed0ffd92635ef669ce37b89378" exitCode=0 Jul 13 07:02:44.037756 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:44.037306 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-56rlp" event={"ID":"2d5a84b8-155e-4dc0-8526-19a67235d08f","Type":"ContainerDied","Data":"8bd79d30e9a9a37bf92f3ebbb77242b98cd00bed0ffd92635ef669ce37b89378"} Jul 13 07:02:44.720229 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:44.720146 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-registry-tls\") pod \"image-registry-69599bff85-z8nf7\" (UID: \"16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d\") " pod="openshift-image-registry/image-registry-69599bff85-z8nf7" Jul 13 07:02:44.720467 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:44.720449 2586 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 13 07:02:44.720541 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:44.720471 2586 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69599bff85-z8nf7: secret "image-registry-tls" not found Jul 13 07:02:44.720541 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:44.720528 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-registry-tls podName:16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d nodeName:}" failed. No retries permitted until 2026-07-13 07:02:48.720509852 +0000 UTC m=+43.556073667 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-registry-tls") pod "image-registry-69599bff85-z8nf7" (UID: "16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d") : secret "image-registry-tls" not found Jul 13 07:02:44.821722 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:44.820980 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5b0bbb6f-a4f2-428a-8c0b-8561d031061c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-74xdm\" (UID: \"5b0bbb6f-a4f2-428a-8c0b-8561d031061c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-74xdm" Jul 13 07:02:44.821722 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:44.821069 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c28d91ed-a056-47d9-ba7c-f789f42dd6cc-service-ca-bundle\") pod \"router-default-7f7c7b8964-kxkhw\" (UID: \"c28d91ed-a056-47d9-ba7c-f789f42dd6cc\") " pod="openshift-ingress/router-default-7f7c7b8964-kxkhw" Jul 13 07:02:44.821722 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:44.821215 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c28d91ed-a056-47d9-ba7c-f789f42dd6cc-service-ca-bundle podName:c28d91ed-a056-47d9-ba7c-f789f42dd6cc nodeName:}" failed. No retries permitted until 2026-07-13 07:02:48.821197803 +0000 UTC m=+43.656761626 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c28d91ed-a056-47d9-ba7c-f789f42dd6cc-service-ca-bundle") pod "router-default-7f7c7b8964-kxkhw" (UID: "c28d91ed-a056-47d9-ba7c-f789f42dd6cc") : configmap references non-existent config key: service-ca.crt Jul 13 07:02:44.821722 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:44.821229 2586 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 13 07:02:44.821722 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:44.821295 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b0bbb6f-a4f2-428a-8c0b-8561d031061c-cluster-monitoring-operator-tls podName:5b0bbb6f-a4f2-428a-8c0b-8561d031061c nodeName:}" failed. No retries permitted until 2026-07-13 07:02:48.821277165 +0000 UTC m=+43.656840977 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5b0bbb6f-a4f2-428a-8c0b-8561d031061c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-74xdm" (UID: "5b0bbb6f-a4f2-428a-8c0b-8561d031061c") : secret "cluster-monitoring-operator-tls" not found Jul 13 07:02:44.821722 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:44.821329 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c28d91ed-a056-47d9-ba7c-f789f42dd6cc-metrics-certs\") pod \"router-default-7f7c7b8964-kxkhw\" (UID: \"c28d91ed-a056-47d9-ba7c-f789f42dd6cc\") " pod="openshift-ingress/router-default-7f7c7b8964-kxkhw" Jul 13 07:02:44.821722 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:44.821361 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a7c70271-ef81-4393-9206-d7957e5c86e1-metrics-tls\") pod \"dns-default-wg6mf\" (UID: \"a7c70271-ef81-4393-9206-d7957e5c86e1\") " pod="openshift-dns/dns-default-wg6mf" Jul 13 07:02:44.821722 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:44.821472 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ddf4f6fd-5259-4c72-94df-245960dc7642-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-8grn6\" (UID: \"ddf4f6fd-5259-4c72-94df-245960dc7642\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-8grn6" Jul 13 07:02:44.821722 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:44.821637 2586 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 13 07:02:44.821722 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:44.821697 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c28d91ed-a056-47d9-ba7c-f789f42dd6cc-metrics-certs podName:c28d91ed-a056-47d9-ba7c-f789f42dd6cc nodeName:}" failed. No retries permitted until 2026-07-13 07:02:48.821668139 +0000 UTC m=+43.657231959 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c28d91ed-a056-47d9-ba7c-f789f42dd6cc-metrics-certs") pod "router-default-7f7c7b8964-kxkhw" (UID: "c28d91ed-a056-47d9-ba7c-f789f42dd6cc") : secret "router-metrics-certs-default" not found Jul 13 07:02:44.822389 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:44.821803 2586 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 07:02:44.822389 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:44.821866 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7c70271-ef81-4393-9206-d7957e5c86e1-metrics-tls podName:a7c70271-ef81-4393-9206-d7957e5c86e1 nodeName:}" failed. No retries permitted until 2026-07-13 07:02:48.821851413 +0000 UTC m=+43.657415268 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a7c70271-ef81-4393-9206-d7957e5c86e1-metrics-tls") pod "dns-default-wg6mf" (UID: "a7c70271-ef81-4393-9206-d7957e5c86e1") : secret "dns-default-metrics-tls" not found Jul 13 07:02:44.822389 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:44.822125 2586 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 13 07:02:44.822389 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:44.822171 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddf4f6fd-5259-4c72-94df-245960dc7642-samples-operator-tls podName:ddf4f6fd-5259-4c72-94df-245960dc7642 nodeName:}" failed. No retries permitted until 2026-07-13 07:02:48.822158764 +0000 UTC m=+43.657722579 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ddf4f6fd-5259-4c72-94df-245960dc7642-samples-operator-tls") pod "cluster-samples-operator-65db858894-8grn6" (UID: "ddf4f6fd-5259-4c72-94df-245960dc7642") : secret "samples-operator-tls" not found Jul 13 07:02:45.023438 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:45.022709 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ec04b338-54ca-471f-b138-eabbf7009550-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-6fw29\" (UID: \"ec04b338-54ca-471f-b138-eabbf7009550\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-6fw29" Jul 13 07:02:45.023438 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:45.022936 2586 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 13 07:02:45.023438 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:45.023014 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec04b338-54ca-471f-b138-eabbf7009550-networking-console-plugin-cert podName:ec04b338-54ca-471f-b138-eabbf7009550 nodeName:}" failed. No retries permitted until 2026-07-13 07:02:49.022993059 +0000 UTC m=+43.858556874 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ec04b338-54ca-471f-b138-eabbf7009550-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-6fw29" (UID: "ec04b338-54ca-471f-b138-eabbf7009550") : secret "networking-console-plugin-cert" not found Jul 13 07:02:45.045264 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:45.045227 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-56rlp" event={"ID":"2d5a84b8-155e-4dc0-8526-19a67235d08f","Type":"ContainerStarted","Data":"a90c17b4884e755e8369c2163dbec1c6d99e33ed82f7dae96b90749a352044b7"} Jul 13 07:02:45.047495 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:45.047464 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-drnth" event={"ID":"c4a2167f-7cee-4066-914f-d9c9e4d3bff7","Type":"ContainerStarted","Data":"01d735afc9db0032e6f2b242b5c990a4bf031ffff78b6bb7d78d5bdf0755fae0"} Jul 13 07:02:45.067356 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:45.067302 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-56rlp" podStartSLOduration=12.135382411 podStartE2EDuration="39.067286404s" totalImagesPullingTime="26.931903993s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 07:02:06 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:02:45.066012492 +0000 UTC m=+39.901576315" watchObservedRunningTime="2026-07-13 07:02:45.067286404 +0000 UTC m=+39.902850237" Jul 13 07:02:45.080812 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:45.080754 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-drnth" podStartSLOduration=12.994878226 podStartE2EDuration="15.080734297s" totalImagesPullingTime="2.085856071s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:02:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:02:45.080508501 +0000 UTC m=+39.916072324" watchObservedRunningTime="2026-07-13 07:02:45.080734297 +0000 UTC m=+39.916298106" Jul 13 07:02:45.201784 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:45.201707 2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xs7q5_2abf1e96-1ba2-4cc1-a632-48e28b374830/konnectivity-agent/0.log" limit=1024 Jul 13 07:02:45.300588 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:45.300521 2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xs7q5_2abf1e96-1ba2-4cc1-a632-48e28b374830/konnectivity-agent/0.log" limit=1024 Jul 13 07:02:47.821395 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:47.821358 2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xs7q5_2abf1e96-1ba2-4cc1-a632-48e28b374830/konnectivity-agent/0.log" limit=1024 Jul 13 07:02:47.876098 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:47.876054 2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xs7q5_2abf1e96-1ba2-4cc1-a632-48e28b374830/konnectivity-agent/0.log" limit=1024 Jul 13 07:02:48.757833 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:48.757788 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-registry-tls\") pod \"image-registry-69599bff85-z8nf7\" (UID: \"16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d\") " pod="openshift-image-registry/image-registry-69599bff85-z8nf7" Jul 13 07:02:48.758018 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:48.757937 2586 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 13 07:02:48.758018 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:48.757959 2586 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69599bff85-z8nf7: secret "image-registry-tls" not found Jul 13 07:02:48.758018 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:48.758018 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-registry-tls podName:16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d nodeName:}" failed. No retries permitted until 2026-07-13 07:02:56.757996418 +0000 UTC m=+51.593560248 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-registry-tls") pod "image-registry-69599bff85-z8nf7" (UID: "16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d") : secret "image-registry-tls" not found Jul 13 07:02:48.859247 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:48.859204 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5b0bbb6f-a4f2-428a-8c0b-8561d031061c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-74xdm\" (UID: \"5b0bbb6f-a4f2-428a-8c0b-8561d031061c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-74xdm" Jul 13 07:02:48.859776 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:48.859298 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c28d91ed-a056-47d9-ba7c-f789f42dd6cc-service-ca-bundle\") pod \"router-default-7f7c7b8964-kxkhw\" (UID: \"c28d91ed-a056-47d9-ba7c-f789f42dd6cc\") " pod="openshift-ingress/router-default-7f7c7b8964-kxkhw" Jul 13 07:02:48.859776 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:48.859338 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c28d91ed-a056-47d9-ba7c-f789f42dd6cc-metrics-certs\") pod \"router-default-7f7c7b8964-kxkhw\" (UID: \"c28d91ed-a056-47d9-ba7c-f789f42dd6cc\") " pod="openshift-ingress/router-default-7f7c7b8964-kxkhw" Jul 13 07:02:48.859776 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:48.859368 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a7c70271-ef81-4393-9206-d7957e5c86e1-metrics-tls\") pod \"dns-default-wg6mf\" (UID: \"a7c70271-ef81-4393-9206-d7957e5c86e1\") " pod="openshift-dns/dns-default-wg6mf" Jul 13 07:02:48.859776 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:48.859386 2586 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 13 07:02:48.859776 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:48.859423 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ddf4f6fd-5259-4c72-94df-245960dc7642-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-8grn6\" (UID: \"ddf4f6fd-5259-4c72-94df-245960dc7642\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-8grn6" Jul 13 07:02:48.859776 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:48.859480 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b0bbb6f-a4f2-428a-8c0b-8561d031061c-cluster-monitoring-operator-tls podName:5b0bbb6f-a4f2-428a-8c0b-8561d031061c nodeName:}" failed. No retries permitted until 2026-07-13 07:02:56.859456308 +0000 UTC m=+51.695020116 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5b0bbb6f-a4f2-428a-8c0b-8561d031061c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-74xdm" (UID: "5b0bbb6f-a4f2-428a-8c0b-8561d031061c") : secret "cluster-monitoring-operator-tls" not found Jul 13 07:02:48.859776 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:48.859518 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c28d91ed-a056-47d9-ba7c-f789f42dd6cc-service-ca-bundle podName:c28d91ed-a056-47d9-ba7c-f789f42dd6cc nodeName:}" failed. No retries permitted until 2026-07-13 07:02:56.859498662 +0000 UTC m=+51.695062477 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c28d91ed-a056-47d9-ba7c-f789f42dd6cc-service-ca-bundle") pod "router-default-7f7c7b8964-kxkhw" (UID: "c28d91ed-a056-47d9-ba7c-f789f42dd6cc") : configmap references non-existent config key: service-ca.crt Jul 13 07:02:48.859776 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:48.859532 2586 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 13 07:02:48.859776 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:48.859577 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddf4f6fd-5259-4c72-94df-245960dc7642-samples-operator-tls podName:ddf4f6fd-5259-4c72-94df-245960dc7642 nodeName:}" failed. No retries permitted until 2026-07-13 07:02:56.859564407 +0000 UTC m=+51.695128218 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ddf4f6fd-5259-4c72-94df-245960dc7642-samples-operator-tls") pod "cluster-samples-operator-65db858894-8grn6" (UID: "ddf4f6fd-5259-4c72-94df-245960dc7642") : secret "samples-operator-tls" not found Jul 13 07:02:48.859776 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:48.859628 2586 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 13 07:02:48.859776 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:48.859635 2586 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 07:02:48.859776 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:48.859658 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7c70271-ef81-4393-9206-d7957e5c86e1-metrics-tls podName:a7c70271-ef81-4393-9206-d7957e5c86e1 nodeName:}" failed. No retries permitted until 2026-07-13 07:02:56.859650351 +0000 UTC m=+51.695214159 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a7c70271-ef81-4393-9206-d7957e5c86e1-metrics-tls") pod "dns-default-wg6mf" (UID: "a7c70271-ef81-4393-9206-d7957e5c86e1") : secret "dns-default-metrics-tls" not found Jul 13 07:02:48.859776 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:48.859673 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c28d91ed-a056-47d9-ba7c-f789f42dd6cc-metrics-certs podName:c28d91ed-a056-47d9-ba7c-f789f42dd6cc nodeName:}" failed. No retries permitted until 2026-07-13 07:02:56.859664485 +0000 UTC m=+51.695228295 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c28d91ed-a056-47d9-ba7c-f789f42dd6cc-metrics-certs") pod "router-default-7f7c7b8964-kxkhw" (UID: "c28d91ed-a056-47d9-ba7c-f789f42dd6cc") : secret "router-metrics-certs-default" not found Jul 13 07:02:48.982562 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:48.982524 2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xs7q5_2abf1e96-1ba2-4cc1-a632-48e28b374830/konnectivity-agent/0.log" limit=1024 Jul 13 07:02:49.033233 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:49.033162 2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xs7q5_2abf1e96-1ba2-4cc1-a632-48e28b374830/konnectivity-agent/0.log" limit=1024 Jul 13 07:02:49.060409 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:49.060376 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ec04b338-54ca-471f-b138-eabbf7009550-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-6fw29\" (UID: \"ec04b338-54ca-471f-b138-eabbf7009550\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-6fw29" Jul 13 07:02:49.060596 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:49.060541 2586 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 13 07:02:49.060667 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:49.060611 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec04b338-54ca-471f-b138-eabbf7009550-networking-console-plugin-cert podName:ec04b338-54ca-471f-b138-eabbf7009550 nodeName:}" failed. No retries permitted until 2026-07-13 07:02:57.060591899 +0000 UTC m=+51.896155708 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ec04b338-54ca-471f-b138-eabbf7009550-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-6fw29" (UID: "ec04b338-54ca-471f-b138-eabbf7009550") : secret "networking-console-plugin-cert" not found Jul 13 07:02:50.068204 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:50.067222 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-z7x8m" event={"ID":"05bf1881-84d9-42e6-a7ed-1a348839015d","Type":"ContainerStarted","Data":"f6a8d27d269f5bd96834dd1c106b975fb41f61b2d1b111277a7289a883616c9d"} Jul 13 07:02:50.070479 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:50.070443 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-m7g8b" event={"ID":"396e6c4f-4cbc-40ed-b6e8-babbbbc48d39","Type":"ContainerStarted","Data":"a151b944da0e3770d8dd786340c936dff1ac46ce5df4e9e1f4bddff2a50f2d39"} Jul 13 07:02:50.073642 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:50.073172 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-lhs5q" event={"ID":"e38c842e-619b-4849-9a60-d65e577ba45e","Type":"ContainerStarted","Data":"b77c5970b5a4da52f404c68b5101ae5b67727dc3e7129d909d29814d8a4f038a"} Jul 13 07:02:50.075903 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:50.075880 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ps2kk" event={"ID":"30689818-dbf9-4665-86dc-b81d96b6bed1","Type":"ContainerStarted","Data":"8c164d4e84966cbfc93e237865007349beaa7a473beecff450cadfa6f91a05bd"} Jul 13 07:02:50.079710 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:50.077880 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-r878q" event={"ID":"293edc99-2691-4915-ba03-7407d88b89b4","Type":"ContainerStarted","Data":"4f466ca5259a2771f1cbaa0a50fba757183fb9b951848aac138f66ff0dbf09ff"} Jul 13 07:02:50.079710 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:50.078750 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-r878q" Jul 13 07:02:50.079991 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:50.079918 2586 patch_prober.go:28] interesting pod/console-operator-7866497494-r878q container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.7:8443/readyz\": dial tcp 10.132.0.7:8443: connect: connection refused" start-of-body= Jul 13 07:02:50.079991 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:50.079967 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-7866497494-r878q" podUID="293edc99-2691-4915-ba03-7407d88b89b4" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.7:8443/readyz\": dial tcp 10.132.0.7:8443: connect: connection refused" Jul 13 07:02:50.080919 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:50.080862 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-bh4jx" event={"ID":"8d761ca6-7ceb-4a5f-be27-3e668d545e84","Type":"ContainerStarted","Data":"6f95c49f011250490787b493e3fd7c788a32f8ee407bbc451ee26612e1c0b184"} Jul 13 07:02:50.098172 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:50.098093 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-ps2kk" podStartSLOduration=2.487333525 podStartE2EDuration="10.098072975s" totalImagesPullingTime="7.61073945s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:02:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:02:50.097377823 +0000 UTC m=+44.932941656" watchObservedRunningTime="2026-07-13 07:02:50.098072975 +0000 UTC m=+44.933636807" Jul 13 07:02:50.098476 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:50.098339 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-z7x8m" podStartSLOduration=12.491821872 podStartE2EDuration="20.0983295s" totalImagesPullingTime="7.606507628s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:02:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:02:50.084138404 +0000 UTC m=+44.919702235" watchObservedRunningTime="2026-07-13 07:02:50.0983295 +0000 UTC m=+44.933893332" Jul 13 07:02:50.119198 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:50.119130 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-r878q" podStartSLOduration=12.509840823 podStartE2EDuration="20.119108171s" totalImagesPullingTime="7.609267348s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:02:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:02:50.117539033 +0000 UTC m=+44.953102866" watchObservedRunningTime="2026-07-13 07:02:50.119108171 +0000 UTC m=+44.954672002" Jul 13 07:02:50.138527 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:50.138466 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-m7g8b" podStartSLOduration=12.52876714 podStartE2EDuration="20.138445753s" totalImagesPullingTime="7.609678613s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:02:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:02:50.13674973 +0000 UTC m=+44.972313564" watchObservedRunningTime="2026-07-13 07:02:50.138445753 +0000 UTC m=+44.974009589" Jul 13 07:02:50.152710 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:50.150217 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-bh4jx" podStartSLOduration=12.534157367 podStartE2EDuration="20.150197834s" totalImagesPullingTime="7.616040467s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:02:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:02:50.149758627 +0000 UTC m=+44.985322462" watchObservedRunningTime="2026-07-13 07:02:50.150197834 +0000 UTC m=+44.985761666" Jul 13 07:02:51.081311 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:51.081251 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-lhs5q" podStartSLOduration=13.437881985 podStartE2EDuration="21.081226059s" totalImagesPullingTime="7.643344074s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:02:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:02:50.166282156 +0000 UTC m=+45.001845986" watchObservedRunningTime="2026-07-13 07:02:51.081226059 +0000 UTC m=+45.916789896" Jul 13 07:02:51.081787 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:51.081613 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-t5kqb"] Jul 13 07:02:51.088991 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:51.088962 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-r878q_293edc99-2691-4915-ba03-7407d88b89b4/console-operator/0.log" Jul 13 07:02:51.089160 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:51.089014 2586 generic.go:356] "Generic (PLEG): container finished" podID="293edc99-2691-4915-ba03-7407d88b89b4" containerID="4f466ca5259a2771f1cbaa0a50fba757183fb9b951848aac138f66ff0dbf09ff" exitCode=255 Jul 13 07:02:51.089639 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:51.089621 2586 scope.go:117] "RemoveContainer" containerID="4f466ca5259a2771f1cbaa0a50fba757183fb9b951848aac138f66ff0dbf09ff" Jul 13 07:02:51.103414 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:51.103381 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-r878q" event={"ID":"293edc99-2691-4915-ba03-7407d88b89b4","Type":"ContainerDied","Data":"4f466ca5259a2771f1cbaa0a50fba757183fb9b951848aac138f66ff0dbf09ff"} Jul 13 07:02:51.103564 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:51.103508 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-t5kqb" Jul 13 07:02:51.104910 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:51.104880 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-t5kqb"] Jul 13 07:02:51.107093 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:51.106014 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-krdct\"" Jul 13 07:02:51.107093 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:51.106349 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 13 07:02:51.107093 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:51.106658 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 13 07:02:51.281470 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:51.281435 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/16425325-d8c8-40cb-a1c0-bfbc147f1416-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t5kqb\" (UID: \"16425325-d8c8-40cb-a1c0-bfbc147f1416\") " pod="openshift-insights/insights-runtime-extractor-t5kqb" Jul 13 07:02:51.282096 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:51.282071 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/16425325-d8c8-40cb-a1c0-bfbc147f1416-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t5kqb\" (UID: \"16425325-d8c8-40cb-a1c0-bfbc147f1416\") " pod="openshift-insights/insights-runtime-extractor-t5kqb" Jul 13 07:02:51.282654 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:51.282441 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/16425325-d8c8-40cb-a1c0-bfbc147f1416-data-volume\") pod \"insights-runtime-extractor-t5kqb\" (UID: \"16425325-d8c8-40cb-a1c0-bfbc147f1416\") " pod="openshift-insights/insights-runtime-extractor-t5kqb" Jul 13 07:02:51.282809 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:51.282721 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/16425325-d8c8-40cb-a1c0-bfbc147f1416-crio-socket\") pod \"insights-runtime-extractor-t5kqb\" (UID: \"16425325-d8c8-40cb-a1c0-bfbc147f1416\") " pod="openshift-insights/insights-runtime-extractor-t5kqb" Jul 13 07:02:51.282809 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:51.282774 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgxdv\" (UniqueName: \"kubernetes.io/projected/16425325-d8c8-40cb-a1c0-bfbc147f1416-kube-api-access-jgxdv\") pod \"insights-runtime-extractor-t5kqb\" (UID: \"16425325-d8c8-40cb-a1c0-bfbc147f1416\") " pod="openshift-insights/insights-runtime-extractor-t5kqb" Jul 13 07:02:51.330325 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:51.330289 2586 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-r878q" Jul 13 07:02:51.383707 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:51.383645 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/16425325-d8c8-40cb-a1c0-bfbc147f1416-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t5kqb\" (UID: \"16425325-d8c8-40cb-a1c0-bfbc147f1416\") " pod="openshift-insights/insights-runtime-extractor-t5kqb" Jul 13 07:02:51.383881 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:51.383777 2586 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 07:02:51.383881 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:51.383854 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/16425325-d8c8-40cb-a1c0-bfbc147f1416-kube-rbac-proxy-cm podName:16425325-d8c8-40cb-a1c0-bfbc147f1416 nodeName:}" failed. No retries permitted until 2026-07-13 07:02:51.883835103 +0000 UTC m=+46.719398935 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/16425325-d8c8-40cb-a1c0-bfbc147f1416-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t5kqb" (UID: "16425325-d8c8-40cb-a1c0-bfbc147f1416") : configmap "kube-rbac-proxy" not found Jul 13 07:02:51.383952 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:51.383894 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/16425325-d8c8-40cb-a1c0-bfbc147f1416-data-volume\") pod \"insights-runtime-extractor-t5kqb\" (UID: \"16425325-d8c8-40cb-a1c0-bfbc147f1416\") " pod="openshift-insights/insights-runtime-extractor-t5kqb" Jul 13 07:02:51.383952 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:51.383940 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/16425325-d8c8-40cb-a1c0-bfbc147f1416-crio-socket\") pod \"insights-runtime-extractor-t5kqb\" (UID: \"16425325-d8c8-40cb-a1c0-bfbc147f1416\") " pod="openshift-insights/insights-runtime-extractor-t5kqb" Jul 13 07:02:51.384036 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:51.383959 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgxdv\" (UniqueName: \"kubernetes.io/projected/16425325-d8c8-40cb-a1c0-bfbc147f1416-kube-api-access-jgxdv\") pod \"insights-runtime-extractor-t5kqb\" (UID: \"16425325-d8c8-40cb-a1c0-bfbc147f1416\") " pod="openshift-insights/insights-runtime-extractor-t5kqb" Jul 13 07:02:51.384081 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:51.384042 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/16425325-d8c8-40cb-a1c0-bfbc147f1416-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t5kqb\" (UID: \"16425325-d8c8-40cb-a1c0-bfbc147f1416\") " pod="openshift-insights/insights-runtime-extractor-t5kqb" Jul 13 07:02:51.384191 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:51.384175 2586 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 07:02:51.384245 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:51.384217 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/16425325-d8c8-40cb-a1c0-bfbc147f1416-crio-socket\") pod \"insights-runtime-extractor-t5kqb\" (UID: \"16425325-d8c8-40cb-a1c0-bfbc147f1416\") " pod="openshift-insights/insights-runtime-extractor-t5kqb" Jul 13 07:02:51.384292 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:51.384229 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16425325-d8c8-40cb-a1c0-bfbc147f1416-insights-runtime-extractor-tls podName:16425325-d8c8-40cb-a1c0-bfbc147f1416 nodeName:}" failed. No retries permitted until 2026-07-13 07:02:51.884214736 +0000 UTC m=+46.719778556 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/16425325-d8c8-40cb-a1c0-bfbc147f1416-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t5kqb" (UID: "16425325-d8c8-40cb-a1c0-bfbc147f1416") : secret "insights-runtime-extractor-tls" not found Jul 13 07:02:51.384332 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:51.384289 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/16425325-d8c8-40cb-a1c0-bfbc147f1416-data-volume\") pod \"insights-runtime-extractor-t5kqb\" (UID: \"16425325-d8c8-40cb-a1c0-bfbc147f1416\") " pod="openshift-insights/insights-runtime-extractor-t5kqb" Jul 13 07:02:51.396278 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:51.396208 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgxdv\" (UniqueName: \"kubernetes.io/projected/16425325-d8c8-40cb-a1c0-bfbc147f1416-kube-api-access-jgxdv\") pod \"insights-runtime-extractor-t5kqb\" (UID: \"16425325-d8c8-40cb-a1c0-bfbc147f1416\") " pod="openshift-insights/insights-runtime-extractor-t5kqb" Jul 13 07:02:51.889267 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:51.889232 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/16425325-d8c8-40cb-a1c0-bfbc147f1416-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t5kqb\" (UID: \"16425325-d8c8-40cb-a1c0-bfbc147f1416\") " pod="openshift-insights/insights-runtime-extractor-t5kqb" Jul 13 07:02:51.889513 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:51.889393 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/16425325-d8c8-40cb-a1c0-bfbc147f1416-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t5kqb\" (UID: \"16425325-d8c8-40cb-a1c0-bfbc147f1416\") " pod="openshift-insights/insights-runtime-extractor-t5kqb" Jul 13 07:02:51.889513 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:51.889399 2586 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 07:02:51.889513 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:51.889471 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16425325-d8c8-40cb-a1c0-bfbc147f1416-insights-runtime-extractor-tls podName:16425325-d8c8-40cb-a1c0-bfbc147f1416 nodeName:}" failed. No retries permitted until 2026-07-13 07:02:52.88945078 +0000 UTC m=+47.725014595 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/16425325-d8c8-40cb-a1c0-bfbc147f1416-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t5kqb" (UID: "16425325-d8c8-40cb-a1c0-bfbc147f1416") : secret "insights-runtime-extractor-tls" not found Jul 13 07:02:51.889671 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:51.889537 2586 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 07:02:51.889671 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:51.889597 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/16425325-d8c8-40cb-a1c0-bfbc147f1416-kube-rbac-proxy-cm podName:16425325-d8c8-40cb-a1c0-bfbc147f1416 nodeName:}" failed. No retries permitted until 2026-07-13 07:02:52.889579774 +0000 UTC m=+47.725143587 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/16425325-d8c8-40cb-a1c0-bfbc147f1416-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t5kqb" (UID: "16425325-d8c8-40cb-a1c0-bfbc147f1416") : configmap "kube-rbac-proxy" not found Jul 13 07:02:52.093903 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:52.093876 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-r878q_293edc99-2691-4915-ba03-7407d88b89b4/console-operator/1.log" Jul 13 07:02:52.094362 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:52.094347 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-r878q_293edc99-2691-4915-ba03-7407d88b89b4/console-operator/0.log" Jul 13 07:02:52.094412 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:52.094383 2586 generic.go:356] "Generic (PLEG): container finished" podID="293edc99-2691-4915-ba03-7407d88b89b4" containerID="08782d39a6c136bde21be37535b69f776081505a3e266f583a008758ce51a725" exitCode=255 Jul 13 07:02:52.094445 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:52.094425 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-r878q" event={"ID":"293edc99-2691-4915-ba03-7407d88b89b4","Type":"ContainerDied","Data":"08782d39a6c136bde21be37535b69f776081505a3e266f583a008758ce51a725"} Jul 13 07:02:52.094488 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:52.094467 2586 scope.go:117] "RemoveContainer" containerID="4f466ca5259a2771f1cbaa0a50fba757183fb9b951848aac138f66ff0dbf09ff" Jul 13 07:02:52.094704 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:52.094662 2586 scope.go:117] "RemoveContainer" containerID="08782d39a6c136bde21be37535b69f776081505a3e266f583a008758ce51a725" Jul 13 07:02:52.094944 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:52.094923 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-r878q_openshift-console-operator(293edc99-2691-4915-ba03-7407d88b89b4)\"" pod="openshift-console-operator/console-operator-7866497494-r878q" podUID="293edc99-2691-4915-ba03-7407d88b89b4" Jul 13 07:02:52.767036 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:52.767003 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-g6kgx_75604003-f810-475e-b0db-af8180c49604/dns-node-resolver/0.log" Jul 13 07:02:52.898188 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:52.898144 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/16425325-d8c8-40cb-a1c0-bfbc147f1416-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t5kqb\" (UID: \"16425325-d8c8-40cb-a1c0-bfbc147f1416\") " pod="openshift-insights/insights-runtime-extractor-t5kqb" Jul 13 07:02:52.898366 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:52.898202 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/16425325-d8c8-40cb-a1c0-bfbc147f1416-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t5kqb\" (UID: \"16425325-d8c8-40cb-a1c0-bfbc147f1416\") " pod="openshift-insights/insights-runtime-extractor-t5kqb" Jul 13 07:02:52.898366 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:52.898316 2586 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 07:02:52.898366 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:52.898328 2586 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 07:02:52.898474 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:52.898370 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/16425325-d8c8-40cb-a1c0-bfbc147f1416-kube-rbac-proxy-cm podName:16425325-d8c8-40cb-a1c0-bfbc147f1416 nodeName:}" failed. No retries permitted until 2026-07-13 07:02:54.898356121 +0000 UTC m=+49.733919930 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/16425325-d8c8-40cb-a1c0-bfbc147f1416-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t5kqb" (UID: "16425325-d8c8-40cb-a1c0-bfbc147f1416") : configmap "kube-rbac-proxy" not found Jul 13 07:02:52.898474 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:52.898405 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16425325-d8c8-40cb-a1c0-bfbc147f1416-insights-runtime-extractor-tls podName:16425325-d8c8-40cb-a1c0-bfbc147f1416 nodeName:}" failed. No retries permitted until 2026-07-13 07:02:54.898389232 +0000 UTC m=+49.733953041 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/16425325-d8c8-40cb-a1c0-bfbc147f1416-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t5kqb" (UID: "16425325-d8c8-40cb-a1c0-bfbc147f1416") : secret "insights-runtime-extractor-tls" not found Jul 13 07:02:53.099357 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:53.099287 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-r878q_293edc99-2691-4915-ba03-7407d88b89b4/console-operator/1.log" Jul 13 07:02:53.099870 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:53.099581 2586 scope.go:117] "RemoveContainer" containerID="08782d39a6c136bde21be37535b69f776081505a3e266f583a008758ce51a725" Jul 13 07:02:53.099870 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:53.099733 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-r878q_openshift-console-operator(293edc99-2691-4915-ba03-7407d88b89b4)\"" pod="openshift-console-operator/console-operator-7866497494-r878q" podUID="293edc99-2691-4915-ba03-7407d88b89b4" Jul 13 07:02:53.966996 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:53.966964 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vxcrv_be89589f-0a40-4e74-9ecf-aeac7e231126/node-ca/0.log" Jul 13 07:02:54.566861 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:54.566833 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-ps2kk_30689818-dbf9-4665-86dc-b81d96b6bed1/serve-healthcheck-canary/0.log" Jul 13 07:02:54.916311 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:54.916220 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/16425325-d8c8-40cb-a1c0-bfbc147f1416-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t5kqb\" (UID: \"16425325-d8c8-40cb-a1c0-bfbc147f1416\") " pod="openshift-insights/insights-runtime-extractor-t5kqb" Jul 13 07:02:54.916311 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:54.916273 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/16425325-d8c8-40cb-a1c0-bfbc147f1416-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t5kqb\" (UID: \"16425325-d8c8-40cb-a1c0-bfbc147f1416\") " pod="openshift-insights/insights-runtime-extractor-t5kqb" Jul 13 07:02:54.916558 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:54.916349 2586 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 07:02:54.916558 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:54.916396 2586 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 07:02:54.916558 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:54.916418 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/16425325-d8c8-40cb-a1c0-bfbc147f1416-kube-rbac-proxy-cm podName:16425325-d8c8-40cb-a1c0-bfbc147f1416 nodeName:}" failed. No retries permitted until 2026-07-13 07:02:58.916400115 +0000 UTC m=+53.751963928 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/16425325-d8c8-40cb-a1c0-bfbc147f1416-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t5kqb" (UID: "16425325-d8c8-40cb-a1c0-bfbc147f1416") : configmap "kube-rbac-proxy" not found Jul 13 07:02:54.916558 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:54.916476 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16425325-d8c8-40cb-a1c0-bfbc147f1416-insights-runtime-extractor-tls podName:16425325-d8c8-40cb-a1c0-bfbc147f1416 nodeName:}" failed. No retries permitted until 2026-07-13 07:02:58.916457192 +0000 UTC m=+53.752021006 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/16425325-d8c8-40cb-a1c0-bfbc147f1416-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t5kqb" (UID: "16425325-d8c8-40cb-a1c0-bfbc147f1416") : secret "insights-runtime-extractor-tls" not found Jul 13 07:02:56.831887 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:56.831851 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-registry-tls\") pod \"image-registry-69599bff85-z8nf7\" (UID: \"16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d\") " pod="openshift-image-registry/image-registry-69599bff85-z8nf7" Jul 13 07:02:56.832285 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:56.831974 2586 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 13 07:02:56.832285 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:56.831986 2586 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69599bff85-z8nf7: secret "image-registry-tls" not found Jul 13 07:02:56.832285 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:56.832033 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-registry-tls podName:16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d nodeName:}" failed. No retries permitted until 2026-07-13 07:03:12.832019813 +0000 UTC m=+67.667583623 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-registry-tls") pod "image-registry-69599bff85-z8nf7" (UID: "16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d") : secret "image-registry-tls" not found Jul 13 07:02:56.932389 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:56.932350 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5b0bbb6f-a4f2-428a-8c0b-8561d031061c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-74xdm\" (UID: \"5b0bbb6f-a4f2-428a-8c0b-8561d031061c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-74xdm" Jul 13 07:02:56.932581 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:56.932420 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c28d91ed-a056-47d9-ba7c-f789f42dd6cc-service-ca-bundle\") pod \"router-default-7f7c7b8964-kxkhw\" (UID: \"c28d91ed-a056-47d9-ba7c-f789f42dd6cc\") " pod="openshift-ingress/router-default-7f7c7b8964-kxkhw" Jul 13 07:02:56.932581 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:56.932449 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c28d91ed-a056-47d9-ba7c-f789f42dd6cc-metrics-certs\") pod \"router-default-7f7c7b8964-kxkhw\" (UID: \"c28d91ed-a056-47d9-ba7c-f789f42dd6cc\") " pod="openshift-ingress/router-default-7f7c7b8964-kxkhw" Jul 13 07:02:56.932581 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:56.932465 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a7c70271-ef81-4393-9206-d7957e5c86e1-metrics-tls\") pod \"dns-default-wg6mf\" (UID: \"a7c70271-ef81-4393-9206-d7957e5c86e1\") " pod="openshift-dns/dns-default-wg6mf" Jul 13 07:02:56.932581 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:56.932506 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ddf4f6fd-5259-4c72-94df-245960dc7642-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-8grn6\" (UID: \"ddf4f6fd-5259-4c72-94df-245960dc7642\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-8grn6" Jul 13 07:02:56.932581 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:56.932506 2586 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 13 07:02:56.932810 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:56.932591 2586 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 13 07:02:56.932810 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:56.932595 2586 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 13 07:02:56.932810 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:56.932609 2586 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 07:02:56.932810 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:56.932606 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b0bbb6f-a4f2-428a-8c0b-8561d031061c-cluster-monitoring-operator-tls podName:5b0bbb6f-a4f2-428a-8c0b-8561d031061c nodeName:}" failed. No retries permitted until 2026-07-13 07:03:12.932568018 +0000 UTC m=+67.768131826 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5b0bbb6f-a4f2-428a-8c0b-8561d031061c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-74xdm" (UID: "5b0bbb6f-a4f2-428a-8c0b-8561d031061c") : secret "cluster-monitoring-operator-tls" not found Jul 13 07:02:56.932810 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:56.932668 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c28d91ed-a056-47d9-ba7c-f789f42dd6cc-service-ca-bundle podName:c28d91ed-a056-47d9-ba7c-f789f42dd6cc nodeName:}" failed. No retries permitted until 2026-07-13 07:03:12.932642391 +0000 UTC m=+67.768206204 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c28d91ed-a056-47d9-ba7c-f789f42dd6cc-service-ca-bundle") pod "router-default-7f7c7b8964-kxkhw" (UID: "c28d91ed-a056-47d9-ba7c-f789f42dd6cc") : configmap references non-existent config key: service-ca.crt Jul 13 07:02:56.932810 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:56.932698 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddf4f6fd-5259-4c72-94df-245960dc7642-samples-operator-tls podName:ddf4f6fd-5259-4c72-94df-245960dc7642 nodeName:}" failed. No retries permitted until 2026-07-13 07:03:12.932673689 +0000 UTC m=+67.768237498 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ddf4f6fd-5259-4c72-94df-245960dc7642-samples-operator-tls") pod "cluster-samples-operator-65db858894-8grn6" (UID: "ddf4f6fd-5259-4c72-94df-245960dc7642") : secret "samples-operator-tls" not found Jul 13 07:02:56.932810 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:56.932712 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c28d91ed-a056-47d9-ba7c-f789f42dd6cc-metrics-certs podName:c28d91ed-a056-47d9-ba7c-f789f42dd6cc nodeName:}" failed. No retries permitted until 2026-07-13 07:03:12.932705503 +0000 UTC m=+67.768269312 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c28d91ed-a056-47d9-ba7c-f789f42dd6cc-metrics-certs") pod "router-default-7f7c7b8964-kxkhw" (UID: "c28d91ed-a056-47d9-ba7c-f789f42dd6cc") : secret "router-metrics-certs-default" not found Jul 13 07:02:56.932810 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:56.932726 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7c70271-ef81-4393-9206-d7957e5c86e1-metrics-tls podName:a7c70271-ef81-4393-9206-d7957e5c86e1 nodeName:}" failed. No retries permitted until 2026-07-13 07:03:12.932722016 +0000 UTC m=+67.768285825 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a7c70271-ef81-4393-9206-d7957e5c86e1-metrics-tls") pod "dns-default-wg6mf" (UID: "a7c70271-ef81-4393-9206-d7957e5c86e1") : secret "dns-default-metrics-tls" not found Jul 13 07:02:57.134543 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:57.134459 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ec04b338-54ca-471f-b138-eabbf7009550-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-6fw29\" (UID: \"ec04b338-54ca-471f-b138-eabbf7009550\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-6fw29" Jul 13 07:02:57.134709 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:57.134608 2586 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 13 07:02:57.134709 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:57.134669 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec04b338-54ca-471f-b138-eabbf7009550-networking-console-plugin-cert podName:ec04b338-54ca-471f-b138-eabbf7009550 nodeName:}" failed. No retries permitted until 2026-07-13 07:03:13.134652314 +0000 UTC m=+67.970216123 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ec04b338-54ca-471f-b138-eabbf7009550-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-6fw29" (UID: "ec04b338-54ca-471f-b138-eabbf7009550") : secret "networking-console-plugin-cert" not found Jul 13 07:02:58.948517 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:58.948479 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/16425325-d8c8-40cb-a1c0-bfbc147f1416-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t5kqb\" (UID: \"16425325-d8c8-40cb-a1c0-bfbc147f1416\") " pod="openshift-insights/insights-runtime-extractor-t5kqb" Jul 13 07:02:58.949017 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:02:58.948545 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/16425325-d8c8-40cb-a1c0-bfbc147f1416-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t5kqb\" (UID: \"16425325-d8c8-40cb-a1c0-bfbc147f1416\") " pod="openshift-insights/insights-runtime-extractor-t5kqb" Jul 13 07:02:58.949017 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:58.948625 2586 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 07:02:58.949017 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:58.948658 2586 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 07:02:58.949017 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:58.948714 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16425325-d8c8-40cb-a1c0-bfbc147f1416-insights-runtime-extractor-tls podName:16425325-d8c8-40cb-a1c0-bfbc147f1416 nodeName:}" failed. No retries permitted until 2026-07-13 07:03:06.948678268 +0000 UTC m=+61.784242077 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/16425325-d8c8-40cb-a1c0-bfbc147f1416-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t5kqb" (UID: "16425325-d8c8-40cb-a1c0-bfbc147f1416") : secret "insights-runtime-extractor-tls" not found Jul 13 07:02:58.949017 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:02:58.948730 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/16425325-d8c8-40cb-a1c0-bfbc147f1416-kube-rbac-proxy-cm podName:16425325-d8c8-40cb-a1c0-bfbc147f1416 nodeName:}" failed. No retries permitted until 2026-07-13 07:03:06.948723241 +0000 UTC m=+61.784287050 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/16425325-d8c8-40cb-a1c0-bfbc147f1416-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t5kqb" (UID: "16425325-d8c8-40cb-a1c0-bfbc147f1416") : configmap "kube-rbac-proxy" not found Jul 13 07:03:00.078435 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:00.078397 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-r878q" Jul 13 07:03:00.078817 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:00.078804 2586 scope.go:117] "RemoveContainer" containerID="08782d39a6c136bde21be37535b69f776081505a3e266f583a008758ce51a725" Jul 13 07:03:00.078978 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:03:00.078950 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-r878q_openshift-console-operator(293edc99-2691-4915-ba03-7407d88b89b4)\"" pod="openshift-console-operator/console-operator-7866497494-r878q" podUID="293edc99-2691-4915-ba03-7407d88b89b4" Jul 13 07:03:01.330227 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:01.330190 2586 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-r878q" Jul 13 07:03:01.330740 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:01.330579 2586 scope.go:117] "RemoveContainer" containerID="08782d39a6c136bde21be37535b69f776081505a3e266f583a008758ce51a725" Jul 13 07:03:02.131791 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:02.131763 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-r878q_293edc99-2691-4915-ba03-7407d88b89b4/console-operator/1.log" Jul 13 07:03:02.131976 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:02.131811 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-r878q" event={"ID":"293edc99-2691-4915-ba03-7407d88b89b4","Type":"ContainerStarted","Data":"9f6957575b5ab0a017827ef0b8077063c5cfdd8052d7fca8affba4f46b578b96"} Jul 13 07:03:02.132172 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:02.132146 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-r878q" Jul 13 07:03:02.136797 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:02.136768 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-r878q" Jul 13 07:03:04.991354 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:04.991325 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7b964" Jul 13 07:03:07.026316 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:07.026274 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/16425325-d8c8-40cb-a1c0-bfbc147f1416-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t5kqb\" (UID: \"16425325-d8c8-40cb-a1c0-bfbc147f1416\") " pod="openshift-insights/insights-runtime-extractor-t5kqb" Jul 13 07:03:07.026719 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:07.026329 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/16425325-d8c8-40cb-a1c0-bfbc147f1416-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t5kqb\" (UID: \"16425325-d8c8-40cb-a1c0-bfbc147f1416\") " pod="openshift-insights/insights-runtime-extractor-t5kqb" Jul 13 07:03:07.026719 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:03:07.026428 2586 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 07:03:07.026719 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:03:07.026494 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16425325-d8c8-40cb-a1c0-bfbc147f1416-insights-runtime-extractor-tls podName:16425325-d8c8-40cb-a1c0-bfbc147f1416 nodeName:}" failed. No retries permitted until 2026-07-13 07:03:23.026478725 +0000 UTC m=+77.862042535 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/16425325-d8c8-40cb-a1c0-bfbc147f1416-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t5kqb" (UID: "16425325-d8c8-40cb-a1c0-bfbc147f1416") : secret "insights-runtime-extractor-tls" not found Jul 13 07:03:07.026871 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:07.026851 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/16425325-d8c8-40cb-a1c0-bfbc147f1416-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t5kqb\" (UID: \"16425325-d8c8-40cb-a1c0-bfbc147f1416\") " pod="openshift-insights/insights-runtime-extractor-t5kqb" Jul 13 07:03:11.363395 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:11.363363 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldwqr\" (UniqueName: \"kubernetes.io/projected/0ade3045-6b94-41a9-baeb-db7ce7a84c7a-kube-api-access-ldwqr\") pod \"network-check-target-6xdjl\" (UID: \"0ade3045-6b94-41a9-baeb-db7ce7a84c7a\") " pod="openshift-network-diagnostics/network-check-target-6xdjl" Jul 13 07:03:11.363824 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:11.363472 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/638764ba-e970-42f1-83ec-cf5c7075d61e-metrics-certs\") pod \"network-metrics-daemon-znhkv\" (UID: \"638764ba-e970-42f1-83ec-cf5c7075d61e\") " pod="openshift-multus/network-metrics-daemon-znhkv" Jul 13 07:03:11.363824 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:11.363500 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c3724c5e-1a2b-410b-a745-b0f90bf27358-original-pull-secret\") pod \"global-pull-secret-syncer-lb8th\" (UID: \"c3724c5e-1a2b-410b-a745-b0f90bf27358\") " pod="kube-system/global-pull-secret-syncer-lb8th" Jul 13 07:03:11.365571 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:11.365547 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 13 07:03:11.365696 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:11.365570 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 13 07:03:11.366108 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:11.366089 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldwqr\" (UniqueName: \"kubernetes.io/projected/0ade3045-6b94-41a9-baeb-db7ce7a84c7a-kube-api-access-ldwqr\") pod \"network-check-target-6xdjl\" (UID: \"0ade3045-6b94-41a9-baeb-db7ce7a84c7a\") " pod="openshift-network-diagnostics/network-check-target-6xdjl" Jul 13 07:03:11.375970 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:11.375946 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/638764ba-e970-42f1-83ec-cf5c7075d61e-metrics-certs\") pod \"network-metrics-daemon-znhkv\" (UID: \"638764ba-e970-42f1-83ec-cf5c7075d61e\") " pod="openshift-multus/network-metrics-daemon-znhkv" Jul 13 07:03:11.376073 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:11.376057 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c3724c5e-1a2b-410b-a745-b0f90bf27358-original-pull-secret\") pod \"global-pull-secret-syncer-lb8th\" (UID: \"c3724c5e-1a2b-410b-a745-b0f90bf27358\") " pod="kube-system/global-pull-secret-syncer-lb8th" Jul 13 07:03:11.608736 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:11.608696 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lb8th" Jul 13 07:03:11.620056 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:11.619984 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-z8tdw\"" Jul 13 07:03:11.628584 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:11.628562 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6xdjl" Jul 13 07:03:11.630088 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:11.630074 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fh46b\"" Jul 13 07:03:11.639599 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:11.639584 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-znhkv" Jul 13 07:03:11.786182 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:11.786148 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lb8th"] Jul 13 07:03:11.789920 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:03:11.789868 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3724c5e_1a2b_410b_a745_b0f90bf27358.slice/crio-03f39c6000989578ef77d6608978ffe3139940aecf39d23ce4277c34233f5ec3 WatchSource:0}: Error finding container 03f39c6000989578ef77d6608978ffe3139940aecf39d23ce4277c34233f5ec3: Status 404 returned error can't find the container with id 03f39c6000989578ef77d6608978ffe3139940aecf39d23ce4277c34233f5ec3 Jul 13 07:03:11.816830 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:11.816796 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6xdjl"] Jul 13 07:03:11.820301 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:03:11.820272 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ade3045_6b94_41a9_baeb_db7ce7a84c7a.slice/crio-110a3f3f3ae82939740775ee360f0f59c09c8464e1eb5b286208edc325d5f8c7 WatchSource:0}: Error finding container 110a3f3f3ae82939740775ee360f0f59c09c8464e1eb5b286208edc325d5f8c7: Status 404 returned error can't find the container with id 110a3f3f3ae82939740775ee360f0f59c09c8464e1eb5b286208edc325d5f8c7 Jul 13 07:03:11.842216 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:11.842190 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-znhkv"] Jul 13 07:03:11.847826 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:03:11.847801 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod638764ba_e970_42f1_83ec_cf5c7075d61e.slice/crio-b252ecbbdc02cf2c635c27d3def2619f44b6f7311e0d16fb9c4fccec0ed8f202 WatchSource:0}: Error finding container b252ecbbdc02cf2c635c27d3def2619f44b6f7311e0d16fb9c4fccec0ed8f202: Status 404 returned error can't find the container with id b252ecbbdc02cf2c635c27d3def2619f44b6f7311e0d16fb9c4fccec0ed8f202 Jul 13 07:03:12.163418 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:12.163334 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-lb8th" event={"ID":"c3724c5e-1a2b-410b-a745-b0f90bf27358","Type":"ContainerStarted","Data":"03f39c6000989578ef77d6608978ffe3139940aecf39d23ce4277c34233f5ec3"} Jul 13 07:03:12.164826 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:12.164802 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6xdjl" event={"ID":"0ade3045-6b94-41a9-baeb-db7ce7a84c7a","Type":"ContainerStarted","Data":"f5fe9dfe61f1048c2c8b4eceec37bbc0b2eafcea282308581001c10da34bc40b"} Jul 13 07:03:12.164958 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:12.164845 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6xdjl" event={"ID":"0ade3045-6b94-41a9-baeb-db7ce7a84c7a","Type":"ContainerStarted","Data":"110a3f3f3ae82939740775ee360f0f59c09c8464e1eb5b286208edc325d5f8c7"} Jul 13 07:03:12.164958 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:12.164871 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-6xdjl" Jul 13 07:03:12.165906 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:12.165885 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-znhkv" event={"ID":"638764ba-e970-42f1-83ec-cf5c7075d61e","Type":"ContainerStarted","Data":"b252ecbbdc02cf2c635c27d3def2619f44b6f7311e0d16fb9c4fccec0ed8f202"} Jul 13 07:03:12.179407 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:12.179363 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-6xdjl" podStartSLOduration=67.179347184 podStartE2EDuration="1m7.179347184s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:02:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:03:12.178636403 +0000 UTC m=+67.014200245" watchObservedRunningTime="2026-07-13 07:03:12.179347184 +0000 UTC m=+67.014911015" Jul 13 07:03:12.878566 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:12.878527 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-registry-tls\") pod \"image-registry-69599bff85-z8nf7\" (UID: \"16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d\") " pod="openshift-image-registry/image-registry-69599bff85-z8nf7" Jul 13 07:03:12.881540 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:12.881515 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-registry-tls\") pod \"image-registry-69599bff85-z8nf7\" (UID: \"16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d\") " pod="openshift-image-registry/image-registry-69599bff85-z8nf7" Jul 13 07:03:12.979992 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:12.979944 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c28d91ed-a056-47d9-ba7c-f789f42dd6cc-service-ca-bundle\") pod \"router-default-7f7c7b8964-kxkhw\" (UID: \"c28d91ed-a056-47d9-ba7c-f789f42dd6cc\") " pod="openshift-ingress/router-default-7f7c7b8964-kxkhw" Jul 13 07:03:12.980173 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:12.980003 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c28d91ed-a056-47d9-ba7c-f789f42dd6cc-metrics-certs\") pod \"router-default-7f7c7b8964-kxkhw\" (UID: \"c28d91ed-a056-47d9-ba7c-f789f42dd6cc\") " pod="openshift-ingress/router-default-7f7c7b8964-kxkhw" Jul 13 07:03:12.980173 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:12.980028 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a7c70271-ef81-4393-9206-d7957e5c86e1-metrics-tls\") pod \"dns-default-wg6mf\" (UID: \"a7c70271-ef81-4393-9206-d7957e5c86e1\") " pod="openshift-dns/dns-default-wg6mf" Jul 13 07:03:12.980173 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:12.980063 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ddf4f6fd-5259-4c72-94df-245960dc7642-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-8grn6\" (UID: \"ddf4f6fd-5259-4c72-94df-245960dc7642\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-8grn6" Jul 13 07:03:12.980173 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:12.980139 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5b0bbb6f-a4f2-428a-8c0b-8561d031061c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-74xdm\" (UID: \"5b0bbb6f-a4f2-428a-8c0b-8561d031061c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-74xdm" Jul 13 07:03:12.981402 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:12.981311 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c28d91ed-a056-47d9-ba7c-f789f42dd6cc-service-ca-bundle\") pod \"router-default-7f7c7b8964-kxkhw\" (UID: \"c28d91ed-a056-47d9-ba7c-f789f42dd6cc\") " pod="openshift-ingress/router-default-7f7c7b8964-kxkhw" Jul 13 07:03:12.984339 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:12.984280 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a7c70271-ef81-4393-9206-d7957e5c86e1-metrics-tls\") pod \"dns-default-wg6mf\" (UID: \"a7c70271-ef81-4393-9206-d7957e5c86e1\") " pod="openshift-dns/dns-default-wg6mf" Jul 13 07:03:12.985173 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:12.985139 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ddf4f6fd-5259-4c72-94df-245960dc7642-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-8grn6\" (UID: \"ddf4f6fd-5259-4c72-94df-245960dc7642\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-8grn6" Jul 13 07:03:12.985805 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:12.985762 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5b0bbb6f-a4f2-428a-8c0b-8561d031061c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-74xdm\" (UID: \"5b0bbb6f-a4f2-428a-8c0b-8561d031061c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-74xdm" Jul 13 07:03:12.986095 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:12.986070 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c28d91ed-a056-47d9-ba7c-f789f42dd6cc-metrics-certs\") pod \"router-default-7f7c7b8964-kxkhw\" (UID: \"c28d91ed-a056-47d9-ba7c-f789f42dd6cc\") " pod="openshift-ingress/router-default-7f7c7b8964-kxkhw" Jul 13 07:03:13.107165 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:13.107141 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-tn7lg\"" Jul 13 07:03:13.115349 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:13.115295 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69599bff85-z8nf7" Jul 13 07:03:13.174171 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:13.174094 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-znhkv" event={"ID":"638764ba-e970-42f1-83ec-cf5c7075d61e","Type":"ContainerStarted","Data":"d363707493b8d6a6b78551fca8b351808579aa371d97b4de3cb499e3396a1df2"} Jul 13 07:03:13.176420 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:13.176388 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-2rcmc\"" Jul 13 07:03:13.182961 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:13.182381 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ec04b338-54ca-471f-b138-eabbf7009550-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-6fw29\" (UID: \"ec04b338-54ca-471f-b138-eabbf7009550\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-6fw29" Jul 13 07:03:13.185364 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:13.184567 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-74xdm" Jul 13 07:03:13.185364 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:13.185148 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-xdpjm\"" Jul 13 07:03:13.187159 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:13.187101 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ec04b338-54ca-471f-b138-eabbf7009550-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-6fw29\" (UID: \"ec04b338-54ca-471f-b138-eabbf7009550\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-6fw29" Jul 13 07:03:13.193736 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:13.193504 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7f7c7b8964-kxkhw" Jul 13 07:03:13.195024 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:13.194992 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-fgjdm\"" Jul 13 07:03:13.203976 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:13.203944 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-8grn6" Jul 13 07:03:13.208922 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:13.208827 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ts2w7\"" Jul 13 07:03:13.217111 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:13.217085 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wg6mf" Jul 13 07:03:13.304060 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:13.303966 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-69599bff85-z8nf7"] Jul 13 07:03:13.395297 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:13.395164 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-74xdm"] Jul 13 07:03:13.411282 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:13.411251 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-d87kp\"" Jul 13 07:03:13.415722 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:03:13.415665 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b0bbb6f_a4f2_428a_8c0b_8561d031061c.slice/crio-0d25884875564d4c92b6f65aab5468a6dc15badefdf7f510d0d4b550d8c40ad9 WatchSource:0}: Error finding container 0d25884875564d4c92b6f65aab5468a6dc15badefdf7f510d0d4b550d8c40ad9: Status 404 returned error can't find the container with id 0d25884875564d4c92b6f65aab5468a6dc15badefdf7f510d0d4b550d8c40ad9 Jul 13 07:03:13.419730 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:13.419607 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-6fw29" Jul 13 07:03:13.420499 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:13.420475 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7f7c7b8964-kxkhw"] Jul 13 07:03:13.426671 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:03:13.426627 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc28d91ed_a056_47d9_ba7c_f789f42dd6cc.slice/crio-00dc5bcf4d30880bc9322840da98ff89c44512d47c45ee30af57933c56bbec73 WatchSource:0}: Error finding container 00dc5bcf4d30880bc9322840da98ff89c44512d47c45ee30af57933c56bbec73: Status 404 returned error can't find the container with id 00dc5bcf4d30880bc9322840da98ff89c44512d47c45ee30af57933c56bbec73 Jul 13 07:03:13.459320 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:13.458905 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wg6mf"] Jul 13 07:03:13.462146 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:03:13.462099 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7c70271_ef81_4393_9206_d7957e5c86e1.slice/crio-467ebae838cafe679455e6dfb29e7d1c29db03e43c1b87fae3ef96afc17f9523 WatchSource:0}: Error finding container 467ebae838cafe679455e6dfb29e7d1c29db03e43c1b87fae3ef96afc17f9523: Status 404 returned error can't find the container with id 467ebae838cafe679455e6dfb29e7d1c29db03e43c1b87fae3ef96afc17f9523 Jul 13 07:03:13.591091 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:13.590992 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-6fw29"] Jul 13 07:03:13.595419 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:03:13.595384 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec04b338_54ca_471f_b138_eabbf7009550.slice/crio-79bd7ba5a7b31e3af58a6d592071818369ff61639805ef28a892b1f1efb843b0 WatchSource:0}: Error finding container 79bd7ba5a7b31e3af58a6d592071818369ff61639805ef28a892b1f1efb843b0: Status 404 returned error can't find the container with id 79bd7ba5a7b31e3af58a6d592071818369ff61639805ef28a892b1f1efb843b0 Jul 13 07:03:13.621136 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:13.621103 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-8grn6"] Jul 13 07:03:14.184336 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:14.184296 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-6fw29" event={"ID":"ec04b338-54ca-471f-b138-eabbf7009550","Type":"ContainerStarted","Data":"79bd7ba5a7b31e3af58a6d592071818369ff61639805ef28a892b1f1efb843b0"} Jul 13 07:03:14.186517 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:14.186473 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-znhkv" event={"ID":"638764ba-e970-42f1-83ec-cf5c7075d61e","Type":"ContainerStarted","Data":"a998ec03e3aad418e6798e414f91bbd9140be441fb95d36c8fc685fd2389a987"} Jul 13 07:03:14.188429 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:14.188397 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-74xdm" event={"ID":"5b0bbb6f-a4f2-428a-8c0b-8561d031061c","Type":"ContainerStarted","Data":"0d25884875564d4c92b6f65aab5468a6dc15badefdf7f510d0d4b550d8c40ad9"} Jul 13 07:03:14.190965 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:14.190843 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69599bff85-z8nf7" event={"ID":"16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d","Type":"ContainerStarted","Data":"a7716f819c361c82a026fec7b59d95b8c8b05279fe2680bc8cc0637783b3d8c1"} Jul 13 07:03:14.190965 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:14.190896 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69599bff85-z8nf7" event={"ID":"16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d","Type":"ContainerStarted","Data":"0ae13f6f86964f7cc701053c263adee25400a1031d40e4c936973b7b3c356688"} Jul 13 07:03:14.191182 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:14.191115 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-69599bff85-z8nf7" Jul 13 07:03:14.192282 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:14.192214 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wg6mf" event={"ID":"a7c70271-ef81-4393-9206-d7957e5c86e1","Type":"ContainerStarted","Data":"467ebae838cafe679455e6dfb29e7d1c29db03e43c1b87fae3ef96afc17f9523"} Jul 13 07:03:14.194349 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:14.194323 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-8grn6" event={"ID":"ddf4f6fd-5259-4c72-94df-245960dc7642","Type":"ContainerStarted","Data":"660f4605cc07a49b9fe77dd6c2f6a1ab3916c832c8c2dc2450dc3cb263464230"} Jul 13 07:03:14.196191 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:14.196170 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7f7c7b8964-kxkhw" event={"ID":"c28d91ed-a056-47d9-ba7c-f789f42dd6cc","Type":"ContainerStarted","Data":"bf3d55bb33e7cab8451c04aa2c9ccc063d23e3fdba931c3c6545f0fb4c9fc3fd"} Jul 13 07:03:14.196306 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:14.196211 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7f7c7b8964-kxkhw" event={"ID":"c28d91ed-a056-47d9-ba7c-f789f42dd6cc","Type":"ContainerStarted","Data":"00dc5bcf4d30880bc9322840da98ff89c44512d47c45ee30af57933c56bbec73"} Jul 13 07:03:14.216147 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:14.215956 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-znhkv" podStartSLOduration=67.038213556 podStartE2EDuration="1m8.215935392s" totalImagesPullingTime="1.177721836s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:02:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:03:14.215819255 +0000 UTC m=+69.051383088" watchObservedRunningTime="2026-07-13 07:03:14.215935392 +0000 UTC m=+69.051499225" Jul 13 07:03:14.243399 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:14.242920 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-69599bff85-z8nf7" podStartSLOduration=68.24290403 podStartE2EDuration="1m8.24290403s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:02:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:03:14.238966792 +0000 UTC m=+69.074530624" watchObservedRunningTime="2026-07-13 07:03:14.24290403 +0000 UTC m=+69.078467861" Jul 13 07:03:15.194180 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:15.194140 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7f7c7b8964-kxkhw" Jul 13 07:03:15.197453 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:15.197425 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7f7c7b8964-kxkhw" Jul 13 07:03:15.202032 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:15.202002 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7f7c7b8964-kxkhw" Jul 13 07:03:15.203328 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:15.203281 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7f7c7b8964-kxkhw" Jul 13 07:03:15.217745 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:15.217611 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-7f7c7b8964-kxkhw" podStartSLOduration=45.217587694 podStartE2EDuration="45.217587694s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:02:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:03:14.258131016 +0000 UTC m=+69.093694858" watchObservedRunningTime="2026-07-13 07:03:15.217587694 +0000 UTC m=+70.053151526" Jul 13 07:03:16.663060 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:16.663032 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-68f4896b47-csf2q"] Jul 13 07:03:16.665985 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:16.665966 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68f4896b47-csf2q" Jul 13 07:03:16.668817 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:16.668651 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 13 07:03:16.668817 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:16.668664 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 13 07:03:16.669010 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:16.668817 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 13 07:03:16.669068 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:16.669043 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 13 07:03:16.679291 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:16.679264 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-68f4896b47-csf2q"] Jul 13 07:03:16.765879 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:16.765847 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-dc982"] Jul 13 07:03:16.770866 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:16.770842 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-dc982" Jul 13 07:03:16.773051 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:16.773020 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 13 07:03:16.773194 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:16.773020 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-7chdl\"" Jul 13 07:03:16.773416 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:16.773399 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 13 07:03:16.783732 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:16.783675 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-dc982"] Jul 13 07:03:16.815950 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:16.815852 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/026e124d-3c4b-4b01-9f55-2930104d0ec8-tmp\") pod \"klusterlet-addon-workmgr-68f4896b47-csf2q\" (UID: \"026e124d-3c4b-4b01-9f55-2930104d0ec8\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68f4896b47-csf2q" Jul 13 07:03:16.816672 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:16.816638 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7phm\" (UniqueName: \"kubernetes.io/projected/026e124d-3c4b-4b01-9f55-2930104d0ec8-kube-api-access-q7phm\") pod \"klusterlet-addon-workmgr-68f4896b47-csf2q\" (UID: \"026e124d-3c4b-4b01-9f55-2930104d0ec8\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68f4896b47-csf2q" Jul 13 07:03:16.816856 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:16.816811 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/026e124d-3c4b-4b01-9f55-2930104d0ec8-klusterlet-config\") pod \"klusterlet-addon-workmgr-68f4896b47-csf2q\" (UID: \"026e124d-3c4b-4b01-9f55-2930104d0ec8\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68f4896b47-csf2q" Jul 13 07:03:16.918244 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:16.917672 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9znt\" (UniqueName: \"kubernetes.io/projected/4c202845-1e5e-4d2d-9cd3-5ec97926bc84-kube-api-access-l9znt\") pod \"downloads-68c9cbb497-dc982\" (UID: \"4c202845-1e5e-4d2d-9cd3-5ec97926bc84\") " pod="openshift-console/downloads-68c9cbb497-dc982" Jul 13 07:03:16.918244 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:16.917754 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/026e124d-3c4b-4b01-9f55-2930104d0ec8-tmp\") pod \"klusterlet-addon-workmgr-68f4896b47-csf2q\" (UID: \"026e124d-3c4b-4b01-9f55-2930104d0ec8\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68f4896b47-csf2q" Jul 13 07:03:16.918244 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:16.917800 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7phm\" (UniqueName: \"kubernetes.io/projected/026e124d-3c4b-4b01-9f55-2930104d0ec8-kube-api-access-q7phm\") pod \"klusterlet-addon-workmgr-68f4896b47-csf2q\" (UID: \"026e124d-3c4b-4b01-9f55-2930104d0ec8\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68f4896b47-csf2q" Jul 13 07:03:16.918244 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:16.917862 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/026e124d-3c4b-4b01-9f55-2930104d0ec8-klusterlet-config\") pod \"klusterlet-addon-workmgr-68f4896b47-csf2q\" (UID: \"026e124d-3c4b-4b01-9f55-2930104d0ec8\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68f4896b47-csf2q" Jul 13 07:03:16.918244 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:16.918149 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/026e124d-3c4b-4b01-9f55-2930104d0ec8-tmp\") pod \"klusterlet-addon-workmgr-68f4896b47-csf2q\" (UID: \"026e124d-3c4b-4b01-9f55-2930104d0ec8\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68f4896b47-csf2q" Jul 13 07:03:16.921027 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:16.920993 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/026e124d-3c4b-4b01-9f55-2930104d0ec8-klusterlet-config\") pod \"klusterlet-addon-workmgr-68f4896b47-csf2q\" (UID: \"026e124d-3c4b-4b01-9f55-2930104d0ec8\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68f4896b47-csf2q" Jul 13 07:03:16.927394 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:16.927363 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7phm\" (UniqueName: \"kubernetes.io/projected/026e124d-3c4b-4b01-9f55-2930104d0ec8-kube-api-access-q7phm\") pod \"klusterlet-addon-workmgr-68f4896b47-csf2q\" (UID: \"026e124d-3c4b-4b01-9f55-2930104d0ec8\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68f4896b47-csf2q" Jul 13 07:03:16.979064 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:16.979034 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68f4896b47-csf2q" Jul 13 07:03:17.018386 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:17.018352 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9znt\" (UniqueName: \"kubernetes.io/projected/4c202845-1e5e-4d2d-9cd3-5ec97926bc84-kube-api-access-l9znt\") pod \"downloads-68c9cbb497-dc982\" (UID: \"4c202845-1e5e-4d2d-9cd3-5ec97926bc84\") " pod="openshift-console/downloads-68c9cbb497-dc982" Jul 13 07:03:17.037644 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:17.037608 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9znt\" (UniqueName: \"kubernetes.io/projected/4c202845-1e5e-4d2d-9cd3-5ec97926bc84-kube-api-access-l9znt\") pod \"downloads-68c9cbb497-dc982\" (UID: \"4c202845-1e5e-4d2d-9cd3-5ec97926bc84\") " pod="openshift-console/downloads-68c9cbb497-dc982" Jul 13 07:03:17.081127 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:17.081088 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-dc982" Jul 13 07:03:18.379337 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:18.379185 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-dc982"] Jul 13 07:03:18.382888 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:03:18.382847 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c202845_1e5e_4d2d_9cd3_5ec97926bc84.slice/crio-82c2b46aa37677313c170c55b10075f595ac233a8efd7994e4f5f7d5402e8058 WatchSource:0}: Error finding container 82c2b46aa37677313c170c55b10075f595ac233a8efd7994e4f5f7d5402e8058: Status 404 returned error can't find the container with id 82c2b46aa37677313c170c55b10075f595ac233a8efd7994e4f5f7d5402e8058 Jul 13 07:03:18.400996 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:18.400967 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-68f4896b47-csf2q"] Jul 13 07:03:18.403811 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:03:18.403770 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod026e124d_3c4b_4b01_9f55_2930104d0ec8.slice/crio-e4e01713b80534f95a73f1f395bc78978f01e65279de94f704826c376baacadd WatchSource:0}: Error finding container e4e01713b80534f95a73f1f395bc78978f01e65279de94f704826c376baacadd: Status 404 returned error can't find the container with id e4e01713b80534f95a73f1f395bc78978f01e65279de94f704826c376baacadd Jul 13 07:03:18.795986 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:18.795947 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-ph8pt"] Jul 13 07:03:18.813431 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:18.813397 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-ph8pt"] Jul 13 07:03:18.813600 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:18.813502 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-ph8pt" Jul 13 07:03:18.815319 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:18.815296 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 13 07:03:18.815447 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:18.815350 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-hx6n6\"" Jul 13 07:03:18.937229 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:18.937185 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a558e17f-d128-45d0-ac7d-835bf2094fdf-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-ph8pt\" (UID: \"a558e17f-d128-45d0-ac7d-835bf2094fdf\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-ph8pt" Jul 13 07:03:19.037805 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:19.037634 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a558e17f-d128-45d0-ac7d-835bf2094fdf-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-ph8pt\" (UID: \"a558e17f-d128-45d0-ac7d-835bf2094fdf\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-ph8pt" Jul 13 07:03:19.038008 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:03:19.037821 2586 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 13 07:03:19.038008 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:03:19.037901 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a558e17f-d128-45d0-ac7d-835bf2094fdf-tls-certificates podName:a558e17f-d128-45d0-ac7d-835bf2094fdf nodeName:}" failed. No retries permitted until 2026-07-13 07:03:19.537879844 +0000 UTC m=+74.373443660 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/a558e17f-d128-45d0-ac7d-835bf2094fdf-tls-certificates") pod "prometheus-operator-admission-webhook-77d98bc786-ph8pt" (UID: "a558e17f-d128-45d0-ac7d-835bf2094fdf") : secret "prometheus-operator-admission-webhook-tls" not found Jul 13 07:03:19.223004 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:19.222914 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-lb8th" event={"ID":"c3724c5e-1a2b-410b-a745-b0f90bf27358","Type":"ContainerStarted","Data":"fb0ebeaf6f5274e40ccf8f67f04ee2d3c421bc24b8f4d777d032a908c5a57edc"} Jul 13 07:03:19.225591 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:19.225369 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-6fw29" event={"ID":"ec04b338-54ca-471f-b138-eabbf7009550","Type":"ContainerStarted","Data":"92bd43cce13c466c87288c3fef0ee7df317c00b9d0039ec1e8b8182b9af229a6"} Jul 13 07:03:19.228269 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:19.228202 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-dc982" event={"ID":"4c202845-1e5e-4d2d-9cd3-5ec97926bc84","Type":"ContainerStarted","Data":"82c2b46aa37677313c170c55b10075f595ac233a8efd7994e4f5f7d5402e8058"} Jul 13 07:03:19.231643 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:19.231111 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-74xdm" event={"ID":"5b0bbb6f-a4f2-428a-8c0b-8561d031061c","Type":"ContainerStarted","Data":"eae20fb15d1969faa5e301ca83626eb1089cf3a26268f199f3f03690777e7e49"} Jul 13 07:03:19.233534 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:19.233500 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68f4896b47-csf2q" event={"ID":"026e124d-3c4b-4b01-9f55-2930104d0ec8","Type":"ContainerStarted","Data":"e4e01713b80534f95a73f1f395bc78978f01e65279de94f704826c376baacadd"} Jul 13 07:03:19.236974 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:19.236902 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wg6mf" event={"ID":"a7c70271-ef81-4393-9206-d7957e5c86e1","Type":"ContainerStarted","Data":"40173e816ac35d0f30ebf471d7c419a15a79f46af94a74626c0d144e129a1b74"} Jul 13 07:03:19.236974 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:19.236963 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wg6mf" event={"ID":"a7c70271-ef81-4393-9206-d7957e5c86e1","Type":"ContainerStarted","Data":"41c6614bfe32443718fa2d68bb247db8435a86635c51830996ab9594f3dfd3bd"} Jul 13 07:03:19.237924 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:19.237907 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-wg6mf" Jul 13 07:03:19.238385 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:19.238305 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-lb8th" podStartSLOduration=66.839398958 podStartE2EDuration="1m13.238289256s" totalImagesPullingTime="6.398890298s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 07:02:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:03:19.236980742 +0000 UTC m=+74.072544574" watchObservedRunningTime="2026-07-13 07:03:19.238289256 +0000 UTC m=+74.073853080" Jul 13 07:03:19.240669 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:19.240515 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-8grn6" event={"ID":"ddf4f6fd-5259-4c72-94df-245960dc7642","Type":"ContainerStarted","Data":"2cf2896d634414229ba482eed971b350a91d21faef2072215684828ba3586670"} Jul 13 07:03:19.240669 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:19.240564 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-8grn6" event={"ID":"ddf4f6fd-5259-4c72-94df-245960dc7642","Type":"ContainerStarted","Data":"53340819142bafc1e5478122fca9b83bf51e322799e307014f12f2c618ab6343"} Jul 13 07:03:19.251917 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:19.251841 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-6fw29" podStartSLOduration=33.661296813999996 podStartE2EDuration="38.251825021s" totalImagesPullingTime="4.590528207s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:02:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:03:19.250242774 +0000 UTC m=+74.085806606" watchObservedRunningTime="2026-07-13 07:03:19.251825021 +0000 UTC m=+74.087388852" Jul 13 07:03:19.265769 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:19.265561 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-74xdm" podStartSLOduration=44.499331696 podStartE2EDuration="49.265538585s" totalImagesPullingTime="4.766206889s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:02:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:03:19.264577648 +0000 UTC m=+74.100141491" watchObservedRunningTime="2026-07-13 07:03:19.265538585 +0000 UTC m=+74.101102417" Jul 13 07:03:19.286138 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:19.285142 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-wg6mf" podStartSLOduration=34.553384418 podStartE2EDuration="39.285119147s" totalImagesPullingTime="4.731734729s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:02:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:03:19.282346239 +0000 UTC m=+74.117910070" watchObservedRunningTime="2026-07-13 07:03:19.285119147 +0000 UTC m=+74.120682980" Jul 13 07:03:19.299009 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:19.298897 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-8grn6" podStartSLOduration=44.783297808 podStartE2EDuration="49.298878565s" totalImagesPullingTime="4.515580757s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:02:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:03:19.298424415 +0000 UTC m=+74.133988248" watchObservedRunningTime="2026-07-13 07:03:19.298878565 +0000 UTC m=+74.134442397" Jul 13 07:03:19.544926 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:19.544880 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a558e17f-d128-45d0-ac7d-835bf2094fdf-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-ph8pt\" (UID: \"a558e17f-d128-45d0-ac7d-835bf2094fdf\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-ph8pt" Jul 13 07:03:19.545317 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:03:19.545113 2586 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 13 07:03:19.545317 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:03:19.545180 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a558e17f-d128-45d0-ac7d-835bf2094fdf-tls-certificates podName:a558e17f-d128-45d0-ac7d-835bf2094fdf nodeName:}" failed. No retries permitted until 2026-07-13 07:03:20.54516069 +0000 UTC m=+75.380724516 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/a558e17f-d128-45d0-ac7d-835bf2094fdf-tls-certificates") pod "prometheus-operator-admission-webhook-77d98bc786-ph8pt" (UID: "a558e17f-d128-45d0-ac7d-835bf2094fdf") : secret "prometheus-operator-admission-webhook-tls" not found Jul 13 07:03:20.556117 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:20.555520 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a558e17f-d128-45d0-ac7d-835bf2094fdf-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-ph8pt\" (UID: \"a558e17f-d128-45d0-ac7d-835bf2094fdf\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-ph8pt" Jul 13 07:03:20.559407 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:20.559355 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a558e17f-d128-45d0-ac7d-835bf2094fdf-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-ph8pt\" (UID: \"a558e17f-d128-45d0-ac7d-835bf2094fdf\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-ph8pt" Jul 13 07:03:20.623252 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:20.623020 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-ph8pt" Jul 13 07:03:20.746417 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:20.745251 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5f68db78d8-p5vg9"] Jul 13 07:03:20.770087 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:20.769377 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f68db78d8-p5vg9"] Jul 13 07:03:20.770087 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:20.769477 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f68db78d8-p5vg9" Jul 13 07:03:20.774058 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:20.772015 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-cxpws\"" Jul 13 07:03:20.774058 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:20.772226 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 13 07:03:20.774058 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:20.772572 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 13 07:03:20.774058 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:20.772788 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 13 07:03:20.774058 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:20.772988 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 13 07:03:20.774058 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:20.773173 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 13 07:03:20.858637 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:20.858552 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/105d89f1-4d9f-4d9a-abe4-5d88a35da28f-console-serving-cert\") pod \"console-5f68db78d8-p5vg9\" (UID: \"105d89f1-4d9f-4d9a-abe4-5d88a35da28f\") " pod="openshift-console/console-5f68db78d8-p5vg9" Jul 13 07:03:20.858637 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:20.858625 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/105d89f1-4d9f-4d9a-abe4-5d88a35da28f-console-oauth-config\") pod \"console-5f68db78d8-p5vg9\" (UID: \"105d89f1-4d9f-4d9a-abe4-5d88a35da28f\") " pod="openshift-console/console-5f68db78d8-p5vg9" Jul 13 07:03:20.858876 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:20.858654 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqsqk\" (UniqueName: \"kubernetes.io/projected/105d89f1-4d9f-4d9a-abe4-5d88a35da28f-kube-api-access-zqsqk\") pod \"console-5f68db78d8-p5vg9\" (UID: \"105d89f1-4d9f-4d9a-abe4-5d88a35da28f\") " pod="openshift-console/console-5f68db78d8-p5vg9" Jul 13 07:03:20.858876 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:20.858703 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/105d89f1-4d9f-4d9a-abe4-5d88a35da28f-service-ca\") pod \"console-5f68db78d8-p5vg9\" (UID: \"105d89f1-4d9f-4d9a-abe4-5d88a35da28f\") " pod="openshift-console/console-5f68db78d8-p5vg9" Jul 13 07:03:20.858876 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:20.858755 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/105d89f1-4d9f-4d9a-abe4-5d88a35da28f-oauth-serving-cert\") pod \"console-5f68db78d8-p5vg9\" (UID: \"105d89f1-4d9f-4d9a-abe4-5d88a35da28f\") " pod="openshift-console/console-5f68db78d8-p5vg9" Jul 13 07:03:20.858876 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:20.858836 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/105d89f1-4d9f-4d9a-abe4-5d88a35da28f-console-config\") pod \"console-5f68db78d8-p5vg9\" (UID: \"105d89f1-4d9f-4d9a-abe4-5d88a35da28f\") " pod="openshift-console/console-5f68db78d8-p5vg9" Jul 13 07:03:20.960447 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:20.960264 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/105d89f1-4d9f-4d9a-abe4-5d88a35da28f-console-config\") pod \"console-5f68db78d8-p5vg9\" (UID: \"105d89f1-4d9f-4d9a-abe4-5d88a35da28f\") " pod="openshift-console/console-5f68db78d8-p5vg9" Jul 13 07:03:20.960447 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:20.960361 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/105d89f1-4d9f-4d9a-abe4-5d88a35da28f-console-serving-cert\") pod \"console-5f68db78d8-p5vg9\" (UID: \"105d89f1-4d9f-4d9a-abe4-5d88a35da28f\") " pod="openshift-console/console-5f68db78d8-p5vg9" Jul 13 07:03:20.961717 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:20.961101 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/105d89f1-4d9f-4d9a-abe4-5d88a35da28f-console-oauth-config\") pod \"console-5f68db78d8-p5vg9\" (UID: \"105d89f1-4d9f-4d9a-abe4-5d88a35da28f\") " pod="openshift-console/console-5f68db78d8-p5vg9" Jul 13 07:03:20.961717 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:20.961136 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqsqk\" (UniqueName: \"kubernetes.io/projected/105d89f1-4d9f-4d9a-abe4-5d88a35da28f-kube-api-access-zqsqk\") pod \"console-5f68db78d8-p5vg9\" (UID: \"105d89f1-4d9f-4d9a-abe4-5d88a35da28f\") " pod="openshift-console/console-5f68db78d8-p5vg9" Jul 13 07:03:20.961717 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:20.961166 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/105d89f1-4d9f-4d9a-abe4-5d88a35da28f-service-ca\") pod \"console-5f68db78d8-p5vg9\" (UID: \"105d89f1-4d9f-4d9a-abe4-5d88a35da28f\") " pod="openshift-console/console-5f68db78d8-p5vg9" Jul 13 07:03:20.961717 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:20.961195 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/105d89f1-4d9f-4d9a-abe4-5d88a35da28f-oauth-serving-cert\") pod \"console-5f68db78d8-p5vg9\" (UID: \"105d89f1-4d9f-4d9a-abe4-5d88a35da28f\") " pod="openshift-console/console-5f68db78d8-p5vg9" Jul 13 07:03:20.963258 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:20.961829 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/105d89f1-4d9f-4d9a-abe4-5d88a35da28f-oauth-serving-cert\") pod \"console-5f68db78d8-p5vg9\" (UID: \"105d89f1-4d9f-4d9a-abe4-5d88a35da28f\") " pod="openshift-console/console-5f68db78d8-p5vg9" Jul 13 07:03:20.963258 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:20.963048 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/105d89f1-4d9f-4d9a-abe4-5d88a35da28f-service-ca\") pod \"console-5f68db78d8-p5vg9\" (UID: \"105d89f1-4d9f-4d9a-abe4-5d88a35da28f\") " pod="openshift-console/console-5f68db78d8-p5vg9" Jul 13 07:03:20.963258 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:20.963133 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/105d89f1-4d9f-4d9a-abe4-5d88a35da28f-console-config\") pod \"console-5f68db78d8-p5vg9\" (UID: \"105d89f1-4d9f-4d9a-abe4-5d88a35da28f\") " pod="openshift-console/console-5f68db78d8-p5vg9" Jul 13 07:03:20.963902 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:20.963862 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/105d89f1-4d9f-4d9a-abe4-5d88a35da28f-console-serving-cert\") pod \"console-5f68db78d8-p5vg9\" (UID: \"105d89f1-4d9f-4d9a-abe4-5d88a35da28f\") " pod="openshift-console/console-5f68db78d8-p5vg9" Jul 13 07:03:20.968348 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:20.968304 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/105d89f1-4d9f-4d9a-abe4-5d88a35da28f-console-oauth-config\") pod \"console-5f68db78d8-p5vg9\" (UID: \"105d89f1-4d9f-4d9a-abe4-5d88a35da28f\") " pod="openshift-console/console-5f68db78d8-p5vg9" Jul 13 07:03:20.977266 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:20.977238 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqsqk\" (UniqueName: \"kubernetes.io/projected/105d89f1-4d9f-4d9a-abe4-5d88a35da28f-kube-api-access-zqsqk\") pod \"console-5f68db78d8-p5vg9\" (UID: \"105d89f1-4d9f-4d9a-abe4-5d88a35da28f\") " pod="openshift-console/console-5f68db78d8-p5vg9" Jul 13 07:03:21.043108 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:21.043026 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-ph8pt"] Jul 13 07:03:21.084534 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:21.084497 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f68db78d8-p5vg9" Jul 13 07:03:22.162233 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:22.162092 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5766df6b45-b59br"] Jul 13 07:03:22.198196 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:22.198141 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5766df6b45-b59br"] Jul 13 07:03:22.198367 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:22.198239 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5766df6b45-b59br" Jul 13 07:03:22.205894 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:22.205863 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 13 07:03:22.273402 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:22.273365 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b1cd91cc-38f3-4b8a-84f1-445884cb80f7-console-serving-cert\") pod \"console-5766df6b45-b59br\" (UID: \"b1cd91cc-38f3-4b8a-84f1-445884cb80f7\") " pod="openshift-console/console-5766df6b45-b59br" Jul 13 07:03:22.273402 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:22.273413 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1cd91cc-38f3-4b8a-84f1-445884cb80f7-trusted-ca-bundle\") pod \"console-5766df6b45-b59br\" (UID: \"b1cd91cc-38f3-4b8a-84f1-445884cb80f7\") " pod="openshift-console/console-5766df6b45-b59br" Jul 13 07:03:22.273659 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:22.273529 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b1cd91cc-38f3-4b8a-84f1-445884cb80f7-console-oauth-config\") pod \"console-5766df6b45-b59br\" (UID: \"b1cd91cc-38f3-4b8a-84f1-445884cb80f7\") " pod="openshift-console/console-5766df6b45-b59br" Jul 13 07:03:22.273659 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:22.273591 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqlvt\" (UniqueName: \"kubernetes.io/projected/b1cd91cc-38f3-4b8a-84f1-445884cb80f7-kube-api-access-cqlvt\") pod \"console-5766df6b45-b59br\" (UID: \"b1cd91cc-38f3-4b8a-84f1-445884cb80f7\") " pod="openshift-console/console-5766df6b45-b59br" Jul 13 07:03:22.273794 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:22.273699 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b1cd91cc-38f3-4b8a-84f1-445884cb80f7-console-config\") pod \"console-5766df6b45-b59br\" (UID: \"b1cd91cc-38f3-4b8a-84f1-445884cb80f7\") " pod="openshift-console/console-5766df6b45-b59br" Jul 13 07:03:22.273794 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:22.273750 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b1cd91cc-38f3-4b8a-84f1-445884cb80f7-service-ca\") pod \"console-5766df6b45-b59br\" (UID: \"b1cd91cc-38f3-4b8a-84f1-445884cb80f7\") " pod="openshift-console/console-5766df6b45-b59br" Jul 13 07:03:22.273794 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:22.273777 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b1cd91cc-38f3-4b8a-84f1-445884cb80f7-oauth-serving-cert\") pod \"console-5766df6b45-b59br\" (UID: \"b1cd91cc-38f3-4b8a-84f1-445884cb80f7\") " pod="openshift-console/console-5766df6b45-b59br" Jul 13 07:03:22.375366 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:22.374553 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b1cd91cc-38f3-4b8a-84f1-445884cb80f7-service-ca\") pod \"console-5766df6b45-b59br\" (UID: \"b1cd91cc-38f3-4b8a-84f1-445884cb80f7\") " pod="openshift-console/console-5766df6b45-b59br" Jul 13 07:03:22.375366 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:22.374610 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b1cd91cc-38f3-4b8a-84f1-445884cb80f7-oauth-serving-cert\") pod \"console-5766df6b45-b59br\" (UID: \"b1cd91cc-38f3-4b8a-84f1-445884cb80f7\") " pod="openshift-console/console-5766df6b45-b59br" Jul 13 07:03:22.375366 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:22.374641 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b1cd91cc-38f3-4b8a-84f1-445884cb80f7-console-serving-cert\") pod \"console-5766df6b45-b59br\" (UID: \"b1cd91cc-38f3-4b8a-84f1-445884cb80f7\") " pod="openshift-console/console-5766df6b45-b59br" Jul 13 07:03:22.375366 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:22.374667 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1cd91cc-38f3-4b8a-84f1-445884cb80f7-trusted-ca-bundle\") pod \"console-5766df6b45-b59br\" (UID: \"b1cd91cc-38f3-4b8a-84f1-445884cb80f7\") " pod="openshift-console/console-5766df6b45-b59br" Jul 13 07:03:22.375366 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:22.374746 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b1cd91cc-38f3-4b8a-84f1-445884cb80f7-console-oauth-config\") pod \"console-5766df6b45-b59br\" (UID: \"b1cd91cc-38f3-4b8a-84f1-445884cb80f7\") " pod="openshift-console/console-5766df6b45-b59br" Jul 13 07:03:22.375366 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:22.374799 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqlvt\" (UniqueName: \"kubernetes.io/projected/b1cd91cc-38f3-4b8a-84f1-445884cb80f7-kube-api-access-cqlvt\") pod \"console-5766df6b45-b59br\" (UID: \"b1cd91cc-38f3-4b8a-84f1-445884cb80f7\") " pod="openshift-console/console-5766df6b45-b59br" Jul 13 07:03:22.375366 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:22.374875 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b1cd91cc-38f3-4b8a-84f1-445884cb80f7-console-config\") pod \"console-5766df6b45-b59br\" (UID: \"b1cd91cc-38f3-4b8a-84f1-445884cb80f7\") " pod="openshift-console/console-5766df6b45-b59br" Jul 13 07:03:22.377137 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:22.376793 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b1cd91cc-38f3-4b8a-84f1-445884cb80f7-oauth-serving-cert\") pod \"console-5766df6b45-b59br\" (UID: \"b1cd91cc-38f3-4b8a-84f1-445884cb80f7\") " pod="openshift-console/console-5766df6b45-b59br" Jul 13 07:03:22.377137 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:22.377032 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b1cd91cc-38f3-4b8a-84f1-445884cb80f7-console-config\") pod \"console-5766df6b45-b59br\" (UID: \"b1cd91cc-38f3-4b8a-84f1-445884cb80f7\") " pod="openshift-console/console-5766df6b45-b59br" Jul 13 07:03:22.381804 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:22.377633 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b1cd91cc-38f3-4b8a-84f1-445884cb80f7-service-ca\") pod \"console-5766df6b45-b59br\" (UID: \"b1cd91cc-38f3-4b8a-84f1-445884cb80f7\") " pod="openshift-console/console-5766df6b45-b59br" Jul 13 07:03:22.381804 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:22.379254 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b1cd91cc-38f3-4b8a-84f1-445884cb80f7-console-oauth-config\") pod \"console-5766df6b45-b59br\" (UID: \"b1cd91cc-38f3-4b8a-84f1-445884cb80f7\") " pod="openshift-console/console-5766df6b45-b59br" Jul 13 07:03:22.381804 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:22.380708 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1cd91cc-38f3-4b8a-84f1-445884cb80f7-trusted-ca-bundle\") pod \"console-5766df6b45-b59br\" (UID: \"b1cd91cc-38f3-4b8a-84f1-445884cb80f7\") " pod="openshift-console/console-5766df6b45-b59br" Jul 13 07:03:22.384391 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:22.384363 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b1cd91cc-38f3-4b8a-84f1-445884cb80f7-console-serving-cert\") pod \"console-5766df6b45-b59br\" (UID: \"b1cd91cc-38f3-4b8a-84f1-445884cb80f7\") " pod="openshift-console/console-5766df6b45-b59br" Jul 13 07:03:22.384517 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:22.384427 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqlvt\" (UniqueName: \"kubernetes.io/projected/b1cd91cc-38f3-4b8a-84f1-445884cb80f7-kube-api-access-cqlvt\") pod \"console-5766df6b45-b59br\" (UID: \"b1cd91cc-38f3-4b8a-84f1-445884cb80f7\") " pod="openshift-console/console-5766df6b45-b59br" Jul 13 07:03:22.465052 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:03:22.465016 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda558e17f_d128_45d0_ac7d_835bf2094fdf.slice/crio-47a45716114442282c7624f8d7562ad334f6557f3a2a8016d5768bfa7d1693c5 WatchSource:0}: Error finding container 47a45716114442282c7624f8d7562ad334f6557f3a2a8016d5768bfa7d1693c5: Status 404 returned error can't find the container with id 47a45716114442282c7624f8d7562ad334f6557f3a2a8016d5768bfa7d1693c5 Jul 13 07:03:22.512192 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:22.512152 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5766df6b45-b59br" Jul 13 07:03:22.613455 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:22.613363 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f68db78d8-p5vg9"] Jul 13 07:03:22.616013 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:03:22.615982 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod105d89f1_4d9f_4d9a_abe4_5d88a35da28f.slice/crio-93c21030ee806733597b303191d36c65339be14cd5151721fccefa0f559b58b3 WatchSource:0}: Error finding container 93c21030ee806733597b303191d36c65339be14cd5151721fccefa0f559b58b3: Status 404 returned error can't find the container with id 93c21030ee806733597b303191d36c65339be14cd5151721fccefa0f559b58b3 Jul 13 07:03:22.674353 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:22.674313 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5766df6b45-b59br"] Jul 13 07:03:22.799308 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:03:22.799233 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1cd91cc_38f3_4b8a_84f1_445884cb80f7.slice/crio-f48fdb0cebe14df2ef9f1ee7ec7e36f0688e4e8c392853e0033c513e4b74e56a WatchSource:0}: Error finding container f48fdb0cebe14df2ef9f1ee7ec7e36f0688e4e8c392853e0033c513e4b74e56a: Status 404 returned error can't find the container with id f48fdb0cebe14df2ef9f1ee7ec7e36f0688e4e8c392853e0033c513e4b74e56a Jul 13 07:03:23.083006 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:23.082933 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/16425325-d8c8-40cb-a1c0-bfbc147f1416-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t5kqb\" (UID: \"16425325-d8c8-40cb-a1c0-bfbc147f1416\") " pod="openshift-insights/insights-runtime-extractor-t5kqb" Jul 13 07:03:23.085241 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:23.085212 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/16425325-d8c8-40cb-a1c0-bfbc147f1416-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t5kqb\" (UID: \"16425325-d8c8-40cb-a1c0-bfbc147f1416\") " pod="openshift-insights/insights-runtime-extractor-t5kqb" Jul 13 07:03:23.221534 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:23.221500 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-krdct\"" Jul 13 07:03:23.230168 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:23.230139 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-t5kqb" Jul 13 07:03:23.263422 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:23.263126 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68f4896b47-csf2q" event={"ID":"026e124d-3c4b-4b01-9f55-2930104d0ec8","Type":"ContainerStarted","Data":"918681669a9159b80957b1ed65da10bcd1c997087b55139f8b3d0ca96ce12aa3"} Jul 13 07:03:23.263559 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:23.263495 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68f4896b47-csf2q" Jul 13 07:03:23.265099 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:23.265075 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68f4896b47-csf2q" Jul 13 07:03:23.265608 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:23.265580 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f68db78d8-p5vg9" event={"ID":"105d89f1-4d9f-4d9a-abe4-5d88a35da28f","Type":"ContainerStarted","Data":"93c21030ee806733597b303191d36c65339be14cd5151721fccefa0f559b58b3"} Jul 13 07:03:23.267388 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:23.267363 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-ph8pt" event={"ID":"a558e17f-d128-45d0-ac7d-835bf2094fdf","Type":"ContainerStarted","Data":"47a45716114442282c7624f8d7562ad334f6557f3a2a8016d5768bfa7d1693c5"} Jul 13 07:03:23.269163 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:23.269112 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5766df6b45-b59br" event={"ID":"b1cd91cc-38f3-4b8a-84f1-445884cb80f7","Type":"ContainerStarted","Data":"f48fdb0cebe14df2ef9f1ee7ec7e36f0688e4e8c392853e0033c513e4b74e56a"} Jul 13 07:03:23.280766 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:23.280713 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68f4896b47-csf2q" podStartSLOduration=2.858131352 podStartE2EDuration="7.280675544s" totalImagesPullingTime="4.422544192s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:03:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:03:23.278343266 +0000 UTC m=+78.113907097" watchObservedRunningTime="2026-07-13 07:03:23.280675544 +0000 UTC m=+78.116239379" Jul 13 07:03:23.415961 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:23.415884 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-t5kqb"] Jul 13 07:03:24.276452 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:24.276370 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t5kqb" event={"ID":"16425325-d8c8-40cb-a1c0-bfbc147f1416","Type":"ContainerStarted","Data":"48663070d72b9834fdeffa37d38913cc3967961bc034cc0591b1b2963c77dd8f"} Jul 13 07:03:25.287327 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:25.286946 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-ph8pt" event={"ID":"a558e17f-d128-45d0-ac7d-835bf2094fdf","Type":"ContainerStarted","Data":"fe571c032115af66ae243a9b359e8fea7c85a8e716c793482a70d272da299432"} Jul 13 07:03:25.287327 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:25.287251 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-ph8pt" Jul 13 07:03:25.295110 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:25.295083 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-ph8pt" Jul 13 07:03:25.303300 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:25.303248 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-ph8pt" podStartSLOduration=5.40532606 podStartE2EDuration="7.303228858s" totalImagesPullingTime="1.897902798s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:03:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:03:25.300526816 +0000 UTC m=+80.136090648" watchObservedRunningTime="2026-07-13 07:03:25.303228858 +0000 UTC m=+80.138792689" Jul 13 07:03:29.249973 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:29.249939 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-wg6mf" Jul 13 07:03:30.312994 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:30.312955 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f68db78d8-p5vg9" event={"ID":"105d89f1-4d9f-4d9a-abe4-5d88a35da28f","Type":"ContainerStarted","Data":"eaefa37be4e64e97feaff4582b50ccff9f16d5ffccb1046d944a7bc6bece938b"} Jul 13 07:03:30.314750 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:30.314717 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5766df6b45-b59br" event={"ID":"b1cd91cc-38f3-4b8a-84f1-445884cb80f7","Type":"ContainerStarted","Data":"c181288d55e99e775aef941af4e1c6009c6ef039f9e193b3b462cdd0d1ebddc1"} Jul 13 07:03:30.316844 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:30.316809 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t5kqb" event={"ID":"16425325-d8c8-40cb-a1c0-bfbc147f1416","Type":"ContainerStarted","Data":"0315dbe6eb4de74d84e9014048c53544e8913f24fa41da1625306a3e241a7939"} Jul 13 07:03:30.316947 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:30.316861 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t5kqb" event={"ID":"16425325-d8c8-40cb-a1c0-bfbc147f1416","Type":"ContainerStarted","Data":"41d29223bc10e71491e6769c1e082760bd75a81641b39d6fee12f8ae45822daf"} Jul 13 07:03:30.338700 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:30.338631 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5f68db78d8-p5vg9" podStartSLOduration=3.581755208 podStartE2EDuration="10.338607101s" totalImagesPullingTime="6.756851893s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:03:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:03:30.332088086 +0000 UTC m=+85.167651916" watchObservedRunningTime="2026-07-13 07:03:30.338607101 +0000 UTC m=+85.174170934" Jul 13 07:03:30.349663 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:30.349601 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5766df6b45-b59br" podStartSLOduration=1.786760296 podStartE2EDuration="8.349577398s" totalImagesPullingTime="6.562817102s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:03:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:03:30.348387707 +0000 UTC m=+85.183951538" watchObservedRunningTime="2026-07-13 07:03:30.349577398 +0000 UTC m=+85.185141228" Jul 13 07:03:31.085068 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:31.084911 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5f68db78d8-p5vg9" Jul 13 07:03:31.085068 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:31.085034 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5f68db78d8-p5vg9" Jul 13 07:03:31.092618 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:31.092577 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5f68db78d8-p5vg9" Jul 13 07:03:31.259868 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:31.259827 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-5qhl8"] Jul 13 07:03:31.263879 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:31.263847 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5qhl8" Jul 13 07:03:31.265972 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:31.265939 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 13 07:03:31.266125 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:31.266043 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 13 07:03:31.266125 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:31.266051 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 13 07:03:31.266228 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:31.266189 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-zxnwr\"" Jul 13 07:03:31.266523 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:31.266501 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 13 07:03:31.328126 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:31.328096 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5f68db78d8-p5vg9" Jul 13 07:03:31.459461 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:31.459417 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c1c21537-9e03-4149-b45f-00aa313d26b0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5qhl8\" (UID: \"c1c21537-9e03-4149-b45f-00aa313d26b0\") " pod="openshift-monitoring/node-exporter-5qhl8" Jul 13 07:03:31.459653 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:31.459579 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c1c21537-9e03-4149-b45f-00aa313d26b0-sys\") pod \"node-exporter-5qhl8\" (UID: \"c1c21537-9e03-4149-b45f-00aa313d26b0\") " pod="openshift-monitoring/node-exporter-5qhl8" Jul 13 07:03:31.459653 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:31.459613 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c1c21537-9e03-4149-b45f-00aa313d26b0-node-exporter-tls\") pod \"node-exporter-5qhl8\" (UID: \"c1c21537-9e03-4149-b45f-00aa313d26b0\") " pod="openshift-monitoring/node-exporter-5qhl8" Jul 13 07:03:31.459876 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:31.459730 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c1c21537-9e03-4149-b45f-00aa313d26b0-node-exporter-accelerators-collector-config\") pod \"node-exporter-5qhl8\" (UID: \"c1c21537-9e03-4149-b45f-00aa313d26b0\") " pod="openshift-monitoring/node-exporter-5qhl8" Jul 13 07:03:31.459876 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:31.459764 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c1c21537-9e03-4149-b45f-00aa313d26b0-root\") pod \"node-exporter-5qhl8\" (UID: \"c1c21537-9e03-4149-b45f-00aa313d26b0\") " pod="openshift-monitoring/node-exporter-5qhl8" Jul 13 07:03:31.459876 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:31.459803 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c1c21537-9e03-4149-b45f-00aa313d26b0-metrics-client-ca\") pod \"node-exporter-5qhl8\" (UID: \"c1c21537-9e03-4149-b45f-00aa313d26b0\") " pod="openshift-monitoring/node-exporter-5qhl8" Jul 13 07:03:31.459876 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:31.459865 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c1c21537-9e03-4149-b45f-00aa313d26b0-node-exporter-textfile\") pod \"node-exporter-5qhl8\" (UID: \"c1c21537-9e03-4149-b45f-00aa313d26b0\") " pod="openshift-monitoring/node-exporter-5qhl8" Jul 13 07:03:31.460086 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:31.459896 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c1c21537-9e03-4149-b45f-00aa313d26b0-node-exporter-wtmp\") pod \"node-exporter-5qhl8\" (UID: \"c1c21537-9e03-4149-b45f-00aa313d26b0\") " pod="openshift-monitoring/node-exporter-5qhl8" Jul 13 07:03:31.460086 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:31.459944 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dhsg\" (UniqueName: \"kubernetes.io/projected/c1c21537-9e03-4149-b45f-00aa313d26b0-kube-api-access-8dhsg\") pod \"node-exporter-5qhl8\" (UID: \"c1c21537-9e03-4149-b45f-00aa313d26b0\") " pod="openshift-monitoring/node-exporter-5qhl8" Jul 13 07:03:31.560615 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:31.560572 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c1c21537-9e03-4149-b45f-00aa313d26b0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5qhl8\" (UID: \"c1c21537-9e03-4149-b45f-00aa313d26b0\") " pod="openshift-monitoring/node-exporter-5qhl8" Jul 13 07:03:31.560884 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:31.560645 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c1c21537-9e03-4149-b45f-00aa313d26b0-sys\") pod \"node-exporter-5qhl8\" (UID: \"c1c21537-9e03-4149-b45f-00aa313d26b0\") " pod="openshift-monitoring/node-exporter-5qhl8" Jul 13 07:03:31.560884 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:31.560675 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c1c21537-9e03-4149-b45f-00aa313d26b0-node-exporter-tls\") pod \"node-exporter-5qhl8\" (UID: \"c1c21537-9e03-4149-b45f-00aa313d26b0\") " pod="openshift-monitoring/node-exporter-5qhl8" Jul 13 07:03:31.560884 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:31.560757 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c1c21537-9e03-4149-b45f-00aa313d26b0-node-exporter-accelerators-collector-config\") pod \"node-exporter-5qhl8\" (UID: \"c1c21537-9e03-4149-b45f-00aa313d26b0\") " pod="openshift-monitoring/node-exporter-5qhl8" Jul 13 07:03:31.560884 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:31.560794 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c1c21537-9e03-4149-b45f-00aa313d26b0-sys\") pod \"node-exporter-5qhl8\" (UID: \"c1c21537-9e03-4149-b45f-00aa313d26b0\") " pod="openshift-monitoring/node-exporter-5qhl8" Jul 13 07:03:31.560884 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:31.560796 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c1c21537-9e03-4149-b45f-00aa313d26b0-root\") pod \"node-exporter-5qhl8\" (UID: \"c1c21537-9e03-4149-b45f-00aa313d26b0\") " pod="openshift-monitoring/node-exporter-5qhl8" Jul 13 07:03:31.560884 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:31.560831 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c1c21537-9e03-4149-b45f-00aa313d26b0-root\") pod \"node-exporter-5qhl8\" (UID: \"c1c21537-9e03-4149-b45f-00aa313d26b0\") " pod="openshift-monitoring/node-exporter-5qhl8" Jul 13 07:03:31.560884 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:31.560843 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c1c21537-9e03-4149-b45f-00aa313d26b0-metrics-client-ca\") pod \"node-exporter-5qhl8\" (UID: \"c1c21537-9e03-4149-b45f-00aa313d26b0\") " pod="openshift-monitoring/node-exporter-5qhl8" Jul 13 07:03:31.560884 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:31.560881 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c1c21537-9e03-4149-b45f-00aa313d26b0-node-exporter-textfile\") pod \"node-exporter-5qhl8\" (UID: \"c1c21537-9e03-4149-b45f-00aa313d26b0\") " pod="openshift-monitoring/node-exporter-5qhl8" Jul 13 07:03:31.561374 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:31.560909 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c1c21537-9e03-4149-b45f-00aa313d26b0-node-exporter-wtmp\") pod \"node-exporter-5qhl8\" (UID: \"c1c21537-9e03-4149-b45f-00aa313d26b0\") " pod="openshift-monitoring/node-exporter-5qhl8" Jul 13 07:03:31.561374 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:31.560939 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dhsg\" (UniqueName: \"kubernetes.io/projected/c1c21537-9e03-4149-b45f-00aa313d26b0-kube-api-access-8dhsg\") pod \"node-exporter-5qhl8\" (UID: \"c1c21537-9e03-4149-b45f-00aa313d26b0\") " pod="openshift-monitoring/node-exporter-5qhl8" Jul 13 07:03:31.561374 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:31.561248 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c1c21537-9e03-4149-b45f-00aa313d26b0-node-exporter-wtmp\") pod \"node-exporter-5qhl8\" (UID: \"c1c21537-9e03-4149-b45f-00aa313d26b0\") " pod="openshift-monitoring/node-exporter-5qhl8" Jul 13 07:03:31.561675 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:31.561648 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c1c21537-9e03-4149-b45f-00aa313d26b0-node-exporter-accelerators-collector-config\") pod \"node-exporter-5qhl8\" (UID: \"c1c21537-9e03-4149-b45f-00aa313d26b0\") " pod="openshift-monitoring/node-exporter-5qhl8" Jul 13 07:03:31.561815 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:31.561785 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c1c21537-9e03-4149-b45f-00aa313d26b0-node-exporter-textfile\") pod \"node-exporter-5qhl8\" (UID: \"c1c21537-9e03-4149-b45f-00aa313d26b0\") " pod="openshift-monitoring/node-exporter-5qhl8" Jul 13 07:03:31.561815 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:31.561798 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c1c21537-9e03-4149-b45f-00aa313d26b0-metrics-client-ca\") pod \"node-exporter-5qhl8\" (UID: \"c1c21537-9e03-4149-b45f-00aa313d26b0\") " pod="openshift-monitoring/node-exporter-5qhl8" Jul 13 07:03:31.563847 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:31.563799 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c1c21537-9e03-4149-b45f-00aa313d26b0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5qhl8\" (UID: \"c1c21537-9e03-4149-b45f-00aa313d26b0\") " pod="openshift-monitoring/node-exporter-5qhl8" Jul 13 07:03:31.563990 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:31.563970 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c1c21537-9e03-4149-b45f-00aa313d26b0-node-exporter-tls\") pod \"node-exporter-5qhl8\" (UID: \"c1c21537-9e03-4149-b45f-00aa313d26b0\") " pod="openshift-monitoring/node-exporter-5qhl8" Jul 13 07:03:31.594659 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:31.594619 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dhsg\" (UniqueName: \"kubernetes.io/projected/c1c21537-9e03-4149-b45f-00aa313d26b0-kube-api-access-8dhsg\") pod \"node-exporter-5qhl8\" (UID: \"c1c21537-9e03-4149-b45f-00aa313d26b0\") " pod="openshift-monitoring/node-exporter-5qhl8" Jul 13 07:03:31.876559 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:31.876461 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5qhl8" Jul 13 07:03:32.513216 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:32.513178 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5766df6b45-b59br" Jul 13 07:03:32.513658 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:32.513246 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5766df6b45-b59br" Jul 13 07:03:32.521055 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:32.521031 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5766df6b45-b59br" Jul 13 07:03:33.120871 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:33.120833 2586 patch_prober.go:28] interesting pod/image-registry-69599bff85-z8nf7 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 13 07:03:33.121033 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:33.120890 2586 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-69599bff85-z8nf7" podUID="16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 13 07:03:33.338927 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:33.338898 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5766df6b45-b59br" Jul 13 07:03:33.382436 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:33.382360 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f68db78d8-p5vg9"] Jul 13 07:03:35.208387 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:35.208355 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-69599bff85-z8nf7" Jul 13 07:03:38.842799 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:38.842759 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-69599bff85-z8nf7"] Jul 13 07:03:39.719342 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:03:39.719299 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1c21537_9e03_4149_b45f_00aa313d26b0.slice/crio-ca283d1f3904db0901062376368dae6475990a62a7db998a7e3fbb78a74c25f4 WatchSource:0}: Error finding container ca283d1f3904db0901062376368dae6475990a62a7db998a7e3fbb78a74c25f4: Status 404 returned error can't find the container with id ca283d1f3904db0901062376368dae6475990a62a7db998a7e3fbb78a74c25f4 Jul 13 07:03:40.370277 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:40.370226 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t5kqb" event={"ID":"16425325-d8c8-40cb-a1c0-bfbc147f1416","Type":"ContainerStarted","Data":"36680d9dd43c482faf0c1e2d8ed55ac443704b572332b699e3e070a4d77e390c"} Jul 13 07:03:40.373694 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:40.373057 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-dc982" event={"ID":"4c202845-1e5e-4d2d-9cd3-5ec97926bc84","Type":"ContainerStarted","Data":"5298e1506746634a7e415dd0d3064c4e5056a7ef3442b2a744e668dfab07cfac"} Jul 13 07:03:40.373947 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:40.373922 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-dc982" Jul 13 07:03:40.376031 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:40.376005 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5qhl8" event={"ID":"c1c21537-9e03-4149-b45f-00aa313d26b0","Type":"ContainerStarted","Data":"ca283d1f3904db0901062376368dae6475990a62a7db998a7e3fbb78a74c25f4"} Jul 13 07:03:40.389325 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:40.387854 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-dc982" Jul 13 07:03:40.389627 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:40.389572 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-t5kqb" podStartSLOduration=33.234001988 podStartE2EDuration="49.389550571s" totalImagesPullingTime="16.155548583s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 07:02:51 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:03:40.387884574 +0000 UTC m=+95.223448406" watchObservedRunningTime="2026-07-13 07:03:40.389550571 +0000 UTC m=+95.225114406" Jul 13 07:03:40.405016 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:40.404942 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-dc982" podStartSLOduration=3.034161128 podStartE2EDuration="24.404925613s" totalImagesPullingTime="21.370764485s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:03:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:03:40.403740893 +0000 UTC m=+95.239304722" watchObservedRunningTime="2026-07-13 07:03:40.404925613 +0000 UTC m=+95.240489444" Jul 13 07:03:41.383537 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:41.383448 2586 generic.go:356] "Generic (PLEG): container finished" podID="c1c21537-9e03-4149-b45f-00aa313d26b0" containerID="6aad4e053e50053b35eab6b0625afa38f45d7e91a67b4842164b10ce9e4ad23d" exitCode=0 Jul 13 07:03:41.384058 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:41.383555 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5qhl8" event={"ID":"c1c21537-9e03-4149-b45f-00aa313d26b0","Type":"ContainerDied","Data":"6aad4e053e50053b35eab6b0625afa38f45d7e91a67b4842164b10ce9e4ad23d"} Jul 13 07:03:42.390515 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:42.390474 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5qhl8" event={"ID":"c1c21537-9e03-4149-b45f-00aa313d26b0","Type":"ContainerStarted","Data":"de4eed09cb65f0ad8f2be2c8a2bd20cd74c50be11f9edc2c80508143baffdfa4"} Jul 13 07:03:42.391084 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:42.390554 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5qhl8" event={"ID":"c1c21537-9e03-4149-b45f-00aa313d26b0","Type":"ContainerStarted","Data":"d5fae48277b94e461a23aa56fdf3405dbeee87fee7f36663de4868202636312e"} Jul 13 07:03:42.420322 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:42.420258 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-5qhl8" podStartSLOduration=10.001865626 podStartE2EDuration="11.420241942s" totalImagesPullingTime="1.348794476s" totalInitContainerRuntime="69.58184ms" isStatefulPod=true podCreationTimestamp="2026-07-13 07:03:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:03:42.419628507 +0000 UTC m=+97.255192351" watchObservedRunningTime="2026-07-13 07:03:42.420241942 +0000 UTC m=+97.255805772" Jul 13 07:03:43.177063 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:43.177030 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-6xdjl" Jul 13 07:03:47.905362 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:47.905314 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5766df6b45-b59br"] Jul 13 07:03:59.365866 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:59.365789 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5f68db78d8-p5vg9" podUID="105d89f1-4d9f-4d9a-abe4-5d88a35da28f" containerName="console" containerID="cri-o://eaefa37be4e64e97feaff4582b50ccff9f16d5ffccb1046d944a7bc6bece938b" gracePeriod=15 Jul 13 07:03:59.643967 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:59.643886 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f68db78d8-p5vg9" Jul 13 07:03:59.713256 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:59.713223 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/105d89f1-4d9f-4d9a-abe4-5d88a35da28f-console-config\") pod \"105d89f1-4d9f-4d9a-abe4-5d88a35da28f\" (UID: \"105d89f1-4d9f-4d9a-abe4-5d88a35da28f\") " Jul 13 07:03:59.713413 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:59.713269 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/105d89f1-4d9f-4d9a-abe4-5d88a35da28f-console-oauth-config\") pod \"105d89f1-4d9f-4d9a-abe4-5d88a35da28f\" (UID: \"105d89f1-4d9f-4d9a-abe4-5d88a35da28f\") " Jul 13 07:03:59.713413 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:59.713304 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/105d89f1-4d9f-4d9a-abe4-5d88a35da28f-service-ca\") pod \"105d89f1-4d9f-4d9a-abe4-5d88a35da28f\" (UID: \"105d89f1-4d9f-4d9a-abe4-5d88a35da28f\") " Jul 13 07:03:59.713413 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:59.713335 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zqsqk\" (UniqueName: \"kubernetes.io/projected/105d89f1-4d9f-4d9a-abe4-5d88a35da28f-kube-api-access-zqsqk\") pod \"105d89f1-4d9f-4d9a-abe4-5d88a35da28f\" (UID: \"105d89f1-4d9f-4d9a-abe4-5d88a35da28f\") " Jul 13 07:03:59.713413 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:59.713355 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/105d89f1-4d9f-4d9a-abe4-5d88a35da28f-oauth-serving-cert\") pod \"105d89f1-4d9f-4d9a-abe4-5d88a35da28f\" (UID: \"105d89f1-4d9f-4d9a-abe4-5d88a35da28f\") " Jul 13 07:03:59.713413 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:59.713391 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/105d89f1-4d9f-4d9a-abe4-5d88a35da28f-console-serving-cert\") pod \"105d89f1-4d9f-4d9a-abe4-5d88a35da28f\" (UID: \"105d89f1-4d9f-4d9a-abe4-5d88a35da28f\") " Jul 13 07:03:59.713776 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:59.713615 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/105d89f1-4d9f-4d9a-abe4-5d88a35da28f-console-config" (OuterVolumeSpecName: "console-config") pod "105d89f1-4d9f-4d9a-abe4-5d88a35da28f" (UID: "105d89f1-4d9f-4d9a-abe4-5d88a35da28f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 07:03:59.713851 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:59.713829 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/105d89f1-4d9f-4d9a-abe4-5d88a35da28f-service-ca" (OuterVolumeSpecName: "service-ca") pod "105d89f1-4d9f-4d9a-abe4-5d88a35da28f" (UID: "105d89f1-4d9f-4d9a-abe4-5d88a35da28f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 07:03:59.713901 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:59.713839 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/105d89f1-4d9f-4d9a-abe4-5d88a35da28f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "105d89f1-4d9f-4d9a-abe4-5d88a35da28f" (UID: "105d89f1-4d9f-4d9a-abe4-5d88a35da28f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 07:03:59.715817 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:59.715790 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/105d89f1-4d9f-4d9a-abe4-5d88a35da28f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "105d89f1-4d9f-4d9a-abe4-5d88a35da28f" (UID: "105d89f1-4d9f-4d9a-abe4-5d88a35da28f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 07:03:59.715817 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:59.715808 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/105d89f1-4d9f-4d9a-abe4-5d88a35da28f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "105d89f1-4d9f-4d9a-abe4-5d88a35da28f" (UID: "105d89f1-4d9f-4d9a-abe4-5d88a35da28f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 07:03:59.715940 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:59.715798 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/105d89f1-4d9f-4d9a-abe4-5d88a35da28f-kube-api-access-zqsqk" (OuterVolumeSpecName: "kube-api-access-zqsqk") pod "105d89f1-4d9f-4d9a-abe4-5d88a35da28f" (UID: "105d89f1-4d9f-4d9a-abe4-5d88a35da28f"). InnerVolumeSpecName "kube-api-access-zqsqk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:03:59.814485 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:59.814462 2586 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/105d89f1-4d9f-4d9a-abe4-5d88a35da28f-service-ca\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:03:59.814485 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:59.814486 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zqsqk\" (UniqueName: \"kubernetes.io/projected/105d89f1-4d9f-4d9a-abe4-5d88a35da28f-kube-api-access-zqsqk\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:03:59.814621 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:59.814496 2586 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/105d89f1-4d9f-4d9a-abe4-5d88a35da28f-oauth-serving-cert\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:03:59.814621 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:59.814504 2586 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/105d89f1-4d9f-4d9a-abe4-5d88a35da28f-console-serving-cert\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:03:59.814621 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:59.814512 2586 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/105d89f1-4d9f-4d9a-abe4-5d88a35da28f-console-config\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:03:59.814621 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:03:59.814520 2586 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/105d89f1-4d9f-4d9a-abe4-5d88a35da28f-console-oauth-config\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:04:00.469406 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:00.469371 2586 generic.go:356] "Generic (PLEG): container finished" podID="105d89f1-4d9f-4d9a-abe4-5d88a35da28f" containerID="eaefa37be4e64e97feaff4582b50ccff9f16d5ffccb1046d944a7bc6bece938b" exitCode=0 Jul 13 07:04:00.469837 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:00.469417 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f68db78d8-p5vg9" Jul 13 07:04:00.469837 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:00.469445 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f68db78d8-p5vg9" event={"ID":"105d89f1-4d9f-4d9a-abe4-5d88a35da28f","Type":"ContainerDied","Data":"eaefa37be4e64e97feaff4582b50ccff9f16d5ffccb1046d944a7bc6bece938b"} Jul 13 07:04:00.469837 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:00.469491 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f68db78d8-p5vg9" event={"ID":"105d89f1-4d9f-4d9a-abe4-5d88a35da28f","Type":"ContainerDied","Data":"93c21030ee806733597b303191d36c65339be14cd5151721fccefa0f559b58b3"} Jul 13 07:04:00.469837 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:00.469511 2586 scope.go:117] "RemoveContainer" containerID="eaefa37be4e64e97feaff4582b50ccff9f16d5ffccb1046d944a7bc6bece938b" Jul 13 07:04:00.477387 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:00.477369 2586 scope.go:117] "RemoveContainer" containerID="eaefa37be4e64e97feaff4582b50ccff9f16d5ffccb1046d944a7bc6bece938b" Jul 13 07:04:00.477656 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:04:00.477635 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eaefa37be4e64e97feaff4582b50ccff9f16d5ffccb1046d944a7bc6bece938b\": container with ID starting with eaefa37be4e64e97feaff4582b50ccff9f16d5ffccb1046d944a7bc6bece938b not found: ID does not exist" containerID="eaefa37be4e64e97feaff4582b50ccff9f16d5ffccb1046d944a7bc6bece938b" Jul 13 07:04:00.477714 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:00.477666 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eaefa37be4e64e97feaff4582b50ccff9f16d5ffccb1046d944a7bc6bece938b"} err="failed to get container status \"eaefa37be4e64e97feaff4582b50ccff9f16d5ffccb1046d944a7bc6bece938b\": rpc error: code = NotFound desc = could not find container \"eaefa37be4e64e97feaff4582b50ccff9f16d5ffccb1046d944a7bc6bece938b\": container with ID starting with eaefa37be4e64e97feaff4582b50ccff9f16d5ffccb1046d944a7bc6bece938b not found: ID does not exist" Jul 13 07:04:00.485664 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:00.485644 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f68db78d8-p5vg9"] Jul 13 07:04:00.489510 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:00.489492 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5f68db78d8-p5vg9"] Jul 13 07:04:01.475271 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:01.475235 2586 generic.go:356] "Generic (PLEG): container finished" podID="05bf1881-84d9-42e6-a7ed-1a348839015d" containerID="f6a8d27d269f5bd96834dd1c106b975fb41f61b2d1b111277a7289a883616c9d" exitCode=0 Jul 13 07:04:01.475698 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:01.475303 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-z7x8m" event={"ID":"05bf1881-84d9-42e6-a7ed-1a348839015d","Type":"ContainerDied","Data":"f6a8d27d269f5bd96834dd1c106b975fb41f61b2d1b111277a7289a883616c9d"} Jul 13 07:04:01.475781 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:01.475720 2586 scope.go:117] "RemoveContainer" containerID="f6a8d27d269f5bd96834dd1c106b975fb41f61b2d1b111277a7289a883616c9d" Jul 13 07:04:01.804028 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:01.803949 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="105d89f1-4d9f-4d9a-abe4-5d88a35da28f" path="/var/lib/kubelet/pods/105d89f1-4d9f-4d9a-abe4-5d88a35da28f/volumes" Jul 13 07:04:02.486646 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:02.486607 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-z7x8m" event={"ID":"05bf1881-84d9-42e6-a7ed-1a348839015d","Type":"ContainerStarted","Data":"ed00c37a5efaf4627a36eadf410e0c2b2fa9ca34ebcd55f21029776503395e59"} Jul 13 07:04:03.868043 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:03.867974 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-69599bff85-z8nf7" podUID="16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d" containerName="registry" containerID="cri-o://a7716f819c361c82a026fec7b59d95b8c8b05279fe2680bc8cc0637783b3d8c1" gracePeriod=30 Jul 13 07:04:04.146302 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:04.146218 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69599bff85-z8nf7" Jul 13 07:04:04.247363 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:04.247331 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-registry-certificates\") pod \"16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d\" (UID: \"16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d\") " Jul 13 07:04:04.247550 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:04.247371 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-installation-pull-secrets\") pod \"16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d\" (UID: \"16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d\") " Jul 13 07:04:04.247550 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:04.247395 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h5psx\" (UniqueName: \"kubernetes.io/projected/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-kube-api-access-h5psx\") pod \"16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d\" (UID: \"16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d\") " Jul 13 07:04:04.247550 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:04.247426 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-ca-trust-extracted\") pod \"16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d\" (UID: \"16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d\") " Jul 13 07:04:04.247550 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:04.247467 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-registry-tls\") pod \"16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d\" (UID: \"16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d\") " Jul 13 07:04:04.247550 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:04.247534 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-bound-sa-token\") pod \"16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d\" (UID: \"16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d\") " Jul 13 07:04:04.247830 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:04.247574 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-trusted-ca\") pod \"16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d\" (UID: \"16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d\") " Jul 13 07:04:04.247830 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:04.247629 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-image-registry-private-configuration\") pod \"16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d\" (UID: \"16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d\") " Jul 13 07:04:04.247980 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:04.247944 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d" (UID: "16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 07:04:04.248099 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:04.248052 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d" (UID: "16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 07:04:04.250171 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:04.250138 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d" (UID: "16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 07:04:04.250300 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:04.250199 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-kube-api-access-h5psx" (OuterVolumeSpecName: "kube-api-access-h5psx") pod "16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d" (UID: "16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d"). InnerVolumeSpecName "kube-api-access-h5psx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:04:04.250378 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:04.250294 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d" (UID: "16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:04:04.250378 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:04.250318 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d" (UID: "16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 07:04:04.250484 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:04.250388 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d" (UID: "16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:04:04.257973 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:04.257941 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d" (UID: "16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:04:04.348455 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:04.348418 2586 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-registry-certificates\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:04:04.348455 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:04.348449 2586 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-installation-pull-secrets\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:04:04.348455 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:04.348459 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h5psx\" (UniqueName: \"kubernetes.io/projected/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-kube-api-access-h5psx\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:04:04.348693 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:04.348468 2586 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-ca-trust-extracted\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:04:04.348693 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:04.348479 2586 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-registry-tls\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:04:04.348693 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:04.348487 2586 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-bound-sa-token\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:04:04.348693 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:04.348495 2586 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-trusted-ca\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:04:04.348693 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:04.348503 2586 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d-image-registry-private-configuration\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:04:04.495365 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:04.495330 2586 generic.go:356] "Generic (PLEG): container finished" podID="16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d" containerID="a7716f819c361c82a026fec7b59d95b8c8b05279fe2680bc8cc0637783b3d8c1" exitCode=0 Jul 13 07:04:04.495552 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:04.495383 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69599bff85-z8nf7" Jul 13 07:04:04.495552 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:04.495386 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69599bff85-z8nf7" event={"ID":"16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d","Type":"ContainerDied","Data":"a7716f819c361c82a026fec7b59d95b8c8b05279fe2680bc8cc0637783b3d8c1"} Jul 13 07:04:04.495552 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:04.495436 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69599bff85-z8nf7" event={"ID":"16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d","Type":"ContainerDied","Data":"0ae13f6f86964f7cc701053c263adee25400a1031d40e4c936973b7b3c356688"} Jul 13 07:04:04.495552 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:04.495466 2586 scope.go:117] "RemoveContainer" containerID="a7716f819c361c82a026fec7b59d95b8c8b05279fe2680bc8cc0637783b3d8c1" Jul 13 07:04:04.504069 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:04.504050 2586 scope.go:117] "RemoveContainer" containerID="a7716f819c361c82a026fec7b59d95b8c8b05279fe2680bc8cc0637783b3d8c1" Jul 13 07:04:04.504372 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:04:04.504353 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7716f819c361c82a026fec7b59d95b8c8b05279fe2680bc8cc0637783b3d8c1\": container with ID starting with a7716f819c361c82a026fec7b59d95b8c8b05279fe2680bc8cc0637783b3d8c1 not found: ID does not exist" containerID="a7716f819c361c82a026fec7b59d95b8c8b05279fe2680bc8cc0637783b3d8c1" Jul 13 07:04:04.504447 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:04.504384 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7716f819c361c82a026fec7b59d95b8c8b05279fe2680bc8cc0637783b3d8c1"} err="failed to get container status \"a7716f819c361c82a026fec7b59d95b8c8b05279fe2680bc8cc0637783b3d8c1\": rpc error: code = NotFound desc = could not find container \"a7716f819c361c82a026fec7b59d95b8c8b05279fe2680bc8cc0637783b3d8c1\": container with ID starting with a7716f819c361c82a026fec7b59d95b8c8b05279fe2680bc8cc0637783b3d8c1 not found: ID does not exist" Jul 13 07:04:04.514698 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:04.514653 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-69599bff85-z8nf7"] Jul 13 07:04:04.519877 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:04.519850 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-69599bff85-z8nf7"] Jul 13 07:04:05.804740 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:05.804700 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d" path="/var/lib/kubelet/pods/16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d/volumes" Jul 13 07:04:12.933878 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:12.933807 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5766df6b45-b59br" podUID="b1cd91cc-38f3-4b8a-84f1-445884cb80f7" containerName="console" containerID="cri-o://c181288d55e99e775aef941af4e1c6009c6ef039f9e193b3b462cdd0d1ebddc1" gracePeriod=15 Jul 13 07:04:13.217238 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:13.217205 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5766df6b45-b59br" Jul 13 07:04:13.330369 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:13.330324 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b1cd91cc-38f3-4b8a-84f1-445884cb80f7-console-oauth-config\") pod \"b1cd91cc-38f3-4b8a-84f1-445884cb80f7\" (UID: \"b1cd91cc-38f3-4b8a-84f1-445884cb80f7\") " Jul 13 07:04:13.330556 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:13.330389 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b1cd91cc-38f3-4b8a-84f1-445884cb80f7-console-config\") pod \"b1cd91cc-38f3-4b8a-84f1-445884cb80f7\" (UID: \"b1cd91cc-38f3-4b8a-84f1-445884cb80f7\") " Jul 13 07:04:13.330556 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:13.330422 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b1cd91cc-38f3-4b8a-84f1-445884cb80f7-console-serving-cert\") pod \"b1cd91cc-38f3-4b8a-84f1-445884cb80f7\" (UID: \"b1cd91cc-38f3-4b8a-84f1-445884cb80f7\") " Jul 13 07:04:13.330556 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:13.330448 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b1cd91cc-38f3-4b8a-84f1-445884cb80f7-service-ca\") pod \"b1cd91cc-38f3-4b8a-84f1-445884cb80f7\" (UID: \"b1cd91cc-38f3-4b8a-84f1-445884cb80f7\") " Jul 13 07:04:13.330556 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:13.330476 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b1cd91cc-38f3-4b8a-84f1-445884cb80f7-oauth-serving-cert\") pod \"b1cd91cc-38f3-4b8a-84f1-445884cb80f7\" (UID: \"b1cd91cc-38f3-4b8a-84f1-445884cb80f7\") " Jul 13 07:04:13.330556 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:13.330531 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1cd91cc-38f3-4b8a-84f1-445884cb80f7-trusted-ca-bundle\") pod \"b1cd91cc-38f3-4b8a-84f1-445884cb80f7\" (UID: \"b1cd91cc-38f3-4b8a-84f1-445884cb80f7\") " Jul 13 07:04:13.330845 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:13.330588 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cqlvt\" (UniqueName: \"kubernetes.io/projected/b1cd91cc-38f3-4b8a-84f1-445884cb80f7-kube-api-access-cqlvt\") pod \"b1cd91cc-38f3-4b8a-84f1-445884cb80f7\" (UID: \"b1cd91cc-38f3-4b8a-84f1-445884cb80f7\") " Jul 13 07:04:13.331013 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:13.330968 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1cd91cc-38f3-4b8a-84f1-445884cb80f7-service-ca" (OuterVolumeSpecName: "service-ca") pod "b1cd91cc-38f3-4b8a-84f1-445884cb80f7" (UID: "b1cd91cc-38f3-4b8a-84f1-445884cb80f7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 07:04:13.331142 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:13.331005 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1cd91cc-38f3-4b8a-84f1-445884cb80f7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b1cd91cc-38f3-4b8a-84f1-445884cb80f7" (UID: "b1cd91cc-38f3-4b8a-84f1-445884cb80f7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 07:04:13.331142 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:13.331078 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1cd91cc-38f3-4b8a-84f1-445884cb80f7-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "b1cd91cc-38f3-4b8a-84f1-445884cb80f7" (UID: "b1cd91cc-38f3-4b8a-84f1-445884cb80f7"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 07:04:13.331142 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:13.331110 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1cd91cc-38f3-4b8a-84f1-445884cb80f7-console-config" (OuterVolumeSpecName: "console-config") pod "b1cd91cc-38f3-4b8a-84f1-445884cb80f7" (UID: "b1cd91cc-38f3-4b8a-84f1-445884cb80f7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 07:04:13.332839 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:13.332807 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1cd91cc-38f3-4b8a-84f1-445884cb80f7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b1cd91cc-38f3-4b8a-84f1-445884cb80f7" (UID: "b1cd91cc-38f3-4b8a-84f1-445884cb80f7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 07:04:13.332943 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:13.332885 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1cd91cc-38f3-4b8a-84f1-445884cb80f7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b1cd91cc-38f3-4b8a-84f1-445884cb80f7" (UID: "b1cd91cc-38f3-4b8a-84f1-445884cb80f7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 07:04:13.333008 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:13.332981 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1cd91cc-38f3-4b8a-84f1-445884cb80f7-kube-api-access-cqlvt" (OuterVolumeSpecName: "kube-api-access-cqlvt") pod "b1cd91cc-38f3-4b8a-84f1-445884cb80f7" (UID: "b1cd91cc-38f3-4b8a-84f1-445884cb80f7"). InnerVolumeSpecName "kube-api-access-cqlvt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:04:13.431567 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:13.431534 2586 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1cd91cc-38f3-4b8a-84f1-445884cb80f7-trusted-ca-bundle\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:04:13.431567 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:13.431575 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cqlvt\" (UniqueName: \"kubernetes.io/projected/b1cd91cc-38f3-4b8a-84f1-445884cb80f7-kube-api-access-cqlvt\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:04:13.431784 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:13.431588 2586 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b1cd91cc-38f3-4b8a-84f1-445884cb80f7-console-oauth-config\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:04:13.431784 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:13.431623 2586 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b1cd91cc-38f3-4b8a-84f1-445884cb80f7-console-config\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:04:13.431784 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:13.431632 2586 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b1cd91cc-38f3-4b8a-84f1-445884cb80f7-console-serving-cert\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:04:13.431784 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:13.431643 2586 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b1cd91cc-38f3-4b8a-84f1-445884cb80f7-service-ca\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:04:13.431784 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:13.431655 2586 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b1cd91cc-38f3-4b8a-84f1-445884cb80f7-oauth-serving-cert\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:04:13.536616 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:13.536519 2586 generic.go:356] "Generic (PLEG): container finished" podID="b1cd91cc-38f3-4b8a-84f1-445884cb80f7" containerID="c181288d55e99e775aef941af4e1c6009c6ef039f9e193b3b462cdd0d1ebddc1" exitCode=0 Jul 13 07:04:13.536616 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:13.536565 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5766df6b45-b59br" Jul 13 07:04:13.536616 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:13.536586 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5766df6b45-b59br" event={"ID":"b1cd91cc-38f3-4b8a-84f1-445884cb80f7","Type":"ContainerDied","Data":"c181288d55e99e775aef941af4e1c6009c6ef039f9e193b3b462cdd0d1ebddc1"} Jul 13 07:04:13.536858 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:13.536663 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5766df6b45-b59br" event={"ID":"b1cd91cc-38f3-4b8a-84f1-445884cb80f7","Type":"ContainerDied","Data":"f48fdb0cebe14df2ef9f1ee7ec7e36f0688e4e8c392853e0033c513e4b74e56a"} Jul 13 07:04:13.536858 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:13.536713 2586 scope.go:117] "RemoveContainer" containerID="c181288d55e99e775aef941af4e1c6009c6ef039f9e193b3b462cdd0d1ebddc1" Jul 13 07:04:13.546798 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:13.546769 2586 scope.go:117] "RemoveContainer" containerID="c181288d55e99e775aef941af4e1c6009c6ef039f9e193b3b462cdd0d1ebddc1" Jul 13 07:04:13.547119 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:04:13.547097 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c181288d55e99e775aef941af4e1c6009c6ef039f9e193b3b462cdd0d1ebddc1\": container with ID starting with c181288d55e99e775aef941af4e1c6009c6ef039f9e193b3b462cdd0d1ebddc1 not found: ID does not exist" containerID="c181288d55e99e775aef941af4e1c6009c6ef039f9e193b3b462cdd0d1ebddc1" Jul 13 07:04:13.547215 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:13.547135 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c181288d55e99e775aef941af4e1c6009c6ef039f9e193b3b462cdd0d1ebddc1"} err="failed to get container status \"c181288d55e99e775aef941af4e1c6009c6ef039f9e193b3b462cdd0d1ebddc1\": rpc error: code = NotFound desc = could not find container \"c181288d55e99e775aef941af4e1c6009c6ef039f9e193b3b462cdd0d1ebddc1\": container with ID starting with c181288d55e99e775aef941af4e1c6009c6ef039f9e193b3b462cdd0d1ebddc1 not found: ID does not exist" Jul 13 07:04:13.559550 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:13.559517 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5766df6b45-b59br"] Jul 13 07:04:13.565360 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:13.565330 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5766df6b45-b59br"] Jul 13 07:04:13.808317 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:13.807834 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1cd91cc-38f3-4b8a-84f1-445884cb80f7" path="/var/lib/kubelet/pods/b1cd91cc-38f3-4b8a-84f1-445884cb80f7/volumes" Jul 13 07:04:15.547031 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:15.546998 2586 generic.go:356] "Generic (PLEG): container finished" podID="e38c842e-619b-4849-9a60-d65e577ba45e" containerID="b77c5970b5a4da52f404c68b5101ae5b67727dc3e7129d909d29814d8a4f038a" exitCode=0 Jul 13 07:04:15.547403 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:15.547072 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-lhs5q" event={"ID":"e38c842e-619b-4849-9a60-d65e577ba45e","Type":"ContainerDied","Data":"b77c5970b5a4da52f404c68b5101ae5b67727dc3e7129d909d29814d8a4f038a"} Jul 13 07:04:15.547451 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:15.547432 2586 scope.go:117] "RemoveContainer" containerID="b77c5970b5a4da52f404c68b5101ae5b67727dc3e7129d909d29814d8a4f038a" Jul 13 07:04:16.555292 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:16.555261 2586 generic.go:356] "Generic (PLEG): container finished" podID="396e6c4f-4cbc-40ed-b6e8-babbbbc48d39" containerID="a151b944da0e3770d8dd786340c936dff1ac46ce5df4e9e1f4bddff2a50f2d39" exitCode=0 Jul 13 07:04:16.555792 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:16.555333 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-m7g8b" event={"ID":"396e6c4f-4cbc-40ed-b6e8-babbbbc48d39","Type":"ContainerDied","Data":"a151b944da0e3770d8dd786340c936dff1ac46ce5df4e9e1f4bddff2a50f2d39"} Jul 13 07:04:16.555792 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:16.555711 2586 scope.go:117] "RemoveContainer" containerID="a151b944da0e3770d8dd786340c936dff1ac46ce5df4e9e1f4bddff2a50f2d39" Jul 13 07:04:16.557173 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:16.557153 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-lhs5q" event={"ID":"e38c842e-619b-4849-9a60-d65e577ba45e","Type":"ContainerStarted","Data":"89d111fe6b08bad131745fb32a19ebe30e8e16a9dd345f216e8a7b9ceebaeb2c"} Jul 13 07:04:17.563873 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:17.563838 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-m7g8b" event={"ID":"396e6c4f-4cbc-40ed-b6e8-babbbbc48d39","Type":"ContainerStarted","Data":"cbd1fbdd7c16f6e5efb8187c028701e9565ecf3a051fc09932b6fff72a6b11b1"} Jul 13 07:04:49.978544 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:49.978502 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7dbf5bccf7-2tmdw"] Jul 13 07:04:49.979065 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:49.978850 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d" containerName="registry" Jul 13 07:04:49.979065 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:49.978864 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d" containerName="registry" Jul 13 07:04:49.979065 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:49.978887 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1cd91cc-38f3-4b8a-84f1-445884cb80f7" containerName="console" Jul 13 07:04:49.979065 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:49.978896 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1cd91cc-38f3-4b8a-84f1-445884cb80f7" containerName="console" Jul 13 07:04:49.979065 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:49.978916 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="105d89f1-4d9f-4d9a-abe4-5d88a35da28f" containerName="console" Jul 13 07:04:49.979065 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:49.978922 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="105d89f1-4d9f-4d9a-abe4-5d88a35da28f" containerName="console" Jul 13 07:04:49.979065 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:49.978975 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="105d89f1-4d9f-4d9a-abe4-5d88a35da28f" containerName="console" Jul 13 07:04:49.979065 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:49.978985 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="16a4c2df-50d1-4c28-ab6b-8b3a6c923f5d" containerName="registry" Jul 13 07:04:49.979065 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:49.978994 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1cd91cc-38f3-4b8a-84f1-445884cb80f7" containerName="console" Jul 13 07:04:49.980997 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:49.980972 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dbf5bccf7-2tmdw" Jul 13 07:04:49.983424 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:49.983398 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 13 07:04:49.983563 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:49.983529 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 13 07:04:49.983636 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:49.983529 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 13 07:04:49.983636 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:49.983583 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-cxpws\"" Jul 13 07:04:49.983781 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:49.983660 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 13 07:04:49.983781 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:49.983697 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 13 07:04:49.989431 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:49.989403 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 13 07:04:49.993999 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:49.993976 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7dbf5bccf7-2tmdw"] Jul 13 07:04:50.012753 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:50.012718 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhhv4\" (UniqueName: \"kubernetes.io/projected/e121cf45-b562-4c35-a7df-094f60dece96-kube-api-access-hhhv4\") pod \"console-7dbf5bccf7-2tmdw\" (UID: \"e121cf45-b562-4c35-a7df-094f60dece96\") " pod="openshift-console/console-7dbf5bccf7-2tmdw" Jul 13 07:04:50.012753 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:50.012756 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e121cf45-b562-4c35-a7df-094f60dece96-console-config\") pod \"console-7dbf5bccf7-2tmdw\" (UID: \"e121cf45-b562-4c35-a7df-094f60dece96\") " pod="openshift-console/console-7dbf5bccf7-2tmdw" Jul 13 07:04:50.012952 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:50.012786 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e121cf45-b562-4c35-a7df-094f60dece96-service-ca\") pod \"console-7dbf5bccf7-2tmdw\" (UID: \"e121cf45-b562-4c35-a7df-094f60dece96\") " pod="openshift-console/console-7dbf5bccf7-2tmdw" Jul 13 07:04:50.012952 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:50.012822 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e121cf45-b562-4c35-a7df-094f60dece96-console-serving-cert\") pod \"console-7dbf5bccf7-2tmdw\" (UID: \"e121cf45-b562-4c35-a7df-094f60dece96\") " pod="openshift-console/console-7dbf5bccf7-2tmdw" Jul 13 07:04:50.012952 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:50.012859 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e121cf45-b562-4c35-a7df-094f60dece96-trusted-ca-bundle\") pod \"console-7dbf5bccf7-2tmdw\" (UID: \"e121cf45-b562-4c35-a7df-094f60dece96\") " pod="openshift-console/console-7dbf5bccf7-2tmdw" Jul 13 07:04:50.012952 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:50.012912 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e121cf45-b562-4c35-a7df-094f60dece96-console-oauth-config\") pod \"console-7dbf5bccf7-2tmdw\" (UID: \"e121cf45-b562-4c35-a7df-094f60dece96\") " pod="openshift-console/console-7dbf5bccf7-2tmdw" Jul 13 07:04:50.012952 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:50.012950 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e121cf45-b562-4c35-a7df-094f60dece96-oauth-serving-cert\") pod \"console-7dbf5bccf7-2tmdw\" (UID: \"e121cf45-b562-4c35-a7df-094f60dece96\") " pod="openshift-console/console-7dbf5bccf7-2tmdw" Jul 13 07:04:50.114321 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:50.114280 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhhv4\" (UniqueName: \"kubernetes.io/projected/e121cf45-b562-4c35-a7df-094f60dece96-kube-api-access-hhhv4\") pod \"console-7dbf5bccf7-2tmdw\" (UID: \"e121cf45-b562-4c35-a7df-094f60dece96\") " pod="openshift-console/console-7dbf5bccf7-2tmdw" Jul 13 07:04:50.114321 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:50.114324 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e121cf45-b562-4c35-a7df-094f60dece96-console-config\") pod \"console-7dbf5bccf7-2tmdw\" (UID: \"e121cf45-b562-4c35-a7df-094f60dece96\") " pod="openshift-console/console-7dbf5bccf7-2tmdw" Jul 13 07:04:50.114604 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:50.114342 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e121cf45-b562-4c35-a7df-094f60dece96-service-ca\") pod \"console-7dbf5bccf7-2tmdw\" (UID: \"e121cf45-b562-4c35-a7df-094f60dece96\") " pod="openshift-console/console-7dbf5bccf7-2tmdw" Jul 13 07:04:50.114604 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:50.114407 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e121cf45-b562-4c35-a7df-094f60dece96-console-serving-cert\") pod \"console-7dbf5bccf7-2tmdw\" (UID: \"e121cf45-b562-4c35-a7df-094f60dece96\") " pod="openshift-console/console-7dbf5bccf7-2tmdw" Jul 13 07:04:50.114604 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:50.114458 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e121cf45-b562-4c35-a7df-094f60dece96-trusted-ca-bundle\") pod \"console-7dbf5bccf7-2tmdw\" (UID: \"e121cf45-b562-4c35-a7df-094f60dece96\") " pod="openshift-console/console-7dbf5bccf7-2tmdw" Jul 13 07:04:50.114604 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:50.114538 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e121cf45-b562-4c35-a7df-094f60dece96-console-oauth-config\") pod \"console-7dbf5bccf7-2tmdw\" (UID: \"e121cf45-b562-4c35-a7df-094f60dece96\") " pod="openshift-console/console-7dbf5bccf7-2tmdw" Jul 13 07:04:50.114604 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:50.114576 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e121cf45-b562-4c35-a7df-094f60dece96-oauth-serving-cert\") pod \"console-7dbf5bccf7-2tmdw\" (UID: \"e121cf45-b562-4c35-a7df-094f60dece96\") " pod="openshift-console/console-7dbf5bccf7-2tmdw" Jul 13 07:04:50.115224 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:50.115198 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e121cf45-b562-4c35-a7df-094f60dece96-console-config\") pod \"console-7dbf5bccf7-2tmdw\" (UID: \"e121cf45-b562-4c35-a7df-094f60dece96\") " pod="openshift-console/console-7dbf5bccf7-2tmdw" Jul 13 07:04:50.115318 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:50.115292 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e121cf45-b562-4c35-a7df-094f60dece96-service-ca\") pod \"console-7dbf5bccf7-2tmdw\" (UID: \"e121cf45-b562-4c35-a7df-094f60dece96\") " pod="openshift-console/console-7dbf5bccf7-2tmdw" Jul 13 07:04:50.115370 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:50.115296 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e121cf45-b562-4c35-a7df-094f60dece96-trusted-ca-bundle\") pod \"console-7dbf5bccf7-2tmdw\" (UID: \"e121cf45-b562-4c35-a7df-094f60dece96\") " pod="openshift-console/console-7dbf5bccf7-2tmdw" Jul 13 07:04:50.115370 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:50.115299 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e121cf45-b562-4c35-a7df-094f60dece96-oauth-serving-cert\") pod \"console-7dbf5bccf7-2tmdw\" (UID: \"e121cf45-b562-4c35-a7df-094f60dece96\") " pod="openshift-console/console-7dbf5bccf7-2tmdw" Jul 13 07:04:50.116909 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:50.116887 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e121cf45-b562-4c35-a7df-094f60dece96-console-oauth-config\") pod \"console-7dbf5bccf7-2tmdw\" (UID: \"e121cf45-b562-4c35-a7df-094f60dece96\") " pod="openshift-console/console-7dbf5bccf7-2tmdw" Jul 13 07:04:50.117039 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:50.117020 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e121cf45-b562-4c35-a7df-094f60dece96-console-serving-cert\") pod \"console-7dbf5bccf7-2tmdw\" (UID: \"e121cf45-b562-4c35-a7df-094f60dece96\") " pod="openshift-console/console-7dbf5bccf7-2tmdw" Jul 13 07:04:50.122407 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:50.122384 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhhv4\" (UniqueName: \"kubernetes.io/projected/e121cf45-b562-4c35-a7df-094f60dece96-kube-api-access-hhhv4\") pod \"console-7dbf5bccf7-2tmdw\" (UID: \"e121cf45-b562-4c35-a7df-094f60dece96\") " pod="openshift-console/console-7dbf5bccf7-2tmdw" Jul 13 07:04:50.292283 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:50.292179 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dbf5bccf7-2tmdw" Jul 13 07:04:50.436515 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:50.436472 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7dbf5bccf7-2tmdw"] Jul 13 07:04:50.440119 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:04:50.440088 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode121cf45_b562_4c35_a7df_094f60dece96.slice/crio-2121ae0c0825898b6c9d1a6d3c8759d6e5754a4df1dece671a965fb9f6f87917 WatchSource:0}: Error finding container 2121ae0c0825898b6c9d1a6d3c8759d6e5754a4df1dece671a965fb9f6f87917: Status 404 returned error can't find the container with id 2121ae0c0825898b6c9d1a6d3c8759d6e5754a4df1dece671a965fb9f6f87917 Jul 13 07:04:50.689010 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:50.688971 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dbf5bccf7-2tmdw" event={"ID":"e121cf45-b562-4c35-a7df-094f60dece96","Type":"ContainerStarted","Data":"cafce630f25a0f7fabe45a4967b11c68ea4430ea368ba5f03af8e19f70f020b7"} Jul 13 07:04:50.689173 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:50.689027 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dbf5bccf7-2tmdw" event={"ID":"e121cf45-b562-4c35-a7df-094f60dece96","Type":"ContainerStarted","Data":"2121ae0c0825898b6c9d1a6d3c8759d6e5754a4df1dece671a965fb9f6f87917"} Jul 13 07:04:50.707423 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:04:50.707319 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7dbf5bccf7-2tmdw" podStartSLOduration=1.70729941 podStartE2EDuration="1.70729941s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:04:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:04:50.707113747 +0000 UTC m=+165.542677588" watchObservedRunningTime="2026-07-13 07:04:50.70729941 +0000 UTC m=+165.542863241" Jul 13 07:05:00.293246 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:05:00.293157 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7dbf5bccf7-2tmdw" Jul 13 07:05:00.293607 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:05:00.293250 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7dbf5bccf7-2tmdw" Jul 13 07:05:00.298044 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:05:00.298016 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7dbf5bccf7-2tmdw" Jul 13 07:05:00.731075 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:05:00.731045 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7dbf5bccf7-2tmdw" Jul 13 07:06:02.461955 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:02.461911 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754072nr8"] Jul 13 07:06:02.464048 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:02.464026 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754072nr8" Jul 13 07:06:02.465984 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:02.465951 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 13 07:06:02.466111 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:02.466048 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 13 07:06:02.466534 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:02.466518 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-j6sdv\"" Jul 13 07:06:02.473430 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:02.473403 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754072nr8"] Jul 13 07:06:02.580246 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:02.580193 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z47b6\" (UniqueName: \"kubernetes.io/projected/672819cf-8e11-43e0-942c-9d36b0eae631-kube-api-access-z47b6\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754072nr8\" (UID: \"672819cf-8e11-43e0-942c-9d36b0eae631\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754072nr8" Jul 13 07:06:02.580440 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:02.580261 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/672819cf-8e11-43e0-942c-9d36b0eae631-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754072nr8\" (UID: \"672819cf-8e11-43e0-942c-9d36b0eae631\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754072nr8" Jul 13 07:06:02.580440 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:02.580311 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/672819cf-8e11-43e0-942c-9d36b0eae631-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754072nr8\" (UID: \"672819cf-8e11-43e0-942c-9d36b0eae631\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754072nr8" Jul 13 07:06:02.680945 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:02.680904 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/672819cf-8e11-43e0-942c-9d36b0eae631-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754072nr8\" (UID: \"672819cf-8e11-43e0-942c-9d36b0eae631\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754072nr8" Jul 13 07:06:02.681096 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:02.680956 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/672819cf-8e11-43e0-942c-9d36b0eae631-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754072nr8\" (UID: \"672819cf-8e11-43e0-942c-9d36b0eae631\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754072nr8" Jul 13 07:06:02.681096 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:02.680996 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z47b6\" (UniqueName: \"kubernetes.io/projected/672819cf-8e11-43e0-942c-9d36b0eae631-kube-api-access-z47b6\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754072nr8\" (UID: \"672819cf-8e11-43e0-942c-9d36b0eae631\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754072nr8" Jul 13 07:06:02.681332 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:02.681309 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/672819cf-8e11-43e0-942c-9d36b0eae631-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754072nr8\" (UID: \"672819cf-8e11-43e0-942c-9d36b0eae631\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754072nr8" Jul 13 07:06:02.681374 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:02.681338 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/672819cf-8e11-43e0-942c-9d36b0eae631-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754072nr8\" (UID: \"672819cf-8e11-43e0-942c-9d36b0eae631\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754072nr8" Jul 13 07:06:02.688878 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:02.688841 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z47b6\" (UniqueName: \"kubernetes.io/projected/672819cf-8e11-43e0-942c-9d36b0eae631-kube-api-access-z47b6\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754072nr8\" (UID: \"672819cf-8e11-43e0-942c-9d36b0eae631\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754072nr8" Jul 13 07:06:02.773749 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:02.773643 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754072nr8" Jul 13 07:06:02.906852 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:02.906811 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754072nr8"] Jul 13 07:06:02.909274 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:06:02.909250 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod672819cf_8e11_43e0_942c_9d36b0eae631.slice/crio-78e75c1b4e3702660398f8688f540cf71a2c7ed4e87ea6f8cca807a142c03e9e WatchSource:0}: Error finding container 78e75c1b4e3702660398f8688f540cf71a2c7ed4e87ea6f8cca807a142c03e9e: Status 404 returned error can't find the container with id 78e75c1b4e3702660398f8688f540cf71a2c7ed4e87ea6f8cca807a142c03e9e Jul 13 07:06:02.956748 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:02.956706 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754072nr8" event={"ID":"672819cf-8e11-43e0-942c-9d36b0eae631","Type":"ContainerStarted","Data":"78e75c1b4e3702660398f8688f540cf71a2c7ed4e87ea6f8cca807a142c03e9e"} Jul 13 07:06:10.992625 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:10.992582 2586 generic.go:356] "Generic (PLEG): container finished" podID="672819cf-8e11-43e0-942c-9d36b0eae631" containerID="d314939e61fb0a6a2a9909ad425a6a023d60c32e096b6cc5c9eb73aa0f5604d0" exitCode=0 Jul 13 07:06:10.992625 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:10.992629 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754072nr8" event={"ID":"672819cf-8e11-43e0-942c-9d36b0eae631","Type":"ContainerDied","Data":"d314939e61fb0a6a2a9909ad425a6a023d60c32e096b6cc5c9eb73aa0f5604d0"} Jul 13 07:06:14.007401 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:14.007359 2586 generic.go:356] "Generic (PLEG): container finished" podID="672819cf-8e11-43e0-942c-9d36b0eae631" containerID="47fc84d87d7c721691a40aaaf0356c3c31b0d5dea6296caad57cbe451a1805b3" exitCode=0 Jul 13 07:06:14.007931 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:14.007419 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754072nr8" event={"ID":"672819cf-8e11-43e0-942c-9d36b0eae631","Type":"ContainerDied","Data":"47fc84d87d7c721691a40aaaf0356c3c31b0d5dea6296caad57cbe451a1805b3"} Jul 13 07:06:23.048093 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:23.047985 2586 generic.go:356] "Generic (PLEG): container finished" podID="672819cf-8e11-43e0-942c-9d36b0eae631" containerID="32294dc40aa420534a8a66a86702c2e07d6ed57cc771573e633fe00a3be4db72" exitCode=0 Jul 13 07:06:23.048093 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:23.048034 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754072nr8" event={"ID":"672819cf-8e11-43e0-942c-9d36b0eae631","Type":"ContainerDied","Data":"32294dc40aa420534a8a66a86702c2e07d6ed57cc771573e633fe00a3be4db72"} Jul 13 07:06:24.185041 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:24.185009 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754072nr8" Jul 13 07:06:24.262059 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:24.262023 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z47b6\" (UniqueName: \"kubernetes.io/projected/672819cf-8e11-43e0-942c-9d36b0eae631-kube-api-access-z47b6\") pod \"672819cf-8e11-43e0-942c-9d36b0eae631\" (UID: \"672819cf-8e11-43e0-942c-9d36b0eae631\") " Jul 13 07:06:24.262240 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:24.262108 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/672819cf-8e11-43e0-942c-9d36b0eae631-util\") pod \"672819cf-8e11-43e0-942c-9d36b0eae631\" (UID: \"672819cf-8e11-43e0-942c-9d36b0eae631\") " Jul 13 07:06:24.262240 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:24.262146 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/672819cf-8e11-43e0-942c-9d36b0eae631-bundle\") pod \"672819cf-8e11-43e0-942c-9d36b0eae631\" (UID: \"672819cf-8e11-43e0-942c-9d36b0eae631\") " Jul 13 07:06:24.262873 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:24.262845 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/672819cf-8e11-43e0-942c-9d36b0eae631-bundle" (OuterVolumeSpecName: "bundle") pod "672819cf-8e11-43e0-942c-9d36b0eae631" (UID: "672819cf-8e11-43e0-942c-9d36b0eae631"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:06:24.264404 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:24.264372 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/672819cf-8e11-43e0-942c-9d36b0eae631-kube-api-access-z47b6" (OuterVolumeSpecName: "kube-api-access-z47b6") pod "672819cf-8e11-43e0-942c-9d36b0eae631" (UID: "672819cf-8e11-43e0-942c-9d36b0eae631"). InnerVolumeSpecName "kube-api-access-z47b6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:06:24.266701 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:24.266665 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/672819cf-8e11-43e0-942c-9d36b0eae631-util" (OuterVolumeSpecName: "util") pod "672819cf-8e11-43e0-942c-9d36b0eae631" (UID: "672819cf-8e11-43e0-942c-9d36b0eae631"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:06:24.362987 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:24.362907 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z47b6\" (UniqueName: \"kubernetes.io/projected/672819cf-8e11-43e0-942c-9d36b0eae631-kube-api-access-z47b6\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:06:24.362987 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:24.362940 2586 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/672819cf-8e11-43e0-942c-9d36b0eae631-util\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:06:24.362987 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:24.362949 2586 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/672819cf-8e11-43e0-942c-9d36b0eae631-bundle\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:06:25.058022 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:25.057979 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754072nr8" event={"ID":"672819cf-8e11-43e0-942c-9d36b0eae631","Type":"ContainerDied","Data":"78e75c1b4e3702660398f8688f540cf71a2c7ed4e87ea6f8cca807a142c03e9e"} Jul 13 07:06:25.058234 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:25.058049 2586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="78e75c1b4e3702660398f8688f540cf71a2c7ed4e87ea6f8cca807a142c03e9e" Jul 13 07:06:25.058234 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:25.057991 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754072nr8" Jul 13 07:06:30.490117 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:30.490035 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mtqsq"] Jul 13 07:06:30.490547 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:30.490352 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="672819cf-8e11-43e0-942c-9d36b0eae631" containerName="util" Jul 13 07:06:30.490547 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:30.490365 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="672819cf-8e11-43e0-942c-9d36b0eae631" containerName="util" Jul 13 07:06:30.490547 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:30.490376 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="672819cf-8e11-43e0-942c-9d36b0eae631" containerName="pull" Jul 13 07:06:30.490547 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:30.490382 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="672819cf-8e11-43e0-942c-9d36b0eae631" containerName="pull" Jul 13 07:06:30.490547 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:30.490393 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="672819cf-8e11-43e0-942c-9d36b0eae631" containerName="extract" Jul 13 07:06:30.490547 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:30.490398 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="672819cf-8e11-43e0-942c-9d36b0eae631" containerName="extract" Jul 13 07:06:30.490547 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:30.490463 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="672819cf-8e11-43e0-942c-9d36b0eae631" containerName="extract" Jul 13 07:06:30.493568 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:30.493549 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mtqsq" Jul 13 07:06:30.495505 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:30.495478 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 13 07:06:30.495650 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:30.495577 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 13 07:06:30.495798 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:30.495785 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-7vbg2\"" Jul 13 07:06:30.507607 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:30.507577 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mtqsq"] Jul 13 07:06:30.614785 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:30.614744 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/05f215d5-6a1b-4b7a-a070-017b1afce436-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-mtqsq\" (UID: \"05f215d5-6a1b-4b7a-a070-017b1afce436\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mtqsq" Jul 13 07:06:30.614963 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:30.614837 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8x7mv\" (UniqueName: \"kubernetes.io/projected/05f215d5-6a1b-4b7a-a070-017b1afce436-kube-api-access-8x7mv\") pod \"cert-manager-operator-controller-manager-55c444bf86-mtqsq\" (UID: \"05f215d5-6a1b-4b7a-a070-017b1afce436\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mtqsq" Jul 13 07:06:30.715845 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:30.715809 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8x7mv\" (UniqueName: \"kubernetes.io/projected/05f215d5-6a1b-4b7a-a070-017b1afce436-kube-api-access-8x7mv\") pod \"cert-manager-operator-controller-manager-55c444bf86-mtqsq\" (UID: \"05f215d5-6a1b-4b7a-a070-017b1afce436\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mtqsq" Jul 13 07:06:30.716025 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:30.715863 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/05f215d5-6a1b-4b7a-a070-017b1afce436-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-mtqsq\" (UID: \"05f215d5-6a1b-4b7a-a070-017b1afce436\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mtqsq" Jul 13 07:06:30.716224 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:30.716207 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/05f215d5-6a1b-4b7a-a070-017b1afce436-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-mtqsq\" (UID: \"05f215d5-6a1b-4b7a-a070-017b1afce436\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mtqsq" Jul 13 07:06:30.723946 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:30.723913 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8x7mv\" (UniqueName: \"kubernetes.io/projected/05f215d5-6a1b-4b7a-a070-017b1afce436-kube-api-access-8x7mv\") pod \"cert-manager-operator-controller-manager-55c444bf86-mtqsq\" (UID: \"05f215d5-6a1b-4b7a-a070-017b1afce436\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mtqsq" Jul 13 07:06:30.803002 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:30.802899 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mtqsq" Jul 13 07:06:30.947046 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:30.947012 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mtqsq"] Jul 13 07:06:30.951886 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:06:30.951844 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05f215d5_6a1b_4b7a_a070_017b1afce436.slice/crio-19305811682bd6485cbb533e1b1fae8117d1dc8f8fa0a7b74c880892fff9fe9c WatchSource:0}: Error finding container 19305811682bd6485cbb533e1b1fae8117d1dc8f8fa0a7b74c880892fff9fe9c: Status 404 returned error can't find the container with id 19305811682bd6485cbb533e1b1fae8117d1dc8f8fa0a7b74c880892fff9fe9c Jul 13 07:06:31.083539 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:31.083446 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mtqsq" event={"ID":"05f215d5-6a1b-4b7a-a070-017b1afce436","Type":"ContainerStarted","Data":"19305811682bd6485cbb533e1b1fae8117d1dc8f8fa0a7b74c880892fff9fe9c"} Jul 13 07:06:33.094316 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:33.094279 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mtqsq" event={"ID":"05f215d5-6a1b-4b7a-a070-017b1afce436","Type":"ContainerStarted","Data":"f1bc5e0f4953f03749797a7543be99461c5deeb9fbb57e8116a1df6970580d48"} Jul 13 07:06:33.113976 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:33.113922 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mtqsq" podStartSLOduration=1.455275138 podStartE2EDuration="3.113905071s" totalImagesPullingTime="1.658629933s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:06:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:06:33.112623745 +0000 UTC m=+267.948187575" watchObservedRunningTime="2026-07-13 07:06:33.113905071 +0000 UTC m=+267.949468902" Jul 13 07:06:34.131665 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:34.131622 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4zk7h"] Jul 13 07:06:34.135062 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:34.135042 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4zk7h" Jul 13 07:06:34.137359 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:34.137329 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-j6sdv\"" Jul 13 07:06:34.137359 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:34.137345 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 13 07:06:34.137513 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:34.137427 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 13 07:06:34.144371 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:34.144343 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4zk7h"] Jul 13 07:06:34.245350 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:34.245308 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/54d98aeb-c658-4281-9e3f-6bfaad92b940-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4zk7h\" (UID: \"54d98aeb-c658-4281-9e3f-6bfaad92b940\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4zk7h" Jul 13 07:06:34.245527 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:34.245390 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/54d98aeb-c658-4281-9e3f-6bfaad92b940-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4zk7h\" (UID: \"54d98aeb-c658-4281-9e3f-6bfaad92b940\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4zk7h" Jul 13 07:06:34.245527 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:34.245469 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jc2f8\" (UniqueName: \"kubernetes.io/projected/54d98aeb-c658-4281-9e3f-6bfaad92b940-kube-api-access-jc2f8\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4zk7h\" (UID: \"54d98aeb-c658-4281-9e3f-6bfaad92b940\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4zk7h" Jul 13 07:06:34.346260 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:34.346220 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jc2f8\" (UniqueName: \"kubernetes.io/projected/54d98aeb-c658-4281-9e3f-6bfaad92b940-kube-api-access-jc2f8\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4zk7h\" (UID: \"54d98aeb-c658-4281-9e3f-6bfaad92b940\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4zk7h" Jul 13 07:06:34.346435 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:34.346291 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/54d98aeb-c658-4281-9e3f-6bfaad92b940-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4zk7h\" (UID: \"54d98aeb-c658-4281-9e3f-6bfaad92b940\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4zk7h" Jul 13 07:06:34.346435 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:34.346321 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/54d98aeb-c658-4281-9e3f-6bfaad92b940-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4zk7h\" (UID: \"54d98aeb-c658-4281-9e3f-6bfaad92b940\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4zk7h" Jul 13 07:06:34.346743 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:34.346723 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/54d98aeb-c658-4281-9e3f-6bfaad92b940-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4zk7h\" (UID: \"54d98aeb-c658-4281-9e3f-6bfaad92b940\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4zk7h" Jul 13 07:06:34.346851 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:34.346769 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/54d98aeb-c658-4281-9e3f-6bfaad92b940-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4zk7h\" (UID: \"54d98aeb-c658-4281-9e3f-6bfaad92b940\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4zk7h" Jul 13 07:06:34.353872 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:34.353852 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jc2f8\" (UniqueName: \"kubernetes.io/projected/54d98aeb-c658-4281-9e3f-6bfaad92b940-kube-api-access-jc2f8\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4zk7h\" (UID: \"54d98aeb-c658-4281-9e3f-6bfaad92b940\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4zk7h" Jul 13 07:06:34.444862 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:34.444829 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4zk7h" Jul 13 07:06:34.587173 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:34.587147 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4zk7h"] Jul 13 07:06:34.589247 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:06:34.589214 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54d98aeb_c658_4281_9e3f_6bfaad92b940.slice/crio-ed16dbf175aebca543181073ae5c0a056dbfaee60aa7fe439935bed13868a3e1 WatchSource:0}: Error finding container ed16dbf175aebca543181073ae5c0a056dbfaee60aa7fe439935bed13868a3e1: Status 404 returned error can't find the container with id ed16dbf175aebca543181073ae5c0a056dbfaee60aa7fe439935bed13868a3e1 Jul 13 07:06:35.104465 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:35.104381 2586 generic.go:356] "Generic (PLEG): container finished" podID="54d98aeb-c658-4281-9e3f-6bfaad92b940" containerID="76a1ec82e6d71a6ff7318410d014e6300e9b2f7ea2c8c83e5aefa2f3bb7b0c69" exitCode=0 Jul 13 07:06:35.104636 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:35.104468 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4zk7h" event={"ID":"54d98aeb-c658-4281-9e3f-6bfaad92b940","Type":"ContainerDied","Data":"76a1ec82e6d71a6ff7318410d014e6300e9b2f7ea2c8c83e5aefa2f3bb7b0c69"} Jul 13 07:06:35.104636 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:35.104524 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4zk7h" event={"ID":"54d98aeb-c658-4281-9e3f-6bfaad92b940","Type":"ContainerStarted","Data":"ed16dbf175aebca543181073ae5c0a056dbfaee60aa7fe439935bed13868a3e1"} Jul 13 07:06:36.891547 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:36.891503 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-xlcxx"] Jul 13 07:06:36.894973 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:36.894946 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-xlcxx" Jul 13 07:06:36.897085 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:36.897063 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 13 07:06:36.897183 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:36.897090 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 13 07:06:36.897814 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:36.897790 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-9sctt\"" Jul 13 07:06:36.905046 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:36.905025 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-xlcxx"] Jul 13 07:06:36.968440 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:36.968419 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4v7r\" (UniqueName: \"kubernetes.io/projected/f61dbded-991c-4a0f-867e-1bf1ad8257e1-kube-api-access-t4v7r\") pod \"cert-manager-cainjector-559cdd8c87-xlcxx\" (UID: \"f61dbded-991c-4a0f-867e-1bf1ad8257e1\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xlcxx" Jul 13 07:06:36.968524 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:36.968476 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f61dbded-991c-4a0f-867e-1bf1ad8257e1-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-xlcxx\" (UID: \"f61dbded-991c-4a0f-867e-1bf1ad8257e1\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xlcxx" Jul 13 07:06:37.069139 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:37.069095 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4v7r\" (UniqueName: \"kubernetes.io/projected/f61dbded-991c-4a0f-867e-1bf1ad8257e1-kube-api-access-t4v7r\") pod \"cert-manager-cainjector-559cdd8c87-xlcxx\" (UID: \"f61dbded-991c-4a0f-867e-1bf1ad8257e1\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xlcxx" Jul 13 07:06:37.069305 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:37.069150 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f61dbded-991c-4a0f-867e-1bf1ad8257e1-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-xlcxx\" (UID: \"f61dbded-991c-4a0f-867e-1bf1ad8257e1\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xlcxx" Jul 13 07:06:37.082459 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:37.082429 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f61dbded-991c-4a0f-867e-1bf1ad8257e1-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-xlcxx\" (UID: \"f61dbded-991c-4a0f-867e-1bf1ad8257e1\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xlcxx" Jul 13 07:06:37.082584 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:37.082516 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4v7r\" (UniqueName: \"kubernetes.io/projected/f61dbded-991c-4a0f-867e-1bf1ad8257e1-kube-api-access-t4v7r\") pod \"cert-manager-cainjector-559cdd8c87-xlcxx\" (UID: \"f61dbded-991c-4a0f-867e-1bf1ad8257e1\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xlcxx" Jul 13 07:06:37.115063 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:37.115035 2586 generic.go:356] "Generic (PLEG): container finished" podID="54d98aeb-c658-4281-9e3f-6bfaad92b940" containerID="c291b7b7da415e0b01279a8f00bedb127ab4c557986ed14f56e8417c50c4ed8b" exitCode=0 Jul 13 07:06:37.115183 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:37.115121 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4zk7h" event={"ID":"54d98aeb-c658-4281-9e3f-6bfaad92b940","Type":"ContainerDied","Data":"c291b7b7da415e0b01279a8f00bedb127ab4c557986ed14f56e8417c50c4ed8b"} Jul 13 07:06:37.213965 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:37.213931 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-xlcxx" Jul 13 07:06:37.342783 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:37.342633 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-xlcxx"] Jul 13 07:06:37.345169 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:06:37.345142 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf61dbded_991c_4a0f_867e_1bf1ad8257e1.slice/crio-894e02f5d669d3c74a755a78e80a4f4a0fc66cda87fc24bc71584695b86101f8 WatchSource:0}: Error finding container 894e02f5d669d3c74a755a78e80a4f4a0fc66cda87fc24bc71584695b86101f8: Status 404 returned error can't find the container with id 894e02f5d669d3c74a755a78e80a4f4a0fc66cda87fc24bc71584695b86101f8 Jul 13 07:06:38.122198 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:38.122162 2586 generic.go:356] "Generic (PLEG): container finished" podID="54d98aeb-c658-4281-9e3f-6bfaad92b940" containerID="d91c800c96dd53beae0669b81d74ac6549b4c931d3a8580490ea6ab11cf9f786" exitCode=0 Jul 13 07:06:38.122652 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:38.122296 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4zk7h" event={"ID":"54d98aeb-c658-4281-9e3f-6bfaad92b940","Type":"ContainerDied","Data":"d91c800c96dd53beae0669b81d74ac6549b4c931d3a8580490ea6ab11cf9f786"} Jul 13 07:06:38.123580 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:38.123553 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-xlcxx" event={"ID":"f61dbded-991c-4a0f-867e-1bf1ad8257e1","Type":"ContainerStarted","Data":"894e02f5d669d3c74a755a78e80a4f4a0fc66cda87fc24bc71584695b86101f8"} Jul 13 07:06:39.949404 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:39.949373 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4zk7h" Jul 13 07:06:40.097493 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:40.097449 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jc2f8\" (UniqueName: \"kubernetes.io/projected/54d98aeb-c658-4281-9e3f-6bfaad92b940-kube-api-access-jc2f8\") pod \"54d98aeb-c658-4281-9e3f-6bfaad92b940\" (UID: \"54d98aeb-c658-4281-9e3f-6bfaad92b940\") " Jul 13 07:06:40.097734 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:40.097511 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/54d98aeb-c658-4281-9e3f-6bfaad92b940-bundle\") pod \"54d98aeb-c658-4281-9e3f-6bfaad92b940\" (UID: \"54d98aeb-c658-4281-9e3f-6bfaad92b940\") " Jul 13 07:06:40.097734 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:40.097557 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/54d98aeb-c658-4281-9e3f-6bfaad92b940-util\") pod \"54d98aeb-c658-4281-9e3f-6bfaad92b940\" (UID: \"54d98aeb-c658-4281-9e3f-6bfaad92b940\") " Jul 13 07:06:40.097970 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:40.097940 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54d98aeb-c658-4281-9e3f-6bfaad92b940-bundle" (OuterVolumeSpecName: "bundle") pod "54d98aeb-c658-4281-9e3f-6bfaad92b940" (UID: "54d98aeb-c658-4281-9e3f-6bfaad92b940"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:06:40.099651 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:40.099625 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54d98aeb-c658-4281-9e3f-6bfaad92b940-kube-api-access-jc2f8" (OuterVolumeSpecName: "kube-api-access-jc2f8") pod "54d98aeb-c658-4281-9e3f-6bfaad92b940" (UID: "54d98aeb-c658-4281-9e3f-6bfaad92b940"). InnerVolumeSpecName "kube-api-access-jc2f8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:06:40.102589 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:40.102554 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54d98aeb-c658-4281-9e3f-6bfaad92b940-util" (OuterVolumeSpecName: "util") pod "54d98aeb-c658-4281-9e3f-6bfaad92b940" (UID: "54d98aeb-c658-4281-9e3f-6bfaad92b940"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:06:40.136162 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:40.136126 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-xlcxx" event={"ID":"f61dbded-991c-4a0f-867e-1bf1ad8257e1","Type":"ContainerStarted","Data":"277881ca370991e3b0d33f702f4519f18fca291f86665446eee1dda5e3665c60"} Jul 13 07:06:40.137774 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:40.137747 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4zk7h" event={"ID":"54d98aeb-c658-4281-9e3f-6bfaad92b940","Type":"ContainerDied","Data":"ed16dbf175aebca543181073ae5c0a056dbfaee60aa7fe439935bed13868a3e1"} Jul 13 07:06:40.137904 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:40.137783 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4zk7h" Jul 13 07:06:40.137904 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:40.137793 2586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ed16dbf175aebca543181073ae5c0a056dbfaee60aa7fe439935bed13868a3e1" Jul 13 07:06:40.151334 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:40.151281 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-xlcxx" podStartSLOduration=1.49944971 podStartE2EDuration="4.151265085s" totalImagesPullingTime="2.651815375s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:06:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:06:40.149258301 +0000 UTC m=+274.984822143" watchObservedRunningTime="2026-07-13 07:06:40.151265085 +0000 UTC m=+274.986828916" Jul 13 07:06:40.198800 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:40.198708 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jc2f8\" (UniqueName: \"kubernetes.io/projected/54d98aeb-c658-4281-9e3f-6bfaad92b940-kube-api-access-jc2f8\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:06:40.198800 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:40.198742 2586 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/54d98aeb-c658-4281-9e3f-6bfaad92b940-bundle\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:06:40.198800 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:40.198753 2586 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/54d98aeb-c658-4281-9e3f-6bfaad92b940-util\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:06:47.276401 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:47.276362 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-rs2ss"] Jul 13 07:06:47.276806 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:47.276646 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54d98aeb-c658-4281-9e3f-6bfaad92b940" containerName="pull" Jul 13 07:06:47.276806 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:47.276657 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="54d98aeb-c658-4281-9e3f-6bfaad92b940" containerName="pull" Jul 13 07:06:47.276806 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:47.276668 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54d98aeb-c658-4281-9e3f-6bfaad92b940" containerName="util" Jul 13 07:06:47.276806 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:47.276673 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="54d98aeb-c658-4281-9e3f-6bfaad92b940" containerName="util" Jul 13 07:06:47.276806 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:47.276700 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54d98aeb-c658-4281-9e3f-6bfaad92b940" containerName="extract" Jul 13 07:06:47.276806 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:47.276706 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="54d98aeb-c658-4281-9e3f-6bfaad92b940" containerName="extract" Jul 13 07:06:47.276806 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:47.276765 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="54d98aeb-c658-4281-9e3f-6bfaad92b940" containerName="extract" Jul 13 07:06:47.282234 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:47.282217 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-rs2ss" Jul 13 07:06:47.284548 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:47.284524 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-wljr9\"" Jul 13 07:06:47.289429 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:47.289407 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-rs2ss"] Jul 13 07:06:47.457949 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:47.457914 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftzbf\" (UniqueName: \"kubernetes.io/projected/413b7419-16ea-4287-9ec6-8f047502a13f-kube-api-access-ftzbf\") pod \"cert-manager-55b4d954d5-rs2ss\" (UID: \"413b7419-16ea-4287-9ec6-8f047502a13f\") " pod="cert-manager/cert-manager-55b4d954d5-rs2ss" Jul 13 07:06:47.457949 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:47.457950 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/413b7419-16ea-4287-9ec6-8f047502a13f-bound-sa-token\") pod \"cert-manager-55b4d954d5-rs2ss\" (UID: \"413b7419-16ea-4287-9ec6-8f047502a13f\") " pod="cert-manager/cert-manager-55b4d954d5-rs2ss" Jul 13 07:06:47.558572 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:47.558491 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftzbf\" (UniqueName: \"kubernetes.io/projected/413b7419-16ea-4287-9ec6-8f047502a13f-kube-api-access-ftzbf\") pod \"cert-manager-55b4d954d5-rs2ss\" (UID: \"413b7419-16ea-4287-9ec6-8f047502a13f\") " pod="cert-manager/cert-manager-55b4d954d5-rs2ss" Jul 13 07:06:47.558572 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:47.558524 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/413b7419-16ea-4287-9ec6-8f047502a13f-bound-sa-token\") pod \"cert-manager-55b4d954d5-rs2ss\" (UID: \"413b7419-16ea-4287-9ec6-8f047502a13f\") " pod="cert-manager/cert-manager-55b4d954d5-rs2ss" Jul 13 07:06:47.569449 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:47.569417 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/413b7419-16ea-4287-9ec6-8f047502a13f-bound-sa-token\") pod \"cert-manager-55b4d954d5-rs2ss\" (UID: \"413b7419-16ea-4287-9ec6-8f047502a13f\") " pod="cert-manager/cert-manager-55b4d954d5-rs2ss" Jul 13 07:06:47.569604 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:47.569585 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftzbf\" (UniqueName: \"kubernetes.io/projected/413b7419-16ea-4287-9ec6-8f047502a13f-kube-api-access-ftzbf\") pod \"cert-manager-55b4d954d5-rs2ss\" (UID: \"413b7419-16ea-4287-9ec6-8f047502a13f\") " pod="cert-manager/cert-manager-55b4d954d5-rs2ss" Jul 13 07:06:47.591729 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:47.591674 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-rs2ss" Jul 13 07:06:47.721414 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:47.721386 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-rs2ss"] Jul 13 07:06:47.723753 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:06:47.723728 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod413b7419_16ea_4287_9ec6_8f047502a13f.slice/crio-cb8c01795571aab2e885439cf7ea60b40023ba03e2eb6e8161dafd653da4995a WatchSource:0}: Error finding container cb8c01795571aab2e885439cf7ea60b40023ba03e2eb6e8161dafd653da4995a: Status 404 returned error can't find the container with id cb8c01795571aab2e885439cf7ea60b40023ba03e2eb6e8161dafd653da4995a Jul 13 07:06:48.182808 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:48.182765 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-rs2ss" event={"ID":"413b7419-16ea-4287-9ec6-8f047502a13f","Type":"ContainerStarted","Data":"625b063b6da53cb08a734743fcd4da156a9793d2108e01d836526cf3fa4e32d5"} Jul 13 07:06:48.182808 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:48.182814 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-rs2ss" event={"ID":"413b7419-16ea-4287-9ec6-8f047502a13f","Type":"ContainerStarted","Data":"cb8c01795571aab2e885439cf7ea60b40023ba03e2eb6e8161dafd653da4995a"} Jul 13 07:06:48.198640 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:48.198586 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-rs2ss" podStartSLOduration=1.198569079 podStartE2EDuration="1.198569079s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:06:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:06:48.19789978 +0000 UTC m=+283.033463611" watchObservedRunningTime="2026-07-13 07:06:48.198569079 +0000 UTC m=+283.034132911" Jul 13 07:06:54.983574 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:54.983536 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz29fs"] Jul 13 07:06:54.988027 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:54.988010 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz29fs" Jul 13 07:06:54.990024 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:54.990004 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 13 07:06:54.990124 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:54.990024 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 13 07:06:54.990529 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:54.990514 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-j6sdv\"" Jul 13 07:06:54.996952 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:54.996930 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz29fs"] Jul 13 07:06:55.124065 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:55.124029 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/038b5d04-56fa-4d2a-9bff-f582dfb67243-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz29fs\" (UID: \"038b5d04-56fa-4d2a-9bff-f582dfb67243\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz29fs" Jul 13 07:06:55.124267 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:55.124087 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgttk\" (UniqueName: \"kubernetes.io/projected/038b5d04-56fa-4d2a-9bff-f582dfb67243-kube-api-access-kgttk\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz29fs\" (UID: \"038b5d04-56fa-4d2a-9bff-f582dfb67243\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz29fs" Jul 13 07:06:55.124267 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:55.124203 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/038b5d04-56fa-4d2a-9bff-f582dfb67243-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz29fs\" (UID: \"038b5d04-56fa-4d2a-9bff-f582dfb67243\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz29fs" Jul 13 07:06:55.225507 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:55.225468 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/038b5d04-56fa-4d2a-9bff-f582dfb67243-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz29fs\" (UID: \"038b5d04-56fa-4d2a-9bff-f582dfb67243\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz29fs" Jul 13 07:06:55.225677 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:55.225556 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgttk\" (UniqueName: \"kubernetes.io/projected/038b5d04-56fa-4d2a-9bff-f582dfb67243-kube-api-access-kgttk\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz29fs\" (UID: \"038b5d04-56fa-4d2a-9bff-f582dfb67243\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz29fs" Jul 13 07:06:55.225677 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:55.225618 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/038b5d04-56fa-4d2a-9bff-f582dfb67243-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz29fs\" (UID: \"038b5d04-56fa-4d2a-9bff-f582dfb67243\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz29fs" Jul 13 07:06:55.225868 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:55.225845 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/038b5d04-56fa-4d2a-9bff-f582dfb67243-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz29fs\" (UID: \"038b5d04-56fa-4d2a-9bff-f582dfb67243\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz29fs" Jul 13 07:06:55.225982 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:55.225962 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/038b5d04-56fa-4d2a-9bff-f582dfb67243-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz29fs\" (UID: \"038b5d04-56fa-4d2a-9bff-f582dfb67243\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz29fs" Jul 13 07:06:55.232896 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:55.232870 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgttk\" (UniqueName: \"kubernetes.io/projected/038b5d04-56fa-4d2a-9bff-f582dfb67243-kube-api-access-kgttk\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz29fs\" (UID: \"038b5d04-56fa-4d2a-9bff-f582dfb67243\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz29fs" Jul 13 07:06:55.297393 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:55.297310 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz29fs" Jul 13 07:06:55.441460 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:55.441426 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz29fs"] Jul 13 07:06:55.442584 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:06:55.442559 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod038b5d04_56fa_4d2a_9bff_f582dfb67243.slice/crio-717447455a4c64a87e119a787074ce563f83cd6493605f2778fa94c67c42a660 WatchSource:0}: Error finding container 717447455a4c64a87e119a787074ce563f83cd6493605f2778fa94c67c42a660: Status 404 returned error can't find the container with id 717447455a4c64a87e119a787074ce563f83cd6493605f2778fa94c67c42a660 Jul 13 07:06:56.217918 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:56.217880 2586 generic.go:356] "Generic (PLEG): container finished" podID="038b5d04-56fa-4d2a-9bff-f582dfb67243" containerID="46e2cee85b84c4772ba8aaac85b9e7e00f88c65674831df5397e3a8a914b810f" exitCode=0 Jul 13 07:06:56.218293 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:56.217964 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz29fs" event={"ID":"038b5d04-56fa-4d2a-9bff-f582dfb67243","Type":"ContainerDied","Data":"46e2cee85b84c4772ba8aaac85b9e7e00f88c65674831df5397e3a8a914b810f"} Jul 13 07:06:56.218293 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:56.218040 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz29fs" event={"ID":"038b5d04-56fa-4d2a-9bff-f582dfb67243","Type":"ContainerStarted","Data":"717447455a4c64a87e119a787074ce563f83cd6493605f2778fa94c67c42a660"} Jul 13 07:06:57.224293 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:57.224258 2586 generic.go:356] "Generic (PLEG): container finished" podID="038b5d04-56fa-4d2a-9bff-f582dfb67243" containerID="8d728a891b9639763b0b5352a2a416e896319e9156786d52704e3d4e037aeb77" exitCode=0 Jul 13 07:06:57.224710 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:57.224344 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz29fs" event={"ID":"038b5d04-56fa-4d2a-9bff-f582dfb67243","Type":"ContainerDied","Data":"8d728a891b9639763b0b5352a2a416e896319e9156786d52704e3d4e037aeb77"} Jul 13 07:06:58.233933 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:58.233899 2586 generic.go:356] "Generic (PLEG): container finished" podID="038b5d04-56fa-4d2a-9bff-f582dfb67243" containerID="33b3ea46ba54358af3a207609b308070d49d94a128995119e74f8505e2edd573" exitCode=0 Jul 13 07:06:58.234302 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:58.233980 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz29fs" event={"ID":"038b5d04-56fa-4d2a-9bff-f582dfb67243","Type":"ContainerDied","Data":"33b3ea46ba54358af3a207609b308070d49d94a128995119e74f8505e2edd573"} Jul 13 07:06:59.366612 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:59.366591 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz29fs" Jul 13 07:06:59.458444 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:59.458417 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/038b5d04-56fa-4d2a-9bff-f582dfb67243-bundle\") pod \"038b5d04-56fa-4d2a-9bff-f582dfb67243\" (UID: \"038b5d04-56fa-4d2a-9bff-f582dfb67243\") " Jul 13 07:06:59.458583 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:59.458455 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kgttk\" (UniqueName: \"kubernetes.io/projected/038b5d04-56fa-4d2a-9bff-f582dfb67243-kube-api-access-kgttk\") pod \"038b5d04-56fa-4d2a-9bff-f582dfb67243\" (UID: \"038b5d04-56fa-4d2a-9bff-f582dfb67243\") " Jul 13 07:06:59.458583 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:59.458499 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/038b5d04-56fa-4d2a-9bff-f582dfb67243-util\") pod \"038b5d04-56fa-4d2a-9bff-f582dfb67243\" (UID: \"038b5d04-56fa-4d2a-9bff-f582dfb67243\") " Jul 13 07:06:59.459197 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:59.459167 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/038b5d04-56fa-4d2a-9bff-f582dfb67243-bundle" (OuterVolumeSpecName: "bundle") pod "038b5d04-56fa-4d2a-9bff-f582dfb67243" (UID: "038b5d04-56fa-4d2a-9bff-f582dfb67243"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:06:59.460594 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:59.460565 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/038b5d04-56fa-4d2a-9bff-f582dfb67243-kube-api-access-kgttk" (OuterVolumeSpecName: "kube-api-access-kgttk") pod "038b5d04-56fa-4d2a-9bff-f582dfb67243" (UID: "038b5d04-56fa-4d2a-9bff-f582dfb67243"). InnerVolumeSpecName "kube-api-access-kgttk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:06:59.464243 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:59.464216 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/038b5d04-56fa-4d2a-9bff-f582dfb67243-util" (OuterVolumeSpecName: "util") pod "038b5d04-56fa-4d2a-9bff-f582dfb67243" (UID: "038b5d04-56fa-4d2a-9bff-f582dfb67243"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:06:59.559099 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:59.559023 2586 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/038b5d04-56fa-4d2a-9bff-f582dfb67243-bundle\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:06:59.559099 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:59.559053 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kgttk\" (UniqueName: \"kubernetes.io/projected/038b5d04-56fa-4d2a-9bff-f582dfb67243-kube-api-access-kgttk\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:06:59.559099 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:06:59.559065 2586 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/038b5d04-56fa-4d2a-9bff-f582dfb67243-util\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:07:00.244351 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:00.244323 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz29fs" Jul 13 07:07:00.244351 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:00.244332 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afz29fs" event={"ID":"038b5d04-56fa-4d2a-9bff-f582dfb67243","Type":"ContainerDied","Data":"717447455a4c64a87e119a787074ce563f83cd6493605f2778fa94c67c42a660"} Jul 13 07:07:00.244555 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:00.244372 2586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="717447455a4c64a87e119a787074ce563f83cd6493605f2778fa94c67c42a660" Jul 13 07:07:05.613855 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:05.613822 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-r878q_293edc99-2691-4915-ba03-7407d88b89b4/console-operator/1.log" Jul 13 07:07:05.617985 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:05.617959 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7b964_3d71897a-4b2e-4e4e-9525-ccd89bd18458/ovn-acl-logging/0.log" Jul 13 07:07:05.618968 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:05.618947 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-r878q_293edc99-2691-4915-ba03-7407d88b89b4/console-operator/1.log" Jul 13 07:07:05.622993 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:05.622971 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7b964_3d71897a-4b2e-4e4e-9525-ccd89bd18458/ovn-acl-logging/0.log" Jul 13 07:07:05.625968 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:05.625945 2586 kubelet.go:1648] "Image garbage collection succeeded" Jul 13 07:07:06.909768 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:06.909733 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7487689bdd-959zg"] Jul 13 07:07:06.911119 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:06.910018 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="038b5d04-56fa-4d2a-9bff-f582dfb67243" containerName="util" Jul 13 07:07:06.911119 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:06.910028 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="038b5d04-56fa-4d2a-9bff-f582dfb67243" containerName="util" Jul 13 07:07:06.911119 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:06.910043 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="038b5d04-56fa-4d2a-9bff-f582dfb67243" containerName="extract" Jul 13 07:07:06.911119 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:06.910049 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="038b5d04-56fa-4d2a-9bff-f582dfb67243" containerName="extract" Jul 13 07:07:06.911119 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:06.910059 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="038b5d04-56fa-4d2a-9bff-f582dfb67243" containerName="pull" Jul 13 07:07:06.911119 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:06.910064 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="038b5d04-56fa-4d2a-9bff-f582dfb67243" containerName="pull" Jul 13 07:07:06.911119 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:06.910142 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="038b5d04-56fa-4d2a-9bff-f582dfb67243" containerName="extract" Jul 13 07:07:06.911988 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:06.911968 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7487689bdd-959zg" Jul 13 07:07:06.914499 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:06.914475 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-28h86\"" Jul 13 07:07:06.914651 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:06.914521 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 13 07:07:06.914858 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:06.914836 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 13 07:07:06.915015 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:06.914998 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 13 07:07:06.916609 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:06.916587 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 13 07:07:06.937200 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:06.937174 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7487689bdd-959zg"] Jul 13 07:07:07.008492 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:07.008458 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/59d14f61-a765-4c3b-a117-8bc2a9ee43a7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7487689bdd-959zg\" (UID: \"59d14f61-a765-4c3b-a117-8bc2a9ee43a7\") " pod="opendatahub/opendatahub-operator-controller-manager-7487689bdd-959zg" Jul 13 07:07:07.008492 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:07.008496 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/59d14f61-a765-4c3b-a117-8bc2a9ee43a7-manifests\") pod \"opendatahub-operator-controller-manager-7487689bdd-959zg\" (UID: \"59d14f61-a765-4c3b-a117-8bc2a9ee43a7\") " pod="opendatahub/opendatahub-operator-controller-manager-7487689bdd-959zg" Jul 13 07:07:07.008700 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:07.008582 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bj2n\" (UniqueName: \"kubernetes.io/projected/59d14f61-a765-4c3b-a117-8bc2a9ee43a7-kube-api-access-8bj2n\") pod \"opendatahub-operator-controller-manager-7487689bdd-959zg\" (UID: \"59d14f61-a765-4c3b-a117-8bc2a9ee43a7\") " pod="opendatahub/opendatahub-operator-controller-manager-7487689bdd-959zg" Jul 13 07:07:07.008700 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:07.008609 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/59d14f61-a765-4c3b-a117-8bc2a9ee43a7-tmp\") pod \"opendatahub-operator-controller-manager-7487689bdd-959zg\" (UID: \"59d14f61-a765-4c3b-a117-8bc2a9ee43a7\") " pod="opendatahub/opendatahub-operator-controller-manager-7487689bdd-959zg" Jul 13 07:07:07.008700 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:07.008636 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/59d14f61-a765-4c3b-a117-8bc2a9ee43a7-webhook-cert\") pod \"opendatahub-operator-controller-manager-7487689bdd-959zg\" (UID: \"59d14f61-a765-4c3b-a117-8bc2a9ee43a7\") " pod="opendatahub/opendatahub-operator-controller-manager-7487689bdd-959zg" Jul 13 07:07:07.109778 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:07.109742 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/59d14f61-a765-4c3b-a117-8bc2a9ee43a7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7487689bdd-959zg\" (UID: \"59d14f61-a765-4c3b-a117-8bc2a9ee43a7\") " pod="opendatahub/opendatahub-operator-controller-manager-7487689bdd-959zg" Jul 13 07:07:07.109778 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:07.109783 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/59d14f61-a765-4c3b-a117-8bc2a9ee43a7-manifests\") pod \"opendatahub-operator-controller-manager-7487689bdd-959zg\" (UID: \"59d14f61-a765-4c3b-a117-8bc2a9ee43a7\") " pod="opendatahub/opendatahub-operator-controller-manager-7487689bdd-959zg" Jul 13 07:07:07.109957 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:07.109827 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bj2n\" (UniqueName: \"kubernetes.io/projected/59d14f61-a765-4c3b-a117-8bc2a9ee43a7-kube-api-access-8bj2n\") pod \"opendatahub-operator-controller-manager-7487689bdd-959zg\" (UID: \"59d14f61-a765-4c3b-a117-8bc2a9ee43a7\") " pod="opendatahub/opendatahub-operator-controller-manager-7487689bdd-959zg" Jul 13 07:07:07.109957 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:07.109851 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/59d14f61-a765-4c3b-a117-8bc2a9ee43a7-tmp\") pod \"opendatahub-operator-controller-manager-7487689bdd-959zg\" (UID: \"59d14f61-a765-4c3b-a117-8bc2a9ee43a7\") " pod="opendatahub/opendatahub-operator-controller-manager-7487689bdd-959zg" Jul 13 07:07:07.109957 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:07.109878 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/59d14f61-a765-4c3b-a117-8bc2a9ee43a7-webhook-cert\") pod \"opendatahub-operator-controller-manager-7487689bdd-959zg\" (UID: \"59d14f61-a765-4c3b-a117-8bc2a9ee43a7\") " pod="opendatahub/opendatahub-operator-controller-manager-7487689bdd-959zg" Jul 13 07:07:07.110304 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:07.110269 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/59d14f61-a765-4c3b-a117-8bc2a9ee43a7-manifests\") pod \"opendatahub-operator-controller-manager-7487689bdd-959zg\" (UID: \"59d14f61-a765-4c3b-a117-8bc2a9ee43a7\") " pod="opendatahub/opendatahub-operator-controller-manager-7487689bdd-959zg" Jul 13 07:07:07.112124 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:07.112104 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/59d14f61-a765-4c3b-a117-8bc2a9ee43a7-tmp\") pod \"opendatahub-operator-controller-manager-7487689bdd-959zg\" (UID: \"59d14f61-a765-4c3b-a117-8bc2a9ee43a7\") " pod="opendatahub/opendatahub-operator-controller-manager-7487689bdd-959zg" Jul 13 07:07:07.112326 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:07.112309 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/59d14f61-a765-4c3b-a117-8bc2a9ee43a7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7487689bdd-959zg\" (UID: \"59d14f61-a765-4c3b-a117-8bc2a9ee43a7\") " pod="opendatahub/opendatahub-operator-controller-manager-7487689bdd-959zg" Jul 13 07:07:07.112326 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:07.112318 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/59d14f61-a765-4c3b-a117-8bc2a9ee43a7-webhook-cert\") pod \"opendatahub-operator-controller-manager-7487689bdd-959zg\" (UID: \"59d14f61-a765-4c3b-a117-8bc2a9ee43a7\") " pod="opendatahub/opendatahub-operator-controller-manager-7487689bdd-959zg" Jul 13 07:07:07.122396 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:07.122372 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bj2n\" (UniqueName: \"kubernetes.io/projected/59d14f61-a765-4c3b-a117-8bc2a9ee43a7-kube-api-access-8bj2n\") pod \"opendatahub-operator-controller-manager-7487689bdd-959zg\" (UID: \"59d14f61-a765-4c3b-a117-8bc2a9ee43a7\") " pod="opendatahub/opendatahub-operator-controller-manager-7487689bdd-959zg" Jul 13 07:07:07.222475 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:07.222443 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7487689bdd-959zg" Jul 13 07:07:07.359204 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:07.359178 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7487689bdd-959zg"] Jul 13 07:07:07.361653 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:07:07.361614 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59d14f61_a765_4c3b_a117_8bc2a9ee43a7.slice/crio-12fa8700d237682cffad5729b0e5144f709d3455ffdd1db95a3ba9c665544358 WatchSource:0}: Error finding container 12fa8700d237682cffad5729b0e5144f709d3455ffdd1db95a3ba9c665544358: Status 404 returned error can't find the container with id 12fa8700d237682cffad5729b0e5144f709d3455ffdd1db95a3ba9c665544358 Jul 13 07:07:07.364153 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:07.364132 2586 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 07:07:08.285878 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:08.285834 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7487689bdd-959zg" event={"ID":"59d14f61-a765-4c3b-a117-8bc2a9ee43a7","Type":"ContainerStarted","Data":"12fa8700d237682cffad5729b0e5144f709d3455ffdd1db95a3ba9c665544358"} Jul 13 07:07:11.301828 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:11.301795 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7487689bdd-959zg" event={"ID":"59d14f61-a765-4c3b-a117-8bc2a9ee43a7","Type":"ContainerStarted","Data":"e94db281dcbf447f97a8ae412a81809f57c8c05a8bbaa58b069b66f905218094"} Jul 13 07:07:12.307984 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:12.307948 2586 generic.go:356] "Generic (PLEG): container finished" podID="59d14f61-a765-4c3b-a117-8bc2a9ee43a7" containerID="e94db281dcbf447f97a8ae412a81809f57c8c05a8bbaa58b069b66f905218094" exitCode=0 Jul 13 07:07:12.308425 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:12.308029 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7487689bdd-959zg" event={"ID":"59d14f61-a765-4c3b-a117-8bc2a9ee43a7","Type":"ContainerDied","Data":"e94db281dcbf447f97a8ae412a81809f57c8c05a8bbaa58b069b66f905218094"} Jul 13 07:07:13.314544 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:13.314506 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7487689bdd-959zg" event={"ID":"59d14f61-a765-4c3b-a117-8bc2a9ee43a7","Type":"ContainerStarted","Data":"e0ba442bcf7198843234c51213cdc6c87a6355b8a1df4334381a4ced84e6ce9f"} Jul 13 07:07:13.314967 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:13.314611 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7487689bdd-959zg" Jul 13 07:07:13.332189 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:13.332138 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7487689bdd-959zg" podStartSLOduration=2.898890386 podStartE2EDuration="7.332124355s" totalImagesPullingTime="3.394701792s" totalInitContainerRuntime="1.038532177s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:07:06 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:07:13.330879951 +0000 UTC m=+308.166443781" watchObservedRunningTime="2026-07-13 07:07:13.332124355 +0000 UTC m=+308.167688188" Jul 13 07:07:24.322235 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:24.322200 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7487689bdd-959zg" Jul 13 07:07:26.751426 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:26.751387 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-9fcd7cd9-5spff"] Jul 13 07:07:26.760423 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:26.760396 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-9fcd7cd9-5spff" Jul 13 07:07:26.762620 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:26.762596 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 13 07:07:26.762907 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:26.762881 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 13 07:07:26.763157 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:26.763137 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-g74rf\"" Jul 13 07:07:26.766186 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:26.766163 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-9fcd7cd9-5spff"] Jul 13 07:07:26.877135 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:26.877093 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/45fe8f55-a887-467e-baf8-61f5b0cc4d78-tmp\") pod \"kube-auth-proxy-9fcd7cd9-5spff\" (UID: \"45fe8f55-a887-467e-baf8-61f5b0cc4d78\") " pod="openshift-ingress/kube-auth-proxy-9fcd7cd9-5spff" Jul 13 07:07:26.877315 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:26.877145 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qspcm\" (UniqueName: \"kubernetes.io/projected/45fe8f55-a887-467e-baf8-61f5b0cc4d78-kube-api-access-qspcm\") pod \"kube-auth-proxy-9fcd7cd9-5spff\" (UID: \"45fe8f55-a887-467e-baf8-61f5b0cc4d78\") " pod="openshift-ingress/kube-auth-proxy-9fcd7cd9-5spff" Jul 13 07:07:26.877315 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:26.877228 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45fe8f55-a887-467e-baf8-61f5b0cc4d78-tls-certs\") pod \"kube-auth-proxy-9fcd7cd9-5spff\" (UID: \"45fe8f55-a887-467e-baf8-61f5b0cc4d78\") " pod="openshift-ingress/kube-auth-proxy-9fcd7cd9-5spff" Jul 13 07:07:26.932022 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:26.931980 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-v77jc"] Jul 13 07:07:26.935098 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:26.935081 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-v77jc" Jul 13 07:07:26.937202 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:26.937175 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-55mjq\"" Jul 13 07:07:26.937333 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:26.937226 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 13 07:07:26.948324 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:26.948296 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-v77jc"] Jul 13 07:07:26.977849 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:26.977820 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/531be0a6-b340-4af9-8064-6f8066f1c69a-cert\") pod \"odh-model-controller-7998c67dc4-v77jc\" (UID: \"531be0a6-b340-4af9-8064-6f8066f1c69a\") " pod="opendatahub/odh-model-controller-7998c67dc4-v77jc" Jul 13 07:07:26.978012 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:26.977882 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/45fe8f55-a887-467e-baf8-61f5b0cc4d78-tmp\") pod \"kube-auth-proxy-9fcd7cd9-5spff\" (UID: \"45fe8f55-a887-467e-baf8-61f5b0cc4d78\") " pod="openshift-ingress/kube-auth-proxy-9fcd7cd9-5spff" Jul 13 07:07:26.978012 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:26.977912 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qspcm\" (UniqueName: \"kubernetes.io/projected/45fe8f55-a887-467e-baf8-61f5b0cc4d78-kube-api-access-qspcm\") pod \"kube-auth-proxy-9fcd7cd9-5spff\" (UID: \"45fe8f55-a887-467e-baf8-61f5b0cc4d78\") " pod="openshift-ingress/kube-auth-proxy-9fcd7cd9-5spff" Jul 13 07:07:26.978012 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:26.977935 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szfck\" (UniqueName: \"kubernetes.io/projected/531be0a6-b340-4af9-8064-6f8066f1c69a-kube-api-access-szfck\") pod \"odh-model-controller-7998c67dc4-v77jc\" (UID: \"531be0a6-b340-4af9-8064-6f8066f1c69a\") " pod="opendatahub/odh-model-controller-7998c67dc4-v77jc" Jul 13 07:07:26.978012 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:26.977957 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45fe8f55-a887-467e-baf8-61f5b0cc4d78-tls-certs\") pod \"kube-auth-proxy-9fcd7cd9-5spff\" (UID: \"45fe8f55-a887-467e-baf8-61f5b0cc4d78\") " pod="openshift-ingress/kube-auth-proxy-9fcd7cd9-5spff" Jul 13 07:07:26.978186 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:07:26.978070 2586 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 13 07:07:26.978186 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:07:26.978154 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45fe8f55-a887-467e-baf8-61f5b0cc4d78-tls-certs podName:45fe8f55-a887-467e-baf8-61f5b0cc4d78 nodeName:}" failed. No retries permitted until 2026-07-13 07:07:27.478131723 +0000 UTC m=+322.313695533 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/45fe8f55-a887-467e-baf8-61f5b0cc4d78-tls-certs") pod "kube-auth-proxy-9fcd7cd9-5spff" (UID: "45fe8f55-a887-467e-baf8-61f5b0cc4d78") : secret "kube-auth-proxy-tls" not found Jul 13 07:07:26.980468 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:26.980442 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/45fe8f55-a887-467e-baf8-61f5b0cc4d78-tmp\") pod \"kube-auth-proxy-9fcd7cd9-5spff\" (UID: \"45fe8f55-a887-467e-baf8-61f5b0cc4d78\") " pod="openshift-ingress/kube-auth-proxy-9fcd7cd9-5spff" Jul 13 07:07:26.985363 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:26.985341 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qspcm\" (UniqueName: \"kubernetes.io/projected/45fe8f55-a887-467e-baf8-61f5b0cc4d78-kube-api-access-qspcm\") pod \"kube-auth-proxy-9fcd7cd9-5spff\" (UID: \"45fe8f55-a887-467e-baf8-61f5b0cc4d78\") " pod="openshift-ingress/kube-auth-proxy-9fcd7cd9-5spff" Jul 13 07:07:27.079417 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:27.079306 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/531be0a6-b340-4af9-8064-6f8066f1c69a-cert\") pod \"odh-model-controller-7998c67dc4-v77jc\" (UID: \"531be0a6-b340-4af9-8064-6f8066f1c69a\") " pod="opendatahub/odh-model-controller-7998c67dc4-v77jc" Jul 13 07:07:27.079583 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:27.079423 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szfck\" (UniqueName: \"kubernetes.io/projected/531be0a6-b340-4af9-8064-6f8066f1c69a-kube-api-access-szfck\") pod \"odh-model-controller-7998c67dc4-v77jc\" (UID: \"531be0a6-b340-4af9-8064-6f8066f1c69a\") " pod="opendatahub/odh-model-controller-7998c67dc4-v77jc" Jul 13 07:07:27.079583 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:07:27.079459 2586 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 13 07:07:27.079583 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:07:27.079546 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/531be0a6-b340-4af9-8064-6f8066f1c69a-cert podName:531be0a6-b340-4af9-8064-6f8066f1c69a nodeName:}" failed. No retries permitted until 2026-07-13 07:07:27.57953161 +0000 UTC m=+322.415095418 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/531be0a6-b340-4af9-8064-6f8066f1c69a-cert") pod "odh-model-controller-7998c67dc4-v77jc" (UID: "531be0a6-b340-4af9-8064-6f8066f1c69a") : secret "odh-model-controller-webhook-cert" not found Jul 13 07:07:27.089517 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:27.089485 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szfck\" (UniqueName: \"kubernetes.io/projected/531be0a6-b340-4af9-8064-6f8066f1c69a-kube-api-access-szfck\") pod \"odh-model-controller-7998c67dc4-v77jc\" (UID: \"531be0a6-b340-4af9-8064-6f8066f1c69a\") " pod="opendatahub/odh-model-controller-7998c67dc4-v77jc" Jul 13 07:07:27.482622 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:27.482586 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45fe8f55-a887-467e-baf8-61f5b0cc4d78-tls-certs\") pod \"kube-auth-proxy-9fcd7cd9-5spff\" (UID: \"45fe8f55-a887-467e-baf8-61f5b0cc4d78\") " pod="openshift-ingress/kube-auth-proxy-9fcd7cd9-5spff" Jul 13 07:07:27.485362 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:27.485330 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45fe8f55-a887-467e-baf8-61f5b0cc4d78-tls-certs\") pod \"kube-auth-proxy-9fcd7cd9-5spff\" (UID: \"45fe8f55-a887-467e-baf8-61f5b0cc4d78\") " pod="openshift-ingress/kube-auth-proxy-9fcd7cd9-5spff" Jul 13 07:07:27.583830 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:27.583794 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/531be0a6-b340-4af9-8064-6f8066f1c69a-cert\") pod \"odh-model-controller-7998c67dc4-v77jc\" (UID: \"531be0a6-b340-4af9-8064-6f8066f1c69a\") " pod="opendatahub/odh-model-controller-7998c67dc4-v77jc" Jul 13 07:07:27.586077 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:27.586048 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/531be0a6-b340-4af9-8064-6f8066f1c69a-cert\") pod \"odh-model-controller-7998c67dc4-v77jc\" (UID: \"531be0a6-b340-4af9-8064-6f8066f1c69a\") " pod="opendatahub/odh-model-controller-7998c67dc4-v77jc" Jul 13 07:07:27.670650 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:27.670618 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-9fcd7cd9-5spff" Jul 13 07:07:27.808893 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:27.808865 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-9fcd7cd9-5spff"] Jul 13 07:07:27.811443 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:07:27.811408 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45fe8f55_a887_467e_baf8_61f5b0cc4d78.slice/crio-69cc5cfcf6d268f662379b75b60faddeab668f9c65196f3b838c4d78e4f09392 WatchSource:0}: Error finding container 69cc5cfcf6d268f662379b75b60faddeab668f9c65196f3b838c4d78e4f09392: Status 404 returned error can't find the container with id 69cc5cfcf6d268f662379b75b60faddeab668f9c65196f3b838c4d78e4f09392 Jul 13 07:07:27.845280 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:27.845258 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-v77jc" Jul 13 07:07:27.975418 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:27.975294 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-v77jc"] Jul 13 07:07:27.977518 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:07:27.977488 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod531be0a6_b340_4af9_8064_6f8066f1c69a.slice/crio-420a17e5676b1f60dcd930cdf55c0e1d23851dcb03b2d230329e1fe958ab23fe WatchSource:0}: Error finding container 420a17e5676b1f60dcd930cdf55c0e1d23851dcb03b2d230329e1fe958ab23fe: Status 404 returned error can't find the container with id 420a17e5676b1f60dcd930cdf55c0e1d23851dcb03b2d230329e1fe958ab23fe Jul 13 07:07:27.979424 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:27.979400 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-66fbs"] Jul 13 07:07:27.982625 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:27.982606 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-66fbs" Jul 13 07:07:27.984599 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:27.984578 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 13 07:07:27.984762 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:27.984616 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 13 07:07:27.984762 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:27.984577 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 13 07:07:27.984762 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:27.984675 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 13 07:07:27.984927 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:27.984862 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-r2ddd\"" Jul 13 07:07:27.998271 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:27.998248 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-66fbs"] Jul 13 07:07:28.089134 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:28.089065 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c-local-certs\") pod \"istiod-openshift-gateway-7574459656-66fbs\" (UID: \"ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-66fbs" Jul 13 07:07:28.089134 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:28.089104 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-66fbs\" (UID: \"ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-66fbs" Jul 13 07:07:28.089134 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:28.089124 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-66fbs\" (UID: \"ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-66fbs" Jul 13 07:07:28.089360 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:28.089144 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c-cacerts\") pod \"istiod-openshift-gateway-7574459656-66fbs\" (UID: \"ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-66fbs" Jul 13 07:07:28.089360 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:28.089220 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-66fbs\" (UID: \"ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-66fbs" Jul 13 07:07:28.089360 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:28.089270 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c-istio-token\") pod \"istiod-openshift-gateway-7574459656-66fbs\" (UID: \"ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-66fbs" Jul 13 07:07:28.089360 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:28.089289 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lnvv\" (UniqueName: \"kubernetes.io/projected/ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c-kube-api-access-9lnvv\") pod \"istiod-openshift-gateway-7574459656-66fbs\" (UID: \"ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-66fbs" Jul 13 07:07:28.190382 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:28.190354 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c-local-certs\") pod \"istiod-openshift-gateway-7574459656-66fbs\" (UID: \"ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-66fbs" Jul 13 07:07:28.190555 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:28.190395 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-66fbs\" (UID: \"ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-66fbs" Jul 13 07:07:28.190555 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:28.190418 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-66fbs\" (UID: \"ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-66fbs" Jul 13 07:07:28.190555 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:28.190445 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c-cacerts\") pod \"istiod-openshift-gateway-7574459656-66fbs\" (UID: \"ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-66fbs" Jul 13 07:07:28.190555 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:28.190476 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-66fbs\" (UID: \"ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-66fbs" Jul 13 07:07:28.190555 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:28.190511 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c-istio-token\") pod \"istiod-openshift-gateway-7574459656-66fbs\" (UID: \"ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-66fbs" Jul 13 07:07:28.190555 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:28.190529 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lnvv\" (UniqueName: \"kubernetes.io/projected/ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c-kube-api-access-9lnvv\") pod \"istiod-openshift-gateway-7574459656-66fbs\" (UID: \"ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-66fbs" Jul 13 07:07:28.191133 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:28.191107 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-66fbs\" (UID: \"ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-66fbs" Jul 13 07:07:28.192625 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:28.192597 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c-local-certs\") pod \"istiod-openshift-gateway-7574459656-66fbs\" (UID: \"ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-66fbs" Jul 13 07:07:28.192854 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:28.192833 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-66fbs\" (UID: \"ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-66fbs" Jul 13 07:07:28.192926 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:28.192883 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-66fbs\" (UID: \"ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-66fbs" Jul 13 07:07:28.192926 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:28.192895 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c-cacerts\") pod \"istiod-openshift-gateway-7574459656-66fbs\" (UID: \"ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-66fbs" Jul 13 07:07:28.203519 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:28.203491 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lnvv\" (UniqueName: \"kubernetes.io/projected/ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c-kube-api-access-9lnvv\") pod \"istiod-openshift-gateway-7574459656-66fbs\" (UID: \"ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-66fbs" Jul 13 07:07:28.204555 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:28.204534 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c-istio-token\") pod \"istiod-openshift-gateway-7574459656-66fbs\" (UID: \"ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-66fbs" Jul 13 07:07:28.299926 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:28.299887 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-66fbs" Jul 13 07:07:28.386550 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:28.386510 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-v77jc" event={"ID":"531be0a6-b340-4af9-8064-6f8066f1c69a","Type":"ContainerStarted","Data":"420a17e5676b1f60dcd930cdf55c0e1d23851dcb03b2d230329e1fe958ab23fe"} Jul 13 07:07:28.387752 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:28.387721 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-9fcd7cd9-5spff" event={"ID":"45fe8f55-a887-467e-baf8-61f5b0cc4d78","Type":"ContainerStarted","Data":"69cc5cfcf6d268f662379b75b60faddeab668f9c65196f3b838c4d78e4f09392"} Jul 13 07:07:28.433304 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:28.433267 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-66fbs"] Jul 13 07:07:28.435214 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:07:28.435191 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebf1cd21_6f40_43b6_b3b2_e9f1a71a416c.slice/crio-7511ba93419495ca4bf4da019e0241ef30ec4cf3259e1f3207269846175b67e8 WatchSource:0}: Error finding container 7511ba93419495ca4bf4da019e0241ef30ec4cf3259e1f3207269846175b67e8: Status 404 returned error can't find the container with id 7511ba93419495ca4bf4da019e0241ef30ec4cf3259e1f3207269846175b67e8 Jul 13 07:07:29.398877 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:29.398793 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-66fbs" event={"ID":"ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c","Type":"ContainerStarted","Data":"7511ba93419495ca4bf4da019e0241ef30ec4cf3259e1f3207269846175b67e8"} Jul 13 07:07:32.806673 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:32.806641 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 07:07:32.806976 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:32.806774 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 07:07:33.424004 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:33.423970 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-v77jc" event={"ID":"531be0a6-b340-4af9-8064-6f8066f1c69a","Type":"ContainerStarted","Data":"98e5c22eb9f42b534862ecb6a05d70b148239ee87db7201dddd43a5745824d08"} Jul 13 07:07:33.424185 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:33.424100 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-v77jc" Jul 13 07:07:33.431902 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:33.431863 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-66fbs" event={"ID":"ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c","Type":"ContainerStarted","Data":"0f9871c68734a68782cd32067894e56b552aa3e7c040488f4530caee367eacb3"} Jul 13 07:07:33.432074 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:33.432052 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-66fbs" Jul 13 07:07:33.433650 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:33.433624 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-9fcd7cd9-5spff" event={"ID":"45fe8f55-a887-467e-baf8-61f5b0cc4d78","Type":"ContainerStarted","Data":"c5db0b0ea08a98613f023869600cab54657227f25c857a5ceba5510f6d476d26"} Jul 13 07:07:33.433940 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:33.433913 2586 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-66fbs container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 13 07:07:33.434051 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:33.433984 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-66fbs" podUID="ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 13 07:07:33.443828 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:33.443789 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-v77jc" podStartSLOduration=2.617106847 podStartE2EDuration="7.443774943s" totalImagesPullingTime="4.826668096s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:07:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:07:33.443065539 +0000 UTC m=+328.278629370" watchObservedRunningTime="2026-07-13 07:07:33.443774943 +0000 UTC m=+328.279338774" Jul 13 07:07:33.458841 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:33.458798 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-9fcd7cd9-5spff" podStartSLOduration=2.471574346 podStartE2EDuration="7.45878593s" totalImagesPullingTime="4.987211584s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:07:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:07:33.457571729 +0000 UTC m=+328.293135586" watchObservedRunningTime="2026-07-13 07:07:33.45878593 +0000 UTC m=+328.294349760" Jul 13 07:07:33.477626 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:33.477577 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-66fbs" podStartSLOduration=2.11152047 podStartE2EDuration="6.477564171s" totalImagesPullingTime="4.366043701s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:07:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:07:33.476508063 +0000 UTC m=+328.312071897" watchObservedRunningTime="2026-07-13 07:07:33.477564171 +0000 UTC m=+328.313128006" Jul 13 07:07:34.443758 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:34.443727 2586 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-66fbs container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 13 07:07:34.444119 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:34.443780 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-66fbs" podUID="ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 13 07:07:37.328373 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:37.328338 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-4mlft"] Jul 13 07:07:37.331731 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:37.331706 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4mlft" Jul 13 07:07:37.333823 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:37.333803 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 13 07:07:37.333907 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:37.333845 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-2l2mk\"" Jul 13 07:07:37.344545 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:37.344522 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-4mlft"] Jul 13 07:07:37.444494 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:37.444465 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-66fbs" Jul 13 07:07:37.473843 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:37.473809 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxclp\" (UniqueName: \"kubernetes.io/projected/ce788bde-51f0-466a-93b9-248e4d3b5f5c-kube-api-access-bxclp\") pod \"llmisvc-controller-manager-777c6bd7d5-4mlft\" (UID: \"ce788bde-51f0-466a-93b9-248e4d3b5f5c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4mlft" Jul 13 07:07:37.473843 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:37.473847 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ce788bde-51f0-466a-93b9-248e4d3b5f5c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-4mlft\" (UID: \"ce788bde-51f0-466a-93b9-248e4d3b5f5c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4mlft" Jul 13 07:07:37.575460 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:37.575427 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxclp\" (UniqueName: \"kubernetes.io/projected/ce788bde-51f0-466a-93b9-248e4d3b5f5c-kube-api-access-bxclp\") pod \"llmisvc-controller-manager-777c6bd7d5-4mlft\" (UID: \"ce788bde-51f0-466a-93b9-248e4d3b5f5c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4mlft" Jul 13 07:07:37.575632 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:37.575574 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ce788bde-51f0-466a-93b9-248e4d3b5f5c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-4mlft\" (UID: \"ce788bde-51f0-466a-93b9-248e4d3b5f5c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4mlft" Jul 13 07:07:37.575844 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:07:37.575823 2586 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 13 07:07:37.575937 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:07:37.575885 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce788bde-51f0-466a-93b9-248e4d3b5f5c-cert podName:ce788bde-51f0-466a-93b9-248e4d3b5f5c nodeName:}" failed. No retries permitted until 2026-07-13 07:07:38.075867139 +0000 UTC m=+332.911430948 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ce788bde-51f0-466a-93b9-248e4d3b5f5c-cert") pod "llmisvc-controller-manager-777c6bd7d5-4mlft" (UID: "ce788bde-51f0-466a-93b9-248e4d3b5f5c") : secret "llmisvc-webhook-server-cert" not found Jul 13 07:07:37.590159 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:37.590099 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxclp\" (UniqueName: \"kubernetes.io/projected/ce788bde-51f0-466a-93b9-248e4d3b5f5c-kube-api-access-bxclp\") pod \"llmisvc-controller-manager-777c6bd7d5-4mlft\" (UID: \"ce788bde-51f0-466a-93b9-248e4d3b5f5c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4mlft" Jul 13 07:07:38.080499 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:38.080465 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ce788bde-51f0-466a-93b9-248e4d3b5f5c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-4mlft\" (UID: \"ce788bde-51f0-466a-93b9-248e4d3b5f5c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4mlft" Jul 13 07:07:38.082927 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:38.082905 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ce788bde-51f0-466a-93b9-248e4d3b5f5c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-4mlft\" (UID: \"ce788bde-51f0-466a-93b9-248e4d3b5f5c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4mlft" Jul 13 07:07:38.243956 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:38.243923 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4mlft" Jul 13 07:07:38.369322 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:38.369216 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-4mlft"] Jul 13 07:07:38.371666 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:07:38.371638 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podce788bde_51f0_466a_93b9_248e4d3b5f5c.slice/crio-6a2fbfae4913043964b22436a680b2aed1afd2905b71c42f1dbec31ac7973503 WatchSource:0}: Error finding container 6a2fbfae4913043964b22436a680b2aed1afd2905b71c42f1dbec31ac7973503: Status 404 returned error can't find the container with id 6a2fbfae4913043964b22436a680b2aed1afd2905b71c42f1dbec31ac7973503 Jul 13 07:07:38.463904 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:38.463869 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4mlft" event={"ID":"ce788bde-51f0-466a-93b9-248e4d3b5f5c","Type":"ContainerStarted","Data":"6a2fbfae4913043964b22436a680b2aed1afd2905b71c42f1dbec31ac7973503"} Jul 13 07:07:38.827405 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:38.827365 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-66fbs"] Jul 13 07:07:38.827564 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:38.827545 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-66fbs" podUID="ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c" containerName="discovery" containerID="cri-o://0f9871c68734a68782cd32067894e56b552aa3e7c040488f4530caee367eacb3" gracePeriod=30 Jul 13 07:07:39.083590 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:39.083531 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-66fbs" Jul 13 07:07:39.190862 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:39.190822 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c-cacerts\") pod \"ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c\" (UID: \"ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c\") " Jul 13 07:07:39.191061 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:39.190868 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c-istio-csr-ca-configmap\") pod \"ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c\" (UID: \"ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c\") " Jul 13 07:07:39.191061 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:39.190939 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9lnvv\" (UniqueName: \"kubernetes.io/projected/ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c-kube-api-access-9lnvv\") pod \"ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c\" (UID: \"ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c\") " Jul 13 07:07:39.191061 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:39.190981 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c-istio-csr-dns-cert\") pod \"ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c\" (UID: \"ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c\") " Jul 13 07:07:39.191061 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:39.191039 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c-istio-token\") pod \"ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c\" (UID: \"ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c\") " Jul 13 07:07:39.191259 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:39.191084 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c-local-certs\") pod \"ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c\" (UID: \"ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c\") " Jul 13 07:07:39.191259 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:39.191118 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c-istio-kubeconfig\") pod \"ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c\" (UID: \"ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c\") " Jul 13 07:07:39.191368 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:39.191255 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c" (UID: "ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 07:07:39.191426 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:39.191400 2586 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c-istio-csr-ca-configmap\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:07:39.194017 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:39.193972 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c" (UID: "ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 07:07:39.194134 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:39.194112 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c" (UID: "ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 07:07:39.194393 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:39.194352 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c-cacerts" (OuterVolumeSpecName: "cacerts") pod "ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c" (UID: "ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 07:07:39.194504 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:39.194403 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c-local-certs" (OuterVolumeSpecName: "local-certs") pod "ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c" (UID: "ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:07:39.194608 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:39.194575 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c-kube-api-access-9lnvv" (OuterVolumeSpecName: "kube-api-access-9lnvv") pod "ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c" (UID: "ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c"). InnerVolumeSpecName "kube-api-access-9lnvv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:07:39.194671 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:39.194626 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c-istio-token" (OuterVolumeSpecName: "istio-token") pod "ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c" (UID: "ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:07:39.291996 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:39.291965 2586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c-istio-token\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:07:39.291996 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:39.291996 2586 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c-local-certs\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:07:39.292184 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:39.292009 2586 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c-istio-kubeconfig\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:07:39.292184 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:39.292021 2586 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c-cacerts\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:07:39.292184 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:39.292033 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9lnvv\" (UniqueName: \"kubernetes.io/projected/ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c-kube-api-access-9lnvv\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:07:39.292184 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:39.292047 2586 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c-istio-csr-dns-cert\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:07:39.470935 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:39.470888 2586 generic.go:356] "Generic (PLEG): container finished" podID="ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c" containerID="0f9871c68734a68782cd32067894e56b552aa3e7c040488f4530caee367eacb3" exitCode=0 Jul 13 07:07:39.471328 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:39.470964 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-66fbs" Jul 13 07:07:39.471328 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:39.470989 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-66fbs" event={"ID":"ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c","Type":"ContainerDied","Data":"0f9871c68734a68782cd32067894e56b552aa3e7c040488f4530caee367eacb3"} Jul 13 07:07:39.471328 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:39.471052 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-66fbs" event={"ID":"ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c","Type":"ContainerDied","Data":"7511ba93419495ca4bf4da019e0241ef30ec4cf3259e1f3207269846175b67e8"} Jul 13 07:07:39.471328 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:39.471077 2586 scope.go:117] "RemoveContainer" containerID="0f9871c68734a68782cd32067894e56b552aa3e7c040488f4530caee367eacb3" Jul 13 07:07:39.497033 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:39.497007 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-66fbs"] Jul 13 07:07:39.499979 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:39.499956 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-66fbs"] Jul 13 07:07:39.592881 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:39.592850 2586 scope.go:117] "RemoveContainer" containerID="0f9871c68734a68782cd32067894e56b552aa3e7c040488f4530caee367eacb3" Jul 13 07:07:39.593251 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:07:39.593227 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f9871c68734a68782cd32067894e56b552aa3e7c040488f4530caee367eacb3\": container with ID starting with 0f9871c68734a68782cd32067894e56b552aa3e7c040488f4530caee367eacb3 not found: ID does not exist" containerID="0f9871c68734a68782cd32067894e56b552aa3e7c040488f4530caee367eacb3" Jul 13 07:07:39.593354 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:39.593261 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f9871c68734a68782cd32067894e56b552aa3e7c040488f4530caee367eacb3"} err="failed to get container status \"0f9871c68734a68782cd32067894e56b552aa3e7c040488f4530caee367eacb3\": rpc error: code = NotFound desc = could not find container \"0f9871c68734a68782cd32067894e56b552aa3e7c040488f4530caee367eacb3\": container with ID starting with 0f9871c68734a68782cd32067894e56b552aa3e7c040488f4530caee367eacb3 not found: ID does not exist" Jul 13 07:07:39.806819 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:39.806744 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c" path="/var/lib/kubelet/pods/ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c/volumes" Jul 13 07:07:42.490941 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:42.490906 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4mlft" event={"ID":"ce788bde-51f0-466a-93b9-248e4d3b5f5c","Type":"ContainerStarted","Data":"8d5c0b30425466dc17b9048d56f4ef30c2b32c6efb39d52f757d6bec4defb806"} Jul 13 07:07:42.491315 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:42.490988 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4mlft" Jul 13 07:07:42.506789 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:42.506744 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4mlft" podStartSLOduration=2.472722166 podStartE2EDuration="5.506730125s" totalImagesPullingTime="3.034007959s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:07:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:07:42.504955697 +0000 UTC m=+337.340519540" watchObservedRunningTime="2026-07-13 07:07:42.506730125 +0000 UTC m=+337.342293956" Jul 13 07:07:44.445518 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:07:44.445486 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-v77jc" Jul 13 07:08:13.498447 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:13.498413 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4mlft" Jul 13 07:08:27.838716 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:27.838665 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-cw6jh"] Jul 13 07:08:27.839114 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:27.839021 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c" containerName="discovery" Jul 13 07:08:27.839114 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:27.839033 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c" containerName="discovery" Jul 13 07:08:27.839114 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:27.839096 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebf1cd21-6f40-43b6-b3b2-e9f1a71a416c" containerName="discovery" Jul 13 07:08:27.854494 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:27.854456 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-cw6jh"] Jul 13 07:08:27.854696 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:27.854472 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-cw6jh" Jul 13 07:08:27.856783 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:27.856753 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 13 07:08:27.856911 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:27.856761 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-bgq6k\"" Jul 13 07:08:27.857287 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:27.857263 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 13 07:08:27.887155 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:27.887110 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xsvb7\" (UniqueName: \"kubernetes.io/projected/9135e325-463e-4e79-99d4-e391b6fba716-kube-api-access-xsvb7\") pod \"kuadrant-operator-catalog-cw6jh\" (UID: \"9135e325-463e-4e79-99d4-e391b6fba716\") " pod="kuadrant-system/kuadrant-operator-catalog-cw6jh" Jul 13 07:08:27.988562 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:27.988512 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xsvb7\" (UniqueName: \"kubernetes.io/projected/9135e325-463e-4e79-99d4-e391b6fba716-kube-api-access-xsvb7\") pod \"kuadrant-operator-catalog-cw6jh\" (UID: \"9135e325-463e-4e79-99d4-e391b6fba716\") " pod="kuadrant-system/kuadrant-operator-catalog-cw6jh" Jul 13 07:08:27.995900 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:27.995864 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xsvb7\" (UniqueName: \"kubernetes.io/projected/9135e325-463e-4e79-99d4-e391b6fba716-kube-api-access-xsvb7\") pod \"kuadrant-operator-catalog-cw6jh\" (UID: \"9135e325-463e-4e79-99d4-e391b6fba716\") " pod="kuadrant-system/kuadrant-operator-catalog-cw6jh" Jul 13 07:08:28.166015 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:28.165924 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-cw6jh" Jul 13 07:08:28.217308 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:28.217215 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-cw6jh"] Jul 13 07:08:28.301867 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:28.301743 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-cw6jh"] Jul 13 07:08:28.304550 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:08:28.304520 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9135e325_463e_4e79_99d4_e391b6fba716.slice/crio-bffbb312ecc2c238a2bcb21af9b8491e62f98aebb767338dfcdf207d4dc9985c WatchSource:0}: Error finding container bffbb312ecc2c238a2bcb21af9b8491e62f98aebb767338dfcdf207d4dc9985c: Status 404 returned error can't find the container with id bffbb312ecc2c238a2bcb21af9b8491e62f98aebb767338dfcdf207d4dc9985c Jul 13 07:08:28.421163 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:28.421081 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-bktks"] Jul 13 07:08:28.424112 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:28.424091 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-bktks" Jul 13 07:08:28.431764 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:28.431735 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-bktks"] Jul 13 07:08:28.494371 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:28.494328 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvmxs\" (UniqueName: \"kubernetes.io/projected/5782c505-1c63-4707-a56e-85ceaf43af36-kube-api-access-zvmxs\") pod \"kuadrant-operator-catalog-bktks\" (UID: \"5782c505-1c63-4707-a56e-85ceaf43af36\") " pod="kuadrant-system/kuadrant-operator-catalog-bktks" Jul 13 07:08:28.595096 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:28.595047 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvmxs\" (UniqueName: \"kubernetes.io/projected/5782c505-1c63-4707-a56e-85ceaf43af36-kube-api-access-zvmxs\") pod \"kuadrant-operator-catalog-bktks\" (UID: \"5782c505-1c63-4707-a56e-85ceaf43af36\") " pod="kuadrant-system/kuadrant-operator-catalog-bktks" Jul 13 07:08:28.603560 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:28.603526 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvmxs\" (UniqueName: \"kubernetes.io/projected/5782c505-1c63-4707-a56e-85ceaf43af36-kube-api-access-zvmxs\") pod \"kuadrant-operator-catalog-bktks\" (UID: \"5782c505-1c63-4707-a56e-85ceaf43af36\") " pod="kuadrant-system/kuadrant-operator-catalog-bktks" Jul 13 07:08:28.700957 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:28.700919 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-cw6jh" event={"ID":"9135e325-463e-4e79-99d4-e391b6fba716","Type":"ContainerStarted","Data":"bffbb312ecc2c238a2bcb21af9b8491e62f98aebb767338dfcdf207d4dc9985c"} Jul 13 07:08:28.735203 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:28.735150 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-bktks" Jul 13 07:08:28.869399 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:28.869374 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-bktks"] Jul 13 07:08:28.871340 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:08:28.871316 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5782c505_1c63_4707_a56e_85ceaf43af36.slice/crio-b64d64fb5dafd595caa1ea71fdfe9aabe49f637be57a4da8d204a7b19be9b001 WatchSource:0}: Error finding container b64d64fb5dafd595caa1ea71fdfe9aabe49f637be57a4da8d204a7b19be9b001: Status 404 returned error can't find the container with id b64d64fb5dafd595caa1ea71fdfe9aabe49f637be57a4da8d204a7b19be9b001 Jul 13 07:08:29.709036 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:29.708989 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-bktks" event={"ID":"5782c505-1c63-4707-a56e-85ceaf43af36","Type":"ContainerStarted","Data":"b64d64fb5dafd595caa1ea71fdfe9aabe49f637be57a4da8d204a7b19be9b001"} Jul 13 07:08:30.715938 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:30.715883 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-bktks" event={"ID":"5782c505-1c63-4707-a56e-85ceaf43af36","Type":"ContainerStarted","Data":"05c153e8b458b2cc4e5592b580d0686efe415d2318c097bf9ade887cb5494270"} Jul 13 07:08:30.717429 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:30.717396 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-cw6jh" event={"ID":"9135e325-463e-4e79-99d4-e391b6fba716","Type":"ContainerStarted","Data":"725a1305cecec9d3680d44f996f15224f75f3b9dddef1c8deaaebf8bb7438741"} Jul 13 07:08:30.717524 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:30.717472 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-cw6jh" podUID="9135e325-463e-4e79-99d4-e391b6fba716" containerName="registry-server" containerID="cri-o://725a1305cecec9d3680d44f996f15224f75f3b9dddef1c8deaaebf8bb7438741" gracePeriod=2 Jul 13 07:08:30.729644 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:30.729596 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-bktks" podStartSLOduration=1.092578431 podStartE2EDuration="2.729582444s" totalImagesPullingTime="1.637004013s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:08:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:08:30.728480434 +0000 UTC m=+385.564044265" watchObservedRunningTime="2026-07-13 07:08:30.729582444 +0000 UTC m=+385.565146275" Jul 13 07:08:30.742925 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:30.742873 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-cw6jh" podStartSLOduration=1.5399507049999999 podStartE2EDuration="3.742856614s" totalImagesPullingTime="2.202905909s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:08:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:08:30.741426762 +0000 UTC m=+385.576990592" watchObservedRunningTime="2026-07-13 07:08:30.742856614 +0000 UTC m=+385.578420446" Jul 13 07:08:30.957099 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:30.957071 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-cw6jh" Jul 13 07:08:31.017256 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:31.017213 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xsvb7\" (UniqueName: \"kubernetes.io/projected/9135e325-463e-4e79-99d4-e391b6fba716-kube-api-access-xsvb7\") pod \"9135e325-463e-4e79-99d4-e391b6fba716\" (UID: \"9135e325-463e-4e79-99d4-e391b6fba716\") " Jul 13 07:08:31.019467 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:31.019438 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9135e325-463e-4e79-99d4-e391b6fba716-kube-api-access-xsvb7" (OuterVolumeSpecName: "kube-api-access-xsvb7") pod "9135e325-463e-4e79-99d4-e391b6fba716" (UID: "9135e325-463e-4e79-99d4-e391b6fba716"). InnerVolumeSpecName "kube-api-access-xsvb7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:08:31.118086 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:31.117993 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xsvb7\" (UniqueName: \"kubernetes.io/projected/9135e325-463e-4e79-99d4-e391b6fba716-kube-api-access-xsvb7\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:08:31.725856 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:31.725825 2586 generic.go:356] "Generic (PLEG): container finished" podID="9135e325-463e-4e79-99d4-e391b6fba716" containerID="725a1305cecec9d3680d44f996f15224f75f3b9dddef1c8deaaebf8bb7438741" exitCode=0 Jul 13 07:08:31.726317 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:31.725868 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-cw6jh" Jul 13 07:08:31.726317 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:31.725905 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-cw6jh" event={"ID":"9135e325-463e-4e79-99d4-e391b6fba716","Type":"ContainerDied","Data":"725a1305cecec9d3680d44f996f15224f75f3b9dddef1c8deaaebf8bb7438741"} Jul 13 07:08:31.726317 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:31.725965 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-cw6jh" event={"ID":"9135e325-463e-4e79-99d4-e391b6fba716","Type":"ContainerDied","Data":"bffbb312ecc2c238a2bcb21af9b8491e62f98aebb767338dfcdf207d4dc9985c"} Jul 13 07:08:31.726317 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:31.725998 2586 scope.go:117] "RemoveContainer" containerID="725a1305cecec9d3680d44f996f15224f75f3b9dddef1c8deaaebf8bb7438741" Jul 13 07:08:31.735198 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:31.735180 2586 scope.go:117] "RemoveContainer" containerID="725a1305cecec9d3680d44f996f15224f75f3b9dddef1c8deaaebf8bb7438741" Jul 13 07:08:31.735478 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:08:31.735461 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"725a1305cecec9d3680d44f996f15224f75f3b9dddef1c8deaaebf8bb7438741\": container with ID starting with 725a1305cecec9d3680d44f996f15224f75f3b9dddef1c8deaaebf8bb7438741 not found: ID does not exist" containerID="725a1305cecec9d3680d44f996f15224f75f3b9dddef1c8deaaebf8bb7438741" Jul 13 07:08:31.735529 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:31.735489 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"725a1305cecec9d3680d44f996f15224f75f3b9dddef1c8deaaebf8bb7438741"} err="failed to get container status \"725a1305cecec9d3680d44f996f15224f75f3b9dddef1c8deaaebf8bb7438741\": rpc error: code = NotFound desc = could not find container \"725a1305cecec9d3680d44f996f15224f75f3b9dddef1c8deaaebf8bb7438741\": container with ID starting with 725a1305cecec9d3680d44f996f15224f75f3b9dddef1c8deaaebf8bb7438741 not found: ID does not exist" Jul 13 07:08:31.744916 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:31.744882 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-cw6jh"] Jul 13 07:08:31.748119 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:31.748096 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-cw6jh"] Jul 13 07:08:31.805222 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:31.805147 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9135e325-463e-4e79-99d4-e391b6fba716" path="/var/lib/kubelet/pods/9135e325-463e-4e79-99d4-e391b6fba716/volumes" Jul 13 07:08:38.735530 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:38.735473 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-bktks" Jul 13 07:08:38.736211 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:38.735554 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-bktks" Jul 13 07:08:38.758615 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:38.758587 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-bktks" Jul 13 07:08:38.783052 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:38.783016 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-bktks" Jul 13 07:08:42.829127 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:42.829093 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rbp4k"] Jul 13 07:08:42.829517 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:42.829418 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9135e325-463e-4e79-99d4-e391b6fba716" containerName="registry-server" Jul 13 07:08:42.829517 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:42.829428 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="9135e325-463e-4e79-99d4-e391b6fba716" containerName="registry-server" Jul 13 07:08:42.829517 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:42.829493 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="9135e325-463e-4e79-99d4-e391b6fba716" containerName="registry-server" Jul 13 07:08:42.834115 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:42.834094 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rbp4k" Jul 13 07:08:42.835984 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:42.835953 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-n6874\"" Jul 13 07:08:42.839492 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:42.839467 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rbp4k"] Jul 13 07:08:42.917712 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:42.917640 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/faa75440-9d18-4c09-b910-0f3d3c73c9d1-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rbp4k\" (UID: \"faa75440-9d18-4c09-b910-0f3d3c73c9d1\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rbp4k" Jul 13 07:08:42.917935 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:42.917731 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2mwf\" (UniqueName: \"kubernetes.io/projected/faa75440-9d18-4c09-b910-0f3d3c73c9d1-kube-api-access-g2mwf\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rbp4k\" (UID: \"faa75440-9d18-4c09-b910-0f3d3c73c9d1\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rbp4k" Jul 13 07:08:42.917935 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:42.917779 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/faa75440-9d18-4c09-b910-0f3d3c73c9d1-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rbp4k\" (UID: \"faa75440-9d18-4c09-b910-0f3d3c73c9d1\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rbp4k" Jul 13 07:08:43.018932 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:43.018886 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/faa75440-9d18-4c09-b910-0f3d3c73c9d1-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rbp4k\" (UID: \"faa75440-9d18-4c09-b910-0f3d3c73c9d1\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rbp4k" Jul 13 07:08:43.018932 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:43.018934 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2mwf\" (UniqueName: \"kubernetes.io/projected/faa75440-9d18-4c09-b910-0f3d3c73c9d1-kube-api-access-g2mwf\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rbp4k\" (UID: \"faa75440-9d18-4c09-b910-0f3d3c73c9d1\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rbp4k" Jul 13 07:08:43.019208 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:43.018968 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/faa75440-9d18-4c09-b910-0f3d3c73c9d1-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rbp4k\" (UID: \"faa75440-9d18-4c09-b910-0f3d3c73c9d1\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rbp4k" Jul 13 07:08:43.019366 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:43.019340 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/faa75440-9d18-4c09-b910-0f3d3c73c9d1-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rbp4k\" (UID: \"faa75440-9d18-4c09-b910-0f3d3c73c9d1\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rbp4k" Jul 13 07:08:43.019366 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:43.019358 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/faa75440-9d18-4c09-b910-0f3d3c73c9d1-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rbp4k\" (UID: \"faa75440-9d18-4c09-b910-0f3d3c73c9d1\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rbp4k" Jul 13 07:08:43.026936 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:43.026910 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2mwf\" (UniqueName: \"kubernetes.io/projected/faa75440-9d18-4c09-b910-0f3d3c73c9d1-kube-api-access-g2mwf\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rbp4k\" (UID: \"faa75440-9d18-4c09-b910-0f3d3c73c9d1\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rbp4k" Jul 13 07:08:43.144564 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:43.144472 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rbp4k" Jul 13 07:08:43.280962 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:43.280933 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rbp4k"] Jul 13 07:08:43.282590 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:08:43.282560 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfaa75440_9d18_4c09_b910_0f3d3c73c9d1.slice/crio-4b09bd3ddf08c7d883b7a175c6a8bdada4f64a574b6165b51d873a1d08c77fde WatchSource:0}: Error finding container 4b09bd3ddf08c7d883b7a175c6a8bdada4f64a574b6165b51d873a1d08c77fde: Status 404 returned error can't find the container with id 4b09bd3ddf08c7d883b7a175c6a8bdada4f64a574b6165b51d873a1d08c77fde Jul 13 07:08:43.422967 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:43.422932 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fvhbn"] Jul 13 07:08:43.426300 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:43.426280 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fvhbn" Jul 13 07:08:43.435858 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:43.435836 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fvhbn"] Jul 13 07:08:43.522532 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:43.522489 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fvhbn\" (UID: \"c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fvhbn" Jul 13 07:08:43.522750 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:43.522577 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5wg7\" (UniqueName: \"kubernetes.io/projected/c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b-kube-api-access-h5wg7\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fvhbn\" (UID: \"c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fvhbn" Jul 13 07:08:43.522750 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:43.522619 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fvhbn\" (UID: \"c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fvhbn" Jul 13 07:08:43.623324 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:43.623285 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5wg7\" (UniqueName: \"kubernetes.io/projected/c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b-kube-api-access-h5wg7\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fvhbn\" (UID: \"c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fvhbn" Jul 13 07:08:43.623505 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:43.623343 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fvhbn\" (UID: \"c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fvhbn" Jul 13 07:08:43.623505 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:43.623387 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fvhbn\" (UID: \"c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fvhbn" Jul 13 07:08:43.623812 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:43.623788 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fvhbn\" (UID: \"c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fvhbn" Jul 13 07:08:43.623928 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:43.623819 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fvhbn\" (UID: \"c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fvhbn" Jul 13 07:08:43.630939 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:43.630914 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5wg7\" (UniqueName: \"kubernetes.io/projected/c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b-kube-api-access-h5wg7\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fvhbn\" (UID: \"c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fvhbn" Jul 13 07:08:43.746661 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:43.746620 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fvhbn" Jul 13 07:08:43.787955 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:43.787911 2586 generic.go:356] "Generic (PLEG): container finished" podID="faa75440-9d18-4c09-b910-0f3d3c73c9d1" containerID="d770009f3f60fbc9112aef3424c8a1f047e12afda410aa9a003b568f8762b44a" exitCode=0 Jul 13 07:08:43.787955 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:43.787948 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rbp4k" event={"ID":"faa75440-9d18-4c09-b910-0f3d3c73c9d1","Type":"ContainerDied","Data":"d770009f3f60fbc9112aef3424c8a1f047e12afda410aa9a003b568f8762b44a"} Jul 13 07:08:43.788196 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:43.787996 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rbp4k" event={"ID":"faa75440-9d18-4c09-b910-0f3d3c73c9d1","Type":"ContainerStarted","Data":"4b09bd3ddf08c7d883b7a175c6a8bdada4f64a574b6165b51d873a1d08c77fde"} Jul 13 07:08:43.833075 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:43.833044 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p4t6d"] Jul 13 07:08:43.838344 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:43.838321 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p4t6d" Jul 13 07:08:43.842335 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:43.842309 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p4t6d"] Jul 13 07:08:43.883264 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:43.883236 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fvhbn"] Jul 13 07:08:43.885222 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:08:43.885194 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2b47d1b_8bc4_4d22_ab1c_bfcf6effdb4b.slice/crio-cafd61f11da0321e0b688ffa16681681c8a11a25530c4e2b266cb06d6abf258b WatchSource:0}: Error finding container cafd61f11da0321e0b688ffa16681681c8a11a25530c4e2b266cb06d6abf258b: Status 404 returned error can't find the container with id cafd61f11da0321e0b688ffa16681681c8a11a25530c4e2b266cb06d6abf258b Jul 13 07:08:43.926305 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:43.926278 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/adace2a3-d312-41d9-b47c-bb4a63aa0663-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p4t6d\" (UID: \"adace2a3-d312-41d9-b47c-bb4a63aa0663\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p4t6d" Jul 13 07:08:43.926418 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:43.926333 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/adace2a3-d312-41d9-b47c-bb4a63aa0663-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p4t6d\" (UID: \"adace2a3-d312-41d9-b47c-bb4a63aa0663\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p4t6d" Jul 13 07:08:43.926418 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:43.926393 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tfhv\" (UniqueName: \"kubernetes.io/projected/adace2a3-d312-41d9-b47c-bb4a63aa0663-kube-api-access-7tfhv\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p4t6d\" (UID: \"adace2a3-d312-41d9-b47c-bb4a63aa0663\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p4t6d" Jul 13 07:08:44.027103 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:44.027033 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tfhv\" (UniqueName: \"kubernetes.io/projected/adace2a3-d312-41d9-b47c-bb4a63aa0663-kube-api-access-7tfhv\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p4t6d\" (UID: \"adace2a3-d312-41d9-b47c-bb4a63aa0663\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p4t6d" Jul 13 07:08:44.027210 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:44.027106 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/adace2a3-d312-41d9-b47c-bb4a63aa0663-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p4t6d\" (UID: \"adace2a3-d312-41d9-b47c-bb4a63aa0663\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p4t6d" Jul 13 07:08:44.027210 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:44.027173 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/adace2a3-d312-41d9-b47c-bb4a63aa0663-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p4t6d\" (UID: \"adace2a3-d312-41d9-b47c-bb4a63aa0663\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p4t6d" Jul 13 07:08:44.027448 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:44.027430 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/adace2a3-d312-41d9-b47c-bb4a63aa0663-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p4t6d\" (UID: \"adace2a3-d312-41d9-b47c-bb4a63aa0663\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p4t6d" Jul 13 07:08:44.027491 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:44.027478 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/adace2a3-d312-41d9-b47c-bb4a63aa0663-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p4t6d\" (UID: \"adace2a3-d312-41d9-b47c-bb4a63aa0663\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p4t6d" Jul 13 07:08:44.034935 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:44.034909 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tfhv\" (UniqueName: \"kubernetes.io/projected/adace2a3-d312-41d9-b47c-bb4a63aa0663-kube-api-access-7tfhv\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p4t6d\" (UID: \"adace2a3-d312-41d9-b47c-bb4a63aa0663\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p4t6d" Jul 13 07:08:44.149993 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:44.149956 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p4t6d" Jul 13 07:08:44.227207 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:44.227174 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738xbs6"] Jul 13 07:08:44.234916 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:44.233586 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738xbs6" Jul 13 07:08:44.242330 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:44.241967 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738xbs6"] Jul 13 07:08:44.286560 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:44.286534 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p4t6d"] Jul 13 07:08:44.289190 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:08:44.289163 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadace2a3_d312_41d9_b47c_bb4a63aa0663.slice/crio-51379b195ce12982f8327046fa478989040d7704b9e4e045e343b84a03512893 WatchSource:0}: Error finding container 51379b195ce12982f8327046fa478989040d7704b9e4e045e343b84a03512893: Status 404 returned error can't find the container with id 51379b195ce12982f8327046fa478989040d7704b9e4e045e343b84a03512893 Jul 13 07:08:44.329767 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:44.329740 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738xbs6\" (UID: \"e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738xbs6" Jul 13 07:08:44.329868 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:44.329793 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sl5px\" (UniqueName: \"kubernetes.io/projected/e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e-kube-api-access-sl5px\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738xbs6\" (UID: \"e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738xbs6" Jul 13 07:08:44.329868 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:44.329860 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738xbs6\" (UID: \"e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738xbs6" Jul 13 07:08:44.430458 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:44.430431 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sl5px\" (UniqueName: \"kubernetes.io/projected/e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e-kube-api-access-sl5px\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738xbs6\" (UID: \"e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738xbs6" Jul 13 07:08:44.430598 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:44.430494 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738xbs6\" (UID: \"e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738xbs6" Jul 13 07:08:44.430598 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:44.430535 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738xbs6\" (UID: \"e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738xbs6" Jul 13 07:08:44.430867 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:44.430849 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738xbs6\" (UID: \"e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738xbs6" Jul 13 07:08:44.430916 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:44.430886 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738xbs6\" (UID: \"e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738xbs6" Jul 13 07:08:44.437873 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:44.437854 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sl5px\" (UniqueName: \"kubernetes.io/projected/e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e-kube-api-access-sl5px\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738xbs6\" (UID: \"e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738xbs6" Jul 13 07:08:44.549941 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:44.549857 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738xbs6" Jul 13 07:08:44.674721 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:44.674695 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738xbs6"] Jul 13 07:08:44.676473 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:08:44.676452 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode62a8b69_94ba_4bb5_a9da_6fa0c7f2173e.slice/crio-1e62f7d1cfb99c8df47b3f7a9833c6daa51735af2a33e1b94a0abe438abe6815 WatchSource:0}: Error finding container 1e62f7d1cfb99c8df47b3f7a9833c6daa51735af2a33e1b94a0abe438abe6815: Status 404 returned error can't find the container with id 1e62f7d1cfb99c8df47b3f7a9833c6daa51735af2a33e1b94a0abe438abe6815 Jul 13 07:08:44.794071 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:44.794033 2586 generic.go:356] "Generic (PLEG): container finished" podID="e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e" containerID="9b73bc3a4d6d73e33de5d4ddeab60beaf0183db863aedbe42eb96a5df0c989bc" exitCode=0 Jul 13 07:08:44.794198 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:44.794110 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738xbs6" event={"ID":"e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e","Type":"ContainerDied","Data":"9b73bc3a4d6d73e33de5d4ddeab60beaf0183db863aedbe42eb96a5df0c989bc"} Jul 13 07:08:44.794198 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:44.794156 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738xbs6" event={"ID":"e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e","Type":"ContainerStarted","Data":"1e62f7d1cfb99c8df47b3f7a9833c6daa51735af2a33e1b94a0abe438abe6815"} Jul 13 07:08:44.796150 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:44.796130 2586 generic.go:356] "Generic (PLEG): container finished" podID="adace2a3-d312-41d9-b47c-bb4a63aa0663" containerID="1b528e300569767751ac97148388e96fe8b56b5df2da988684a1e76f07ec369a" exitCode=0 Jul 13 07:08:44.796256 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:44.796210 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p4t6d" event={"ID":"adace2a3-d312-41d9-b47c-bb4a63aa0663","Type":"ContainerDied","Data":"1b528e300569767751ac97148388e96fe8b56b5df2da988684a1e76f07ec369a"} Jul 13 07:08:44.796316 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:44.796258 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p4t6d" event={"ID":"adace2a3-d312-41d9-b47c-bb4a63aa0663","Type":"ContainerStarted","Data":"51379b195ce12982f8327046fa478989040d7704b9e4e045e343b84a03512893"} Jul 13 07:08:44.797824 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:44.797801 2586 generic.go:356] "Generic (PLEG): container finished" podID="c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b" containerID="d5776250c0af046bf6e9eedb6aeac62d09650d4d1bbd10e6281e43045c67c74e" exitCode=0 Jul 13 07:08:44.797926 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:44.797833 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fvhbn" event={"ID":"c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b","Type":"ContainerDied","Data":"d5776250c0af046bf6e9eedb6aeac62d09650d4d1bbd10e6281e43045c67c74e"} Jul 13 07:08:44.797926 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:44.797865 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fvhbn" event={"ID":"c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b","Type":"ContainerStarted","Data":"cafd61f11da0321e0b688ffa16681681c8a11a25530c4e2b266cb06d6abf258b"} Jul 13 07:08:46.810489 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:46.810455 2586 generic.go:356] "Generic (PLEG): container finished" podID="adace2a3-d312-41d9-b47c-bb4a63aa0663" containerID="0c0a5fcacebbf96d96cd6b8e2c26f9e25067c6c830090e678d3cab78af93b89f" exitCode=0 Jul 13 07:08:46.810920 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:46.810531 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p4t6d" event={"ID":"adace2a3-d312-41d9-b47c-bb4a63aa0663","Type":"ContainerDied","Data":"0c0a5fcacebbf96d96cd6b8e2c26f9e25067c6c830090e678d3cab78af93b89f"} Jul 13 07:08:46.812351 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:46.812330 2586 generic.go:356] "Generic (PLEG): container finished" podID="faa75440-9d18-4c09-b910-0f3d3c73c9d1" containerID="c1740b4fd8223629e3df61b00ad82e46f89825f7d46f399fae35526b4a350234" exitCode=0 Jul 13 07:08:46.812440 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:46.812415 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rbp4k" event={"ID":"faa75440-9d18-4c09-b910-0f3d3c73c9d1","Type":"ContainerDied","Data":"c1740b4fd8223629e3df61b00ad82e46f89825f7d46f399fae35526b4a350234"} Jul 13 07:08:46.814307 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:46.814282 2586 generic.go:356] "Generic (PLEG): container finished" podID="c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b" containerID="1e79c6da9b2cf4322e33965c63831c1e4a1fb2b04436ed7a6a99608429efded0" exitCode=0 Jul 13 07:08:46.814398 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:46.814356 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fvhbn" event={"ID":"c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b","Type":"ContainerDied","Data":"1e79c6da9b2cf4322e33965c63831c1e4a1fb2b04436ed7a6a99608429efded0"} Jul 13 07:08:46.816306 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:46.816285 2586 generic.go:356] "Generic (PLEG): container finished" podID="e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e" containerID="62b4dc15c4d43452174c55c19f97112361cf5baae928f53e055088cb3a688baf" exitCode=0 Jul 13 07:08:46.816611 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:46.816326 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738xbs6" event={"ID":"e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e","Type":"ContainerDied","Data":"62b4dc15c4d43452174c55c19f97112361cf5baae928f53e055088cb3a688baf"} Jul 13 07:08:47.823883 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:47.823847 2586 generic.go:356] "Generic (PLEG): container finished" podID="adace2a3-d312-41d9-b47c-bb4a63aa0663" containerID="f78c160571a7c0fe269dcd3c58efbca4d5df3a84f039f08bb386afb32b3cf291" exitCode=0 Jul 13 07:08:47.824304 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:47.823910 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p4t6d" event={"ID":"adace2a3-d312-41d9-b47c-bb4a63aa0663","Type":"ContainerDied","Data":"f78c160571a7c0fe269dcd3c58efbca4d5df3a84f039f08bb386afb32b3cf291"} Jul 13 07:08:47.826025 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:47.825997 2586 generic.go:356] "Generic (PLEG): container finished" podID="faa75440-9d18-4c09-b910-0f3d3c73c9d1" containerID="d6f09ca2782106c25495e1c5286436413495d375f70c082d09446ddd2147f2a5" exitCode=0 Jul 13 07:08:47.826134 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:47.826062 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rbp4k" event={"ID":"faa75440-9d18-4c09-b910-0f3d3c73c9d1","Type":"ContainerDied","Data":"d6f09ca2782106c25495e1c5286436413495d375f70c082d09446ddd2147f2a5"} Jul 13 07:08:47.828213 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:47.828188 2586 generic.go:356] "Generic (PLEG): container finished" podID="c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b" containerID="63cfdc441fd24b15ce899c282428f074665a7dc48c5fd485cd0f8ec19b8bed36" exitCode=0 Jul 13 07:08:47.828323 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:47.828224 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fvhbn" event={"ID":"c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b","Type":"ContainerDied","Data":"63cfdc441fd24b15ce899c282428f074665a7dc48c5fd485cd0f8ec19b8bed36"} Jul 13 07:08:47.830266 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:47.830241 2586 generic.go:356] "Generic (PLEG): container finished" podID="e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e" containerID="0e4842e1aaff82c52c43020d648fcae918a6bb804789ec66ebaa38cce5599a0e" exitCode=0 Jul 13 07:08:47.830371 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:47.830294 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738xbs6" event={"ID":"e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e","Type":"ContainerDied","Data":"0e4842e1aaff82c52c43020d648fcae918a6bb804789ec66ebaa38cce5599a0e"} Jul 13 07:08:48.985887 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:48.985852 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p4t6d" Jul 13 07:08:49.039344 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.039309 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rbp4k" Jul 13 07:08:49.075537 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.075506 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fvhbn" Jul 13 07:08:49.077287 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.077262 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/adace2a3-d312-41d9-b47c-bb4a63aa0663-bundle\") pod \"adace2a3-d312-41d9-b47c-bb4a63aa0663\" (UID: \"adace2a3-d312-41d9-b47c-bb4a63aa0663\") " Jul 13 07:08:49.077406 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.077391 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/adace2a3-d312-41d9-b47c-bb4a63aa0663-util\") pod \"adace2a3-d312-41d9-b47c-bb4a63aa0663\" (UID: \"adace2a3-d312-41d9-b47c-bb4a63aa0663\") " Jul 13 07:08:49.077458 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.077444 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7tfhv\" (UniqueName: \"kubernetes.io/projected/adace2a3-d312-41d9-b47c-bb4a63aa0663-kube-api-access-7tfhv\") pod \"adace2a3-d312-41d9-b47c-bb4a63aa0663\" (UID: \"adace2a3-d312-41d9-b47c-bb4a63aa0663\") " Jul 13 07:08:49.077854 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.077828 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/adace2a3-d312-41d9-b47c-bb4a63aa0663-bundle" (OuterVolumeSpecName: "bundle") pod "adace2a3-d312-41d9-b47c-bb4a63aa0663" (UID: "adace2a3-d312-41d9-b47c-bb4a63aa0663"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:08:49.078875 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.078844 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738xbs6" Jul 13 07:08:49.079546 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.079524 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/adace2a3-d312-41d9-b47c-bb4a63aa0663-kube-api-access-7tfhv" (OuterVolumeSpecName: "kube-api-access-7tfhv") pod "adace2a3-d312-41d9-b47c-bb4a63aa0663" (UID: "adace2a3-d312-41d9-b47c-bb4a63aa0663"). InnerVolumeSpecName "kube-api-access-7tfhv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:08:49.083071 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.083044 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/adace2a3-d312-41d9-b47c-bb4a63aa0663-util" (OuterVolumeSpecName: "util") pod "adace2a3-d312-41d9-b47c-bb4a63aa0663" (UID: "adace2a3-d312-41d9-b47c-bb4a63aa0663"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:08:49.178862 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.178836 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/faa75440-9d18-4c09-b910-0f3d3c73c9d1-util\") pod \"faa75440-9d18-4c09-b910-0f3d3c73c9d1\" (UID: \"faa75440-9d18-4c09-b910-0f3d3c73c9d1\") " Jul 13 07:08:49.179026 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.178877 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b-util\") pod \"c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b\" (UID: \"c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b\") " Jul 13 07:08:49.179026 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.178900 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e-bundle\") pod \"e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e\" (UID: \"e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e\") " Jul 13 07:08:49.179026 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.178925 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sl5px\" (UniqueName: \"kubernetes.io/projected/e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e-kube-api-access-sl5px\") pod \"e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e\" (UID: \"e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e\") " Jul 13 07:08:49.179026 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.178940 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e-util\") pod \"e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e\" (UID: \"e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e\") " Jul 13 07:08:49.179026 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.178965 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b-bundle\") pod \"c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b\" (UID: \"c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b\") " Jul 13 07:08:49.179294 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.179044 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/faa75440-9d18-4c09-b910-0f3d3c73c9d1-bundle\") pod \"faa75440-9d18-4c09-b910-0f3d3c73c9d1\" (UID: \"faa75440-9d18-4c09-b910-0f3d3c73c9d1\") " Jul 13 07:08:49.179294 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.179072 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h5wg7\" (UniqueName: \"kubernetes.io/projected/c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b-kube-api-access-h5wg7\") pod \"c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b\" (UID: \"c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b\") " Jul 13 07:08:49.179294 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.179118 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g2mwf\" (UniqueName: \"kubernetes.io/projected/faa75440-9d18-4c09-b910-0f3d3c73c9d1-kube-api-access-g2mwf\") pod \"faa75440-9d18-4c09-b910-0f3d3c73c9d1\" (UID: \"faa75440-9d18-4c09-b910-0f3d3c73c9d1\") " Jul 13 07:08:49.181347 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.180219 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8d967f6bf-47qmq"] Jul 13 07:08:49.181347 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.180473 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b-bundle" (OuterVolumeSpecName: "bundle") pod "c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b" (UID: "c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:08:49.181347 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.180727 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="adace2a3-d312-41d9-b47c-bb4a63aa0663" containerName="util" Jul 13 07:08:49.181347 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.180747 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="adace2a3-d312-41d9-b47c-bb4a63aa0663" containerName="util" Jul 13 07:08:49.181347 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.180765 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e" containerName="util" Jul 13 07:08:49.181347 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.180774 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e" containerName="util" Jul 13 07:08:49.181347 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.180784 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b" containerName="extract" Jul 13 07:08:49.181347 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.180793 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b" containerName="extract" Jul 13 07:08:49.181347 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.180803 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="adace2a3-d312-41d9-b47c-bb4a63aa0663" containerName="extract" Jul 13 07:08:49.183302 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.182024 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="adace2a3-d312-41d9-b47c-bb4a63aa0663" containerName="extract" Jul 13 07:08:49.183302 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.182056 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e" containerName="pull" Jul 13 07:08:49.183302 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.182067 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e" containerName="pull" Jul 13 07:08:49.183302 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.182081 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b" containerName="pull" Jul 13 07:08:49.183302 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.182089 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b" containerName="pull" Jul 13 07:08:49.183302 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.182103 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b" containerName="util" Jul 13 07:08:49.183302 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.182099 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e-kube-api-access-sl5px" (OuterVolumeSpecName: "kube-api-access-sl5px") pod "e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e" (UID: "e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e"). InnerVolumeSpecName "kube-api-access-sl5px". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:08:49.183302 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.182111 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b" containerName="util" Jul 13 07:08:49.183302 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.182144 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e" containerName="extract" Jul 13 07:08:49.183302 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.182156 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e" containerName="extract" Jul 13 07:08:49.183302 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.182170 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="faa75440-9d18-4c09-b910-0f3d3c73c9d1" containerName="pull" Jul 13 07:08:49.183302 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.182178 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="faa75440-9d18-4c09-b910-0f3d3c73c9d1" containerName="pull" Jul 13 07:08:49.183302 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.182189 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="faa75440-9d18-4c09-b910-0f3d3c73c9d1" containerName="extract" Jul 13 07:08:49.183302 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.182196 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="faa75440-9d18-4c09-b910-0f3d3c73c9d1" containerName="extract" Jul 13 07:08:49.183302 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.182211 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="adace2a3-d312-41d9-b47c-bb4a63aa0663" containerName="pull" Jul 13 07:08:49.183302 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.182219 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="adace2a3-d312-41d9-b47c-bb4a63aa0663" containerName="pull" Jul 13 07:08:49.183302 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.182231 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="faa75440-9d18-4c09-b910-0f3d3c73c9d1" containerName="util" Jul 13 07:08:49.183302 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.182239 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="faa75440-9d18-4c09-b910-0f3d3c73c9d1" containerName="util" Jul 13 07:08:49.183302 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.182388 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e" containerName="extract" Jul 13 07:08:49.183302 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.182401 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b" containerName="extract" Jul 13 07:08:49.183302 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.182410 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="faa75440-9d18-4c09-b910-0f3d3c73c9d1" containerName="extract" Jul 13 07:08:49.183302 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.182423 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="adace2a3-d312-41d9-b47c-bb4a63aa0663" containerName="extract" Jul 13 07:08:49.183302 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.182843 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e-bundle" (OuterVolumeSpecName: "bundle") pod "e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e" (UID: "e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:08:49.183302 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.183046 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/faa75440-9d18-4c09-b910-0f3d3c73c9d1-bundle" (OuterVolumeSpecName: "bundle") pod "faa75440-9d18-4c09-b910-0f3d3c73c9d1" (UID: "faa75440-9d18-4c09-b910-0f3d3c73c9d1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:08:49.183302 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.183171 2586 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/faa75440-9d18-4c09-b910-0f3d3c73c9d1-bundle\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:08:49.183302 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.183189 2586 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/adace2a3-d312-41d9-b47c-bb4a63aa0663-util\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:08:49.183302 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.183202 2586 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e-bundle\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:08:49.183302 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.183216 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7tfhv\" (UniqueName: \"kubernetes.io/projected/adace2a3-d312-41d9-b47c-bb4a63aa0663-kube-api-access-7tfhv\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:08:49.183302 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.183231 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sl5px\" (UniqueName: \"kubernetes.io/projected/e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e-kube-api-access-sl5px\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:08:49.183302 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.183245 2586 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b-bundle\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:08:49.184939 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.183257 2586 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/adace2a3-d312-41d9-b47c-bb4a63aa0663-bundle\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:08:49.184939 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.184740 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b-kube-api-access-h5wg7" (OuterVolumeSpecName: "kube-api-access-h5wg7") pod "c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b" (UID: "c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b"). InnerVolumeSpecName "kube-api-access-h5wg7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:08:49.185603 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.185574 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/faa75440-9d18-4c09-b910-0f3d3c73c9d1-kube-api-access-g2mwf" (OuterVolumeSpecName: "kube-api-access-g2mwf") pod "faa75440-9d18-4c09-b910-0f3d3c73c9d1" (UID: "faa75440-9d18-4c09-b910-0f3d3c73c9d1"). InnerVolumeSpecName "kube-api-access-g2mwf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:08:49.187641 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.187614 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/faa75440-9d18-4c09-b910-0f3d3c73c9d1-util" (OuterVolumeSpecName: "util") pod "faa75440-9d18-4c09-b910-0f3d3c73c9d1" (UID: "faa75440-9d18-4c09-b910-0f3d3c73c9d1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:08:49.188109 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.188092 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8d967f6bf-47qmq" Jul 13 07:08:49.188486 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.188403 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b-util" (OuterVolumeSpecName: "util") pod "c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b" (UID: "c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:08:49.188901 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.188863 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e-util" (OuterVolumeSpecName: "util") pod "e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e" (UID: "e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:08:49.192982 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.192950 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8d967f6bf-47qmq"] Jul 13 07:08:49.284222 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.284179 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvvfh\" (UniqueName: \"kubernetes.io/projected/4f410a58-dbcb-4901-943c-eb158ae798a9-kube-api-access-tvvfh\") pod \"console-8d967f6bf-47qmq\" (UID: \"4f410a58-dbcb-4901-943c-eb158ae798a9\") " pod="openshift-console/console-8d967f6bf-47qmq" Jul 13 07:08:49.284222 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.284219 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4f410a58-dbcb-4901-943c-eb158ae798a9-oauth-serving-cert\") pod \"console-8d967f6bf-47qmq\" (UID: \"4f410a58-dbcb-4901-943c-eb158ae798a9\") " pod="openshift-console/console-8d967f6bf-47qmq" Jul 13 07:08:49.284397 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.284247 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4f410a58-dbcb-4901-943c-eb158ae798a9-console-oauth-config\") pod \"console-8d967f6bf-47qmq\" (UID: \"4f410a58-dbcb-4901-943c-eb158ae798a9\") " pod="openshift-console/console-8d967f6bf-47qmq" Jul 13 07:08:49.284397 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.284329 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4f410a58-dbcb-4901-943c-eb158ae798a9-service-ca\") pod \"console-8d967f6bf-47qmq\" (UID: \"4f410a58-dbcb-4901-943c-eb158ae798a9\") " pod="openshift-console/console-8d967f6bf-47qmq" Jul 13 07:08:49.284397 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.284388 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4f410a58-dbcb-4901-943c-eb158ae798a9-console-config\") pod \"console-8d967f6bf-47qmq\" (UID: \"4f410a58-dbcb-4901-943c-eb158ae798a9\") " pod="openshift-console/console-8d967f6bf-47qmq" Jul 13 07:08:49.284510 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.284449 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4f410a58-dbcb-4901-943c-eb158ae798a9-console-serving-cert\") pod \"console-8d967f6bf-47qmq\" (UID: \"4f410a58-dbcb-4901-943c-eb158ae798a9\") " pod="openshift-console/console-8d967f6bf-47qmq" Jul 13 07:08:49.284510 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.284478 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f410a58-dbcb-4901-943c-eb158ae798a9-trusted-ca-bundle\") pod \"console-8d967f6bf-47qmq\" (UID: \"4f410a58-dbcb-4901-943c-eb158ae798a9\") " pod="openshift-console/console-8d967f6bf-47qmq" Jul 13 07:08:49.284570 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.284528 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h5wg7\" (UniqueName: \"kubernetes.io/projected/c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b-kube-api-access-h5wg7\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:08:49.284570 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.284540 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g2mwf\" (UniqueName: \"kubernetes.io/projected/faa75440-9d18-4c09-b910-0f3d3c73c9d1-kube-api-access-g2mwf\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:08:49.284570 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.284549 2586 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/faa75440-9d18-4c09-b910-0f3d3c73c9d1-util\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:08:49.284570 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.284558 2586 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b-util\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:08:49.284570 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.284565 2586 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e-util\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:08:49.385446 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.385411 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4f410a58-dbcb-4901-943c-eb158ae798a9-console-config\") pod \"console-8d967f6bf-47qmq\" (UID: \"4f410a58-dbcb-4901-943c-eb158ae798a9\") " pod="openshift-console/console-8d967f6bf-47qmq" Jul 13 07:08:49.385563 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.385471 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4f410a58-dbcb-4901-943c-eb158ae798a9-console-serving-cert\") pod \"console-8d967f6bf-47qmq\" (UID: \"4f410a58-dbcb-4901-943c-eb158ae798a9\") " pod="openshift-console/console-8d967f6bf-47qmq" Jul 13 07:08:49.385563 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.385497 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f410a58-dbcb-4901-943c-eb158ae798a9-trusted-ca-bundle\") pod \"console-8d967f6bf-47qmq\" (UID: \"4f410a58-dbcb-4901-943c-eb158ae798a9\") " pod="openshift-console/console-8d967f6bf-47qmq" Jul 13 07:08:49.385563 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.385520 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvvfh\" (UniqueName: \"kubernetes.io/projected/4f410a58-dbcb-4901-943c-eb158ae798a9-kube-api-access-tvvfh\") pod \"console-8d967f6bf-47qmq\" (UID: \"4f410a58-dbcb-4901-943c-eb158ae798a9\") " pod="openshift-console/console-8d967f6bf-47qmq" Jul 13 07:08:49.385563 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.385536 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4f410a58-dbcb-4901-943c-eb158ae798a9-oauth-serving-cert\") pod \"console-8d967f6bf-47qmq\" (UID: \"4f410a58-dbcb-4901-943c-eb158ae798a9\") " pod="openshift-console/console-8d967f6bf-47qmq" Jul 13 07:08:49.385801 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.385574 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4f410a58-dbcb-4901-943c-eb158ae798a9-console-oauth-config\") pod \"console-8d967f6bf-47qmq\" (UID: \"4f410a58-dbcb-4901-943c-eb158ae798a9\") " pod="openshift-console/console-8d967f6bf-47qmq" Jul 13 07:08:49.385801 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.385643 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4f410a58-dbcb-4901-943c-eb158ae798a9-service-ca\") pod \"console-8d967f6bf-47qmq\" (UID: \"4f410a58-dbcb-4901-943c-eb158ae798a9\") " pod="openshift-console/console-8d967f6bf-47qmq" Jul 13 07:08:49.386282 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.386254 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4f410a58-dbcb-4901-943c-eb158ae798a9-oauth-serving-cert\") pod \"console-8d967f6bf-47qmq\" (UID: \"4f410a58-dbcb-4901-943c-eb158ae798a9\") " pod="openshift-console/console-8d967f6bf-47qmq" Jul 13 07:08:49.386377 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.386302 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4f410a58-dbcb-4901-943c-eb158ae798a9-console-config\") pod \"console-8d967f6bf-47qmq\" (UID: \"4f410a58-dbcb-4901-943c-eb158ae798a9\") " pod="openshift-console/console-8d967f6bf-47qmq" Jul 13 07:08:49.386377 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.386302 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4f410a58-dbcb-4901-943c-eb158ae798a9-service-ca\") pod \"console-8d967f6bf-47qmq\" (UID: \"4f410a58-dbcb-4901-943c-eb158ae798a9\") " pod="openshift-console/console-8d967f6bf-47qmq" Jul 13 07:08:49.386448 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.386418 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f410a58-dbcb-4901-943c-eb158ae798a9-trusted-ca-bundle\") pod \"console-8d967f6bf-47qmq\" (UID: \"4f410a58-dbcb-4901-943c-eb158ae798a9\") " pod="openshift-console/console-8d967f6bf-47qmq" Jul 13 07:08:49.387971 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.387946 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4f410a58-dbcb-4901-943c-eb158ae798a9-console-serving-cert\") pod \"console-8d967f6bf-47qmq\" (UID: \"4f410a58-dbcb-4901-943c-eb158ae798a9\") " pod="openshift-console/console-8d967f6bf-47qmq" Jul 13 07:08:49.388084 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.388065 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4f410a58-dbcb-4901-943c-eb158ae798a9-console-oauth-config\") pod \"console-8d967f6bf-47qmq\" (UID: \"4f410a58-dbcb-4901-943c-eb158ae798a9\") " pod="openshift-console/console-8d967f6bf-47qmq" Jul 13 07:08:49.394075 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.394054 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvvfh\" (UniqueName: \"kubernetes.io/projected/4f410a58-dbcb-4901-943c-eb158ae798a9-kube-api-access-tvvfh\") pod \"console-8d967f6bf-47qmq\" (UID: \"4f410a58-dbcb-4901-943c-eb158ae798a9\") " pod="openshift-console/console-8d967f6bf-47qmq" Jul 13 07:08:49.501448 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.501354 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8d967f6bf-47qmq" Jul 13 07:08:49.651989 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.651962 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8d967f6bf-47qmq"] Jul 13 07:08:49.653801 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:08:49.653777 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f410a58_dbcb_4901_943c_eb158ae798a9.slice/crio-a3d40e2c4563e29e08b7f249a7a222cf84c27e8b9e7678f67900600298ca4680 WatchSource:0}: Error finding container a3d40e2c4563e29e08b7f249a7a222cf84c27e8b9e7678f67900600298ca4680: Status 404 returned error can't find the container with id a3d40e2c4563e29e08b7f249a7a222cf84c27e8b9e7678f67900600298ca4680 Jul 13 07:08:49.843595 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.843565 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738xbs6" Jul 13 07:08:49.843805 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.843596 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738xbs6" event={"ID":"e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e","Type":"ContainerDied","Data":"1e62f7d1cfb99c8df47b3f7a9833c6daa51735af2a33e1b94a0abe438abe6815"} Jul 13 07:08:49.843805 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.843673 2586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1e62f7d1cfb99c8df47b3f7a9833c6daa51735af2a33e1b94a0abe438abe6815" Jul 13 07:08:49.845752 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.845439 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p4t6d" Jul 13 07:08:49.845866 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.845452 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p4t6d" event={"ID":"adace2a3-d312-41d9-b47c-bb4a63aa0663","Type":"ContainerDied","Data":"51379b195ce12982f8327046fa478989040d7704b9e4e045e343b84a03512893"} Jul 13 07:08:49.845866 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.845830 2586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="51379b195ce12982f8327046fa478989040d7704b9e4e045e343b84a03512893" Jul 13 07:08:49.847582 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.847536 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rbp4k" event={"ID":"faa75440-9d18-4c09-b910-0f3d3c73c9d1","Type":"ContainerDied","Data":"4b09bd3ddf08c7d883b7a175c6a8bdada4f64a574b6165b51d873a1d08c77fde"} Jul 13 07:08:49.847721 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.847598 2586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4b09bd3ddf08c7d883b7a175c6a8bdada4f64a574b6165b51d873a1d08c77fde" Jul 13 07:08:49.847721 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.847550 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rbp4k" Jul 13 07:08:49.849831 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.849804 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fvhbn" event={"ID":"c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b","Type":"ContainerDied","Data":"cafd61f11da0321e0b688ffa16681681c8a11a25530c4e2b266cb06d6abf258b"} Jul 13 07:08:49.849831 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.849826 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fvhbn" Jul 13 07:08:49.850007 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.849842 2586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cafd61f11da0321e0b688ffa16681681c8a11a25530c4e2b266cb06d6abf258b" Jul 13 07:08:49.851555 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.851502 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8d967f6bf-47qmq" event={"ID":"4f410a58-dbcb-4901-943c-eb158ae798a9","Type":"ContainerStarted","Data":"89013f59665a001517c725c36050c823e2d1609f67b1a006fe75e9fc22bef5d3"} Jul 13 07:08:49.851555 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.851541 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8d967f6bf-47qmq" event={"ID":"4f410a58-dbcb-4901-943c-eb158ae798a9","Type":"ContainerStarted","Data":"a3d40e2c4563e29e08b7f249a7a222cf84c27e8b9e7678f67900600298ca4680"} Jul 13 07:08:49.887279 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:49.887224 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8d967f6bf-47qmq" podStartSLOduration=0.887209341 podStartE2EDuration="887.209341ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:08:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:08:49.884860512 +0000 UTC m=+404.720424344" watchObservedRunningTime="2026-07-13 07:08:49.887209341 +0000 UTC m=+404.722773171" Jul 13 07:08:58.766510 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:58.766464 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wf9s7"] Jul 13 07:08:58.769969 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:58.769947 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wf9s7" Jul 13 07:08:58.772057 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:58.772033 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-l6h62\"" Jul 13 07:08:58.781365 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:58.781339 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wf9s7"] Jul 13 07:08:58.867891 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:58.867850 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5wz4\" (UniqueName: \"kubernetes.io/projected/ddc269fa-145a-479d-b0c2-c30a15829f8d-kube-api-access-f5wz4\") pod \"limitador-operator-controller-manager-859d7b78bd-wf9s7\" (UID: \"ddc269fa-145a-479d-b0c2-c30a15829f8d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wf9s7" Jul 13 07:08:58.968758 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:58.968717 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5wz4\" (UniqueName: \"kubernetes.io/projected/ddc269fa-145a-479d-b0c2-c30a15829f8d-kube-api-access-f5wz4\") pod \"limitador-operator-controller-manager-859d7b78bd-wf9s7\" (UID: \"ddc269fa-145a-479d-b0c2-c30a15829f8d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wf9s7" Jul 13 07:08:58.995891 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:58.995850 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5wz4\" (UniqueName: \"kubernetes.io/projected/ddc269fa-145a-479d-b0c2-c30a15829f8d-kube-api-access-f5wz4\") pod \"limitador-operator-controller-manager-859d7b78bd-wf9s7\" (UID: \"ddc269fa-145a-479d-b0c2-c30a15829f8d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wf9s7" Jul 13 07:08:59.081353 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:59.081259 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wf9s7" Jul 13 07:08:59.224183 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:59.224155 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wf9s7"] Jul 13 07:08:59.226429 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:08:59.226398 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddc269fa_145a_479d_b0c2_c30a15829f8d.slice/crio-7df774cef06f644dba83e1aec9ea83e37470d04b7962832826f5fa1310c1302e WatchSource:0}: Error finding container 7df774cef06f644dba83e1aec9ea83e37470d04b7962832826f5fa1310c1302e: Status 404 returned error can't find the container with id 7df774cef06f644dba83e1aec9ea83e37470d04b7962832826f5fa1310c1302e Jul 13 07:08:59.501795 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:59.501758 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8d967f6bf-47qmq" Jul 13 07:08:59.502103 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:59.502077 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8d967f6bf-47qmq" Jul 13 07:08:59.507573 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:59.507546 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8d967f6bf-47qmq" Jul 13 07:08:59.902797 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:59.902711 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wf9s7" event={"ID":"ddc269fa-145a-479d-b0c2-c30a15829f8d","Type":"ContainerStarted","Data":"7df774cef06f644dba83e1aec9ea83e37470d04b7962832826f5fa1310c1302e"} Jul 13 07:08:59.907134 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:59.907108 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8d967f6bf-47qmq" Jul 13 07:08:59.968028 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:08:59.967986 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7dbf5bccf7-2tmdw"] Jul 13 07:09:01.916750 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:01.916714 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wf9s7" event={"ID":"ddc269fa-145a-479d-b0c2-c30a15829f8d","Type":"ContainerStarted","Data":"c7b1117fae9419ea178b2776ef23275391f177f5b859e364b383ff323e545644"} Jul 13 07:09:01.917227 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:01.916819 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wf9s7" Jul 13 07:09:01.933612 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:01.933557 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wf9s7" podStartSLOduration=2.053311483 podStartE2EDuration="3.933540529s" totalImagesPullingTime="1.880229046s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:08:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:09:01.932764148 +0000 UTC m=+416.768327979" watchObservedRunningTime="2026-07-13 07:09:01.933540529 +0000 UTC m=+416.769104359" Jul 13 07:09:03.699417 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:03.699384 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c2br8"] Jul 13 07:09:03.706887 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:03.706860 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c2br8" Jul 13 07:09:03.708767 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:03.708733 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lk92\" (UniqueName: \"kubernetes.io/projected/9a8feaa1-e0e4-469c-80cd-fda2d9f7fab9-kube-api-access-4lk92\") pod \"kuadrant-operator-controller-manager-8684c9ff49-c2br8\" (UID: \"9a8feaa1-e0e4-469c-80cd-fda2d9f7fab9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c2br8" Jul 13 07:09:03.708902 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:03.708801 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9a8feaa1-e0e4-469c-80cd-fda2d9f7fab9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-c2br8\" (UID: \"9a8feaa1-e0e4-469c-80cd-fda2d9f7fab9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c2br8" Jul 13 07:09:03.708902 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:03.708847 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-rhv55\"" Jul 13 07:09:03.716191 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:03.716164 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c2br8"] Jul 13 07:09:03.810270 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:03.810234 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lk92\" (UniqueName: \"kubernetes.io/projected/9a8feaa1-e0e4-469c-80cd-fda2d9f7fab9-kube-api-access-4lk92\") pod \"kuadrant-operator-controller-manager-8684c9ff49-c2br8\" (UID: \"9a8feaa1-e0e4-469c-80cd-fda2d9f7fab9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c2br8" Jul 13 07:09:03.810270 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:03.810273 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9a8feaa1-e0e4-469c-80cd-fda2d9f7fab9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-c2br8\" (UID: \"9a8feaa1-e0e4-469c-80cd-fda2d9f7fab9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c2br8" Jul 13 07:09:03.810638 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:03.810622 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9a8feaa1-e0e4-469c-80cd-fda2d9f7fab9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-c2br8\" (UID: \"9a8feaa1-e0e4-469c-80cd-fda2d9f7fab9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c2br8" Jul 13 07:09:03.821909 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:03.821880 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lk92\" (UniqueName: \"kubernetes.io/projected/9a8feaa1-e0e4-469c-80cd-fda2d9f7fab9-kube-api-access-4lk92\") pod \"kuadrant-operator-controller-manager-8684c9ff49-c2br8\" (UID: \"9a8feaa1-e0e4-469c-80cd-fda2d9f7fab9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c2br8" Jul 13 07:09:04.017723 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:04.017615 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c2br8" Jul 13 07:09:04.163091 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:04.163050 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c2br8"] Jul 13 07:09:04.164369 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:09:04.164341 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a8feaa1_e0e4_469c_80cd_fda2d9f7fab9.slice/crio-7f582ff4d435de4898dcca1357283e62264fd9982d1bc6e71107491388801134 WatchSource:0}: Error finding container 7f582ff4d435de4898dcca1357283e62264fd9982d1bc6e71107491388801134: Status 404 returned error can't find the container with id 7f582ff4d435de4898dcca1357283e62264fd9982d1bc6e71107491388801134 Jul 13 07:09:04.938953 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:04.938915 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c2br8" event={"ID":"9a8feaa1-e0e4-469c-80cd-fda2d9f7fab9","Type":"ContainerStarted","Data":"7f582ff4d435de4898dcca1357283e62264fd9982d1bc6e71107491388801134"} Jul 13 07:09:06.748673 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:06.748635 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-g4z7k"] Jul 13 07:09:06.753637 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:06.753619 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-g4z7k" Jul 13 07:09:06.755867 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:06.755849 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-sxvcz\"" Jul 13 07:09:06.761322 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:06.761204 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-g4z7k"] Jul 13 07:09:06.830671 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:06.830635 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wc9h5\" (UniqueName: \"kubernetes.io/projected/63d0ee4f-2e8e-49e0-9fa2-68e29d18b112-kube-api-access-wc9h5\") pod \"authorino-operator-75cc8d5cb7-g4z7k\" (UID: \"63d0ee4f-2e8e-49e0-9fa2-68e29d18b112\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-g4z7k" Jul 13 07:09:06.931298 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:06.931258 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wc9h5\" (UniqueName: \"kubernetes.io/projected/63d0ee4f-2e8e-49e0-9fa2-68e29d18b112-kube-api-access-wc9h5\") pod \"authorino-operator-75cc8d5cb7-g4z7k\" (UID: \"63d0ee4f-2e8e-49e0-9fa2-68e29d18b112\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-g4z7k" Jul 13 07:09:06.942334 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:06.942301 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wc9h5\" (UniqueName: \"kubernetes.io/projected/63d0ee4f-2e8e-49e0-9fa2-68e29d18b112-kube-api-access-wc9h5\") pod \"authorino-operator-75cc8d5cb7-g4z7k\" (UID: \"63d0ee4f-2e8e-49e0-9fa2-68e29d18b112\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-g4z7k" Jul 13 07:09:07.065665 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:07.065579 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-g4z7k" Jul 13 07:09:07.224491 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:07.224459 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-g4z7k"] Jul 13 07:09:07.226204 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:09:07.226170 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63d0ee4f_2e8e_49e0_9fa2_68e29d18b112.slice/crio-bbf0379352c55106745e1d8e509151d8726c9e6b4446792040ba9d33a70c9f7e WatchSource:0}: Error finding container bbf0379352c55106745e1d8e509151d8726c9e6b4446792040ba9d33a70c9f7e: Status 404 returned error can't find the container with id bbf0379352c55106745e1d8e509151d8726c9e6b4446792040ba9d33a70c9f7e Jul 13 07:09:07.955571 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:07.955518 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-g4z7k" event={"ID":"63d0ee4f-2e8e-49e0-9fa2-68e29d18b112","Type":"ContainerStarted","Data":"bbf0379352c55106745e1d8e509151d8726c9e6b4446792040ba9d33a70c9f7e"} Jul 13 07:09:09.969018 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:09.968972 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-g4z7k" event={"ID":"63d0ee4f-2e8e-49e0-9fa2-68e29d18b112","Type":"ContainerStarted","Data":"b1a6f81ebe3fa761c2feb9d91d8d8e77d59ab50991a73224bd657d4c34e6d647"} Jul 13 07:09:09.969508 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:09.969054 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-g4z7k" Jul 13 07:09:09.989304 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:09.989193 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-g4z7k" podStartSLOduration=1.492840493 podStartE2EDuration="3.989174155s" totalImagesPullingTime="2.496333662s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:09:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:09:09.986521939 +0000 UTC m=+424.822085773" watchObservedRunningTime="2026-07-13 07:09:09.989174155 +0000 UTC m=+424.824737987" Jul 13 07:09:11.982264 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:11.982175 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c2br8" event={"ID":"9a8feaa1-e0e4-469c-80cd-fda2d9f7fab9","Type":"ContainerStarted","Data":"c43e07cf00d5b76cecb32a28f64b09bffab8f16f1f4240829afb390ba49ea18e"} Jul 13 07:09:11.982756 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:11.982306 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c2br8" Jul 13 07:09:12.001877 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:12.001825 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c2br8" podStartSLOduration=2.126973355 podStartE2EDuration="9.001811284s" totalImagesPullingTime="6.874837929s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:09:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:09:11.999869822 +0000 UTC m=+426.835433651" watchObservedRunningTime="2026-07-13 07:09:12.001811284 +0000 UTC m=+426.837375114" Jul 13 07:09:12.924473 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:12.924436 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wf9s7" Jul 13 07:09:20.977085 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:20.977049 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-g4z7k" Jul 13 07:09:22.990903 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:22.990877 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c2br8" Jul 13 07:09:24.725082 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:24.725048 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c2br8"] Jul 13 07:09:24.725463 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:24.725277 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c2br8" podUID="9a8feaa1-e0e4-469c-80cd-fda2d9f7fab9" containerName="manager" containerID="cri-o://c43e07cf00d5b76cecb32a28f64b09bffab8f16f1f4240829afb390ba49ea18e" gracePeriod=2 Jul 13 07:09:24.734900 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:24.734865 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c2br8"] Jul 13 07:09:24.748800 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:24.748768 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wf9s7"] Jul 13 07:09:24.749085 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:24.749058 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wf9s7" podUID="ddc269fa-145a-479d-b0c2-c30a15829f8d" containerName="manager" containerID="cri-o://c7b1117fae9419ea178b2776ef23275391f177f5b859e364b383ff323e545644" gracePeriod=2 Jul 13 07:09:24.751556 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:09:24.751518 2586 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-wf9s7\" is forbidden: User \"system:node:ip-10-0-141-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-37.ec2.internal' and this object" podUID="ddc269fa-145a-479d-b0c2-c30a15829f8d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wf9s7" Jul 13 07:09:24.753203 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:09:24.753174 2586 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-c2br8\" is forbidden: User \"system:node:ip-10-0-141-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-37.ec2.internal' and this object" podUID="9a8feaa1-e0e4-469c-80cd-fda2d9f7fab9" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c2br8" Jul 13 07:09:24.753774 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:24.753504 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tzcqk"] Jul 13 07:09:24.754089 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:24.754068 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a8feaa1-e0e4-469c-80cd-fda2d9f7fab9" containerName="manager" Jul 13 07:09:24.754175 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:24.754092 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a8feaa1-e0e4-469c-80cd-fda2d9f7fab9" containerName="manager" Jul 13 07:09:24.754260 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:24.754245 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a8feaa1-e0e4-469c-80cd-fda2d9f7fab9" containerName="manager" Jul 13 07:09:24.757653 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:24.757631 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wf9s7"] Jul 13 07:09:24.757772 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:24.757753 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tzcqk" Jul 13 07:09:24.780748 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:24.780715 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tzcqk"] Jul 13 07:09:24.783068 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:24.783043 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7xfkf"] Jul 13 07:09:24.783398 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:24.783383 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddc269fa-145a-479d-b0c2-c30a15829f8d" containerName="manager" Jul 13 07:09:24.783398 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:24.783398 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddc269fa-145a-479d-b0c2-c30a15829f8d" containerName="manager" Jul 13 07:09:24.783528 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:24.783471 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddc269fa-145a-479d-b0c2-c30a15829f8d" containerName="manager" Jul 13 07:09:24.786564 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:24.786545 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7xfkf" Jul 13 07:09:24.798963 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:24.798938 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7xfkf"] Jul 13 07:09:24.808226 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:09:24.808176 2586 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-wf9s7\" is forbidden: User \"system:node:ip-10-0-141-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-37.ec2.internal' and this object" podUID="ddc269fa-145a-479d-b0c2-c30a15829f8d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wf9s7" Jul 13 07:09:24.809763 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:09:24.809736 2586 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-c2br8\" is forbidden: User \"system:node:ip-10-0-141-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-37.ec2.internal' and this object" podUID="9a8feaa1-e0e4-469c-80cd-fda2d9f7fab9" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c2br8" Jul 13 07:09:24.811043 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:09:24.811016 2586 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-wf9s7\" is forbidden: User \"system:node:ip-10-0-141-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-37.ec2.internal' and this object" podUID="ddc269fa-145a-479d-b0c2-c30a15829f8d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wf9s7" Jul 13 07:09:24.812581 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:09:24.812549 2586 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-c2br8\" is forbidden: User \"system:node:ip-10-0-141-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-37.ec2.internal' and this object" podUID="9a8feaa1-e0e4-469c-80cd-fda2d9f7fab9" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c2br8" Jul 13 07:09:24.891853 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:24.891821 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxq6k\" (UniqueName: \"kubernetes.io/projected/ff395397-599a-49c9-9ef2-0273bb1d8c87-kube-api-access-fxq6k\") pod \"limitador-operator-controller-manager-859d7b78bd-7xfkf\" (UID: \"ff395397-599a-49c9-9ef2-0273bb1d8c87\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7xfkf" Jul 13 07:09:24.892002 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:24.891892 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9pbf\" (UniqueName: \"kubernetes.io/projected/f3d5aa6b-20fb-404c-b7cd-72fc73f113b3-kube-api-access-t9pbf\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tzcqk\" (UID: \"f3d5aa6b-20fb-404c-b7cd-72fc73f113b3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tzcqk" Jul 13 07:09:24.892002 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:24.891939 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f3d5aa6b-20fb-404c-b7cd-72fc73f113b3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tzcqk\" (UID: \"f3d5aa6b-20fb-404c-b7cd-72fc73f113b3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tzcqk" Jul 13 07:09:24.992626 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:24.992593 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxq6k\" (UniqueName: \"kubernetes.io/projected/ff395397-599a-49c9-9ef2-0273bb1d8c87-kube-api-access-fxq6k\") pod \"limitador-operator-controller-manager-859d7b78bd-7xfkf\" (UID: \"ff395397-599a-49c9-9ef2-0273bb1d8c87\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7xfkf" Jul 13 07:09:24.992766 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:24.992645 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9pbf\" (UniqueName: \"kubernetes.io/projected/f3d5aa6b-20fb-404c-b7cd-72fc73f113b3-kube-api-access-t9pbf\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tzcqk\" (UID: \"f3d5aa6b-20fb-404c-b7cd-72fc73f113b3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tzcqk" Jul 13 07:09:24.992766 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:24.992745 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f3d5aa6b-20fb-404c-b7cd-72fc73f113b3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tzcqk\" (UID: \"f3d5aa6b-20fb-404c-b7cd-72fc73f113b3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tzcqk" Jul 13 07:09:24.993246 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:24.993222 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f3d5aa6b-20fb-404c-b7cd-72fc73f113b3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tzcqk\" (UID: \"f3d5aa6b-20fb-404c-b7cd-72fc73f113b3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tzcqk" Jul 13 07:09:24.994583 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:24.994558 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7dbf5bccf7-2tmdw" podUID="e121cf45-b562-4c35-a7df-094f60dece96" containerName="console" containerID="cri-o://cafce630f25a0f7fabe45a4967b11c68ea4430ea368ba5f03af8e19f70f020b7" gracePeriod=15 Jul 13 07:09:24.996290 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:24.996271 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c2br8" Jul 13 07:09:24.998017 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:09:24.997989 2586 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-wf9s7\" is forbidden: User \"system:node:ip-10-0-141-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-37.ec2.internal' and this object" podUID="ddc269fa-145a-479d-b0c2-c30a15829f8d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wf9s7" Jul 13 07:09:24.999278 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:09:24.999253 2586 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-c2br8\" is forbidden: User \"system:node:ip-10-0-141-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-37.ec2.internal' and this object" podUID="9a8feaa1-e0e4-469c-80cd-fda2d9f7fab9" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c2br8" Jul 13 07:09:24.999631 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:24.999613 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wf9s7" Jul 13 07:09:25.000988 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:09:25.000966 2586 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-c2br8\" is forbidden: User \"system:node:ip-10-0-141-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-37.ec2.internal' and this object" podUID="9a8feaa1-e0e4-469c-80cd-fda2d9f7fab9" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c2br8" Jul 13 07:09:25.002338 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:09:25.002309 2586 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-wf9s7\" is forbidden: User \"system:node:ip-10-0-141-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-37.ec2.internal' and this object" podUID="ddc269fa-145a-479d-b0c2-c30a15829f8d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wf9s7" Jul 13 07:09:25.005360 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:25.005329 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9pbf\" (UniqueName: \"kubernetes.io/projected/f3d5aa6b-20fb-404c-b7cd-72fc73f113b3-kube-api-access-t9pbf\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tzcqk\" (UID: \"f3d5aa6b-20fb-404c-b7cd-72fc73f113b3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tzcqk" Jul 13 07:09:25.005487 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:25.005373 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxq6k\" (UniqueName: \"kubernetes.io/projected/ff395397-599a-49c9-9ef2-0273bb1d8c87-kube-api-access-fxq6k\") pod \"limitador-operator-controller-manager-859d7b78bd-7xfkf\" (UID: \"ff395397-599a-49c9-9ef2-0273bb1d8c87\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7xfkf" Jul 13 07:09:25.060545 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:25.060511 2586 generic.go:356] "Generic (PLEG): container finished" podID="9a8feaa1-e0e4-469c-80cd-fda2d9f7fab9" containerID="c43e07cf00d5b76cecb32a28f64b09bffab8f16f1f4240829afb390ba49ea18e" exitCode=0 Jul 13 07:09:25.060713 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:25.060561 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c2br8" Jul 13 07:09:25.060713 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:25.060594 2586 scope.go:117] "RemoveContainer" containerID="c43e07cf00d5b76cecb32a28f64b09bffab8f16f1f4240829afb390ba49ea18e" Jul 13 07:09:25.062077 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:25.062058 2586 generic.go:356] "Generic (PLEG): container finished" podID="ddc269fa-145a-479d-b0c2-c30a15829f8d" containerID="c7b1117fae9419ea178b2776ef23275391f177f5b859e364b383ff323e545644" exitCode=0 Jul 13 07:09:25.062193 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:25.062098 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wf9s7" Jul 13 07:09:25.062327 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:09:25.062306 2586 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-c2br8\" is forbidden: User \"system:node:ip-10-0-141-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-37.ec2.internal' and this object" podUID="9a8feaa1-e0e4-469c-80cd-fda2d9f7fab9" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c2br8" Jul 13 07:09:25.063609 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:09:25.063580 2586 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-wf9s7\" is forbidden: User \"system:node:ip-10-0-141-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-37.ec2.internal' and this object" podUID="ddc269fa-145a-479d-b0c2-c30a15829f8d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wf9s7" Jul 13 07:09:25.064914 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:09:25.064891 2586 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-wf9s7\" is forbidden: User \"system:node:ip-10-0-141-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-37.ec2.internal' and this object" podUID="ddc269fa-145a-479d-b0c2-c30a15829f8d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wf9s7" Jul 13 07:09:25.066110 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:09:25.066088 2586 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-c2br8\" is forbidden: User \"system:node:ip-10-0-141-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-37.ec2.internal' and this object" podUID="9a8feaa1-e0e4-469c-80cd-fda2d9f7fab9" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c2br8" Jul 13 07:09:25.093339 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:25.093312 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4lk92\" (UniqueName: \"kubernetes.io/projected/9a8feaa1-e0e4-469c-80cd-fda2d9f7fab9-kube-api-access-4lk92\") pod \"9a8feaa1-e0e4-469c-80cd-fda2d9f7fab9\" (UID: \"9a8feaa1-e0e4-469c-80cd-fda2d9f7fab9\") " Jul 13 07:09:25.093589 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:25.093356 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f5wz4\" (UniqueName: \"kubernetes.io/projected/ddc269fa-145a-479d-b0c2-c30a15829f8d-kube-api-access-f5wz4\") pod \"ddc269fa-145a-479d-b0c2-c30a15829f8d\" (UID: \"ddc269fa-145a-479d-b0c2-c30a15829f8d\") " Jul 13 07:09:25.093589 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:25.093468 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9a8feaa1-e0e4-469c-80cd-fda2d9f7fab9-extensions-socket-volume\") pod \"9a8feaa1-e0e4-469c-80cd-fda2d9f7fab9\" (UID: \"9a8feaa1-e0e4-469c-80cd-fda2d9f7fab9\") " Jul 13 07:09:25.094032 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:25.093990 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a8feaa1-e0e4-469c-80cd-fda2d9f7fab9-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "9a8feaa1-e0e4-469c-80cd-fda2d9f7fab9" (UID: "9a8feaa1-e0e4-469c-80cd-fda2d9f7fab9"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:09:25.095428 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:25.095404 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddc269fa-145a-479d-b0c2-c30a15829f8d-kube-api-access-f5wz4" (OuterVolumeSpecName: "kube-api-access-f5wz4") pod "ddc269fa-145a-479d-b0c2-c30a15829f8d" (UID: "ddc269fa-145a-479d-b0c2-c30a15829f8d"). InnerVolumeSpecName "kube-api-access-f5wz4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:09:25.095519 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:25.095457 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a8feaa1-e0e4-469c-80cd-fda2d9f7fab9-kube-api-access-4lk92" (OuterVolumeSpecName: "kube-api-access-4lk92") pod "9a8feaa1-e0e4-469c-80cd-fda2d9f7fab9" (UID: "9a8feaa1-e0e4-469c-80cd-fda2d9f7fab9"). InnerVolumeSpecName "kube-api-access-4lk92". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:09:25.106856 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:25.106834 2586 scope.go:117] "RemoveContainer" containerID="c43e07cf00d5b76cecb32a28f64b09bffab8f16f1f4240829afb390ba49ea18e" Jul 13 07:09:25.107237 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:09:25.107213 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c43e07cf00d5b76cecb32a28f64b09bffab8f16f1f4240829afb390ba49ea18e\": container with ID starting with c43e07cf00d5b76cecb32a28f64b09bffab8f16f1f4240829afb390ba49ea18e not found: ID does not exist" containerID="c43e07cf00d5b76cecb32a28f64b09bffab8f16f1f4240829afb390ba49ea18e" Jul 13 07:09:25.107330 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:25.107243 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c43e07cf00d5b76cecb32a28f64b09bffab8f16f1f4240829afb390ba49ea18e"} err="failed to get container status \"c43e07cf00d5b76cecb32a28f64b09bffab8f16f1f4240829afb390ba49ea18e\": rpc error: code = NotFound desc = could not find container \"c43e07cf00d5b76cecb32a28f64b09bffab8f16f1f4240829afb390ba49ea18e\": container with ID starting with c43e07cf00d5b76cecb32a28f64b09bffab8f16f1f4240829afb390ba49ea18e not found: ID does not exist" Jul 13 07:09:25.107330 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:25.107263 2586 scope.go:117] "RemoveContainer" containerID="c7b1117fae9419ea178b2776ef23275391f177f5b859e364b383ff323e545644" Jul 13 07:09:25.115751 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:25.115732 2586 scope.go:117] "RemoveContainer" containerID="c7b1117fae9419ea178b2776ef23275391f177f5b859e364b383ff323e545644" Jul 13 07:09:25.115982 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:09:25.115965 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7b1117fae9419ea178b2776ef23275391f177f5b859e364b383ff323e545644\": container with ID starting with c7b1117fae9419ea178b2776ef23275391f177f5b859e364b383ff323e545644 not found: ID does not exist" containerID="c7b1117fae9419ea178b2776ef23275391f177f5b859e364b383ff323e545644" Jul 13 07:09:25.116034 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:25.115988 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7b1117fae9419ea178b2776ef23275391f177f5b859e364b383ff323e545644"} err="failed to get container status \"c7b1117fae9419ea178b2776ef23275391f177f5b859e364b383ff323e545644\": rpc error: code = NotFound desc = could not find container \"c7b1117fae9419ea178b2776ef23275391f177f5b859e364b383ff323e545644\": container with ID starting with c7b1117fae9419ea178b2776ef23275391f177f5b859e364b383ff323e545644 not found: ID does not exist" Jul 13 07:09:25.177004 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:25.176972 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tzcqk" Jul 13 07:09:25.183534 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:25.183485 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7xfkf" Jul 13 07:09:25.195026 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:25.194999 2586 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9a8feaa1-e0e4-469c-80cd-fda2d9f7fab9-extensions-socket-volume\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:09:25.195026 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:25.195025 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4lk92\" (UniqueName: \"kubernetes.io/projected/9a8feaa1-e0e4-469c-80cd-fda2d9f7fab9-kube-api-access-4lk92\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:09:25.195237 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:25.195040 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f5wz4\" (UniqueName: \"kubernetes.io/projected/ddc269fa-145a-479d-b0c2-c30a15829f8d-kube-api-access-f5wz4\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:09:25.253310 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:25.253025 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dbf5bccf7-2tmdw" Jul 13 07:09:25.254959 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:09:25.254881 2586 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-c2br8\" is forbidden: User \"system:node:ip-10-0-141-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-37.ec2.internal' and this object" podUID="9a8feaa1-e0e4-469c-80cd-fda2d9f7fab9" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c2br8" Jul 13 07:09:25.277881 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:09:25.277834 2586 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-wf9s7\" is forbidden: User \"system:node:ip-10-0-141-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-37.ec2.internal' and this object" podUID="ddc269fa-145a-479d-b0c2-c30a15829f8d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wf9s7" Jul 13 07:09:25.296046 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:25.296018 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e121cf45-b562-4c35-a7df-094f60dece96-trusted-ca-bundle\") pod \"e121cf45-b562-4c35-a7df-094f60dece96\" (UID: \"e121cf45-b562-4c35-a7df-094f60dece96\") " Jul 13 07:09:25.296220 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:25.296055 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e121cf45-b562-4c35-a7df-094f60dece96-service-ca\") pod \"e121cf45-b562-4c35-a7df-094f60dece96\" (UID: \"e121cf45-b562-4c35-a7df-094f60dece96\") " Jul 13 07:09:25.296220 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:25.296097 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e121cf45-b562-4c35-a7df-094f60dece96-oauth-serving-cert\") pod \"e121cf45-b562-4c35-a7df-094f60dece96\" (UID: \"e121cf45-b562-4c35-a7df-094f60dece96\") " Jul 13 07:09:25.296220 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:25.296119 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e121cf45-b562-4c35-a7df-094f60dece96-console-serving-cert\") pod \"e121cf45-b562-4c35-a7df-094f60dece96\" (UID: \"e121cf45-b562-4c35-a7df-094f60dece96\") " Jul 13 07:09:25.296220 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:25.296149 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hhhv4\" (UniqueName: \"kubernetes.io/projected/e121cf45-b562-4c35-a7df-094f60dece96-kube-api-access-hhhv4\") pod \"e121cf45-b562-4c35-a7df-094f60dece96\" (UID: \"e121cf45-b562-4c35-a7df-094f60dece96\") " Jul 13 07:09:25.296220 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:25.296170 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e121cf45-b562-4c35-a7df-094f60dece96-console-config\") pod \"e121cf45-b562-4c35-a7df-094f60dece96\" (UID: \"e121cf45-b562-4c35-a7df-094f60dece96\") " Jul 13 07:09:25.297159 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:25.296627 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e121cf45-b562-4c35-a7df-094f60dece96-service-ca" (OuterVolumeSpecName: "service-ca") pod "e121cf45-b562-4c35-a7df-094f60dece96" (UID: "e121cf45-b562-4c35-a7df-094f60dece96"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 07:09:25.297159 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:25.296690 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e121cf45-b562-4c35-a7df-094f60dece96-console-oauth-config\") pod \"e121cf45-b562-4c35-a7df-094f60dece96\" (UID: \"e121cf45-b562-4c35-a7df-094f60dece96\") " Jul 13 07:09:25.297159 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:25.296899 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e121cf45-b562-4c35-a7df-094f60dece96-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e121cf45-b562-4c35-a7df-094f60dece96" (UID: "e121cf45-b562-4c35-a7df-094f60dece96"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 07:09:25.297159 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:25.296960 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e121cf45-b562-4c35-a7df-094f60dece96-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e121cf45-b562-4c35-a7df-094f60dece96" (UID: "e121cf45-b562-4c35-a7df-094f60dece96"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 07:09:25.297159 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:25.297066 2586 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e121cf45-b562-4c35-a7df-094f60dece96-trusted-ca-bundle\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:09:25.297159 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:25.297085 2586 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e121cf45-b562-4c35-a7df-094f60dece96-service-ca\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:09:25.297159 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:25.297100 2586 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e121cf45-b562-4c35-a7df-094f60dece96-oauth-serving-cert\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:09:25.297159 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:25.297129 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e121cf45-b562-4c35-a7df-094f60dece96-console-config" (OuterVolumeSpecName: "console-config") pod "e121cf45-b562-4c35-a7df-094f60dece96" (UID: "e121cf45-b562-4c35-a7df-094f60dece96"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 07:09:25.299784 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:25.299754 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e121cf45-b562-4c35-a7df-094f60dece96-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e121cf45-b562-4c35-a7df-094f60dece96" (UID: "e121cf45-b562-4c35-a7df-094f60dece96"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 07:09:25.301418 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:25.301387 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e121cf45-b562-4c35-a7df-094f60dece96-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e121cf45-b562-4c35-a7df-094f60dece96" (UID: "e121cf45-b562-4c35-a7df-094f60dece96"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 07:09:25.301517 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:25.301499 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e121cf45-b562-4c35-a7df-094f60dece96-kube-api-access-hhhv4" (OuterVolumeSpecName: "kube-api-access-hhhv4") pod "e121cf45-b562-4c35-a7df-094f60dece96" (UID: "e121cf45-b562-4c35-a7df-094f60dece96"). InnerVolumeSpecName "kube-api-access-hhhv4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:09:25.334938 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:25.334900 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7xfkf"] Jul 13 07:09:25.336149 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:09:25.336114 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff395397_599a_49c9_9ef2_0273bb1d8c87.slice/crio-44f196a87c1aa2db2c07d9cb512cd7e56027f745ed45f4ac6af67e7122dff730 WatchSource:0}: Error finding container 44f196a87c1aa2db2c07d9cb512cd7e56027f745ed45f4ac6af67e7122dff730: Status 404 returned error can't find the container with id 44f196a87c1aa2db2c07d9cb512cd7e56027f745ed45f4ac6af67e7122dff730 Jul 13 07:09:25.360137 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:25.360074 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tzcqk"] Jul 13 07:09:25.369927 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:09:25.369883 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3d5aa6b_20fb_404c_b7cd_72fc73f113b3.slice/crio-ff73ce4aa3c74c255846bd0d7929fb3eaa7cf4bb5bd52036801aa9cac0f76055 WatchSource:0}: Error finding container ff73ce4aa3c74c255846bd0d7929fb3eaa7cf4bb5bd52036801aa9cac0f76055: Status 404 returned error can't find the container with id ff73ce4aa3c74c255846bd0d7929fb3eaa7cf4bb5bd52036801aa9cac0f76055 Jul 13 07:09:25.375523 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:09:25.375493 2586 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-wf9s7\" is forbidden: User \"system:node:ip-10-0-141-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-37.ec2.internal' and this object" podUID="ddc269fa-145a-479d-b0c2-c30a15829f8d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wf9s7" Jul 13 07:09:25.377231 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:09:25.377202 2586 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-c2br8\" is forbidden: User \"system:node:ip-10-0-141-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-37.ec2.internal' and this object" podUID="9a8feaa1-e0e4-469c-80cd-fda2d9f7fab9" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c2br8" Jul 13 07:09:25.378577 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:09:25.378551 2586 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-c2br8\" is forbidden: User \"system:node:ip-10-0-141-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-37.ec2.internal' and this object" podUID="9a8feaa1-e0e4-469c-80cd-fda2d9f7fab9" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c2br8" Jul 13 07:09:25.379824 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:09:25.379792 2586 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-wf9s7\" is forbidden: User \"system:node:ip-10-0-141-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-37.ec2.internal' and this object" podUID="ddc269fa-145a-479d-b0c2-c30a15829f8d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wf9s7" Jul 13 07:09:25.397813 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:25.397791 2586 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e121cf45-b562-4c35-a7df-094f60dece96-console-oauth-config\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:09:25.397813 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:25.397814 2586 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e121cf45-b562-4c35-a7df-094f60dece96-console-serving-cert\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:09:25.397959 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:25.397823 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhhv4\" (UniqueName: \"kubernetes.io/projected/e121cf45-b562-4c35-a7df-094f60dece96-kube-api-access-hhhv4\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:09:25.397959 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:25.397833 2586 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e121cf45-b562-4c35-a7df-094f60dece96-console-config\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:09:25.804915 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:09:25.804878 2586 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-wf9s7\" is forbidden: User \"system:node:ip-10-0-141-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-37.ec2.internal' and this object" podUID="ddc269fa-145a-479d-b0c2-c30a15829f8d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wf9s7" Jul 13 07:09:25.806170 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:09:25.806139 2586 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-c2br8\" is forbidden: User \"system:node:ip-10-0-141-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-37.ec2.internal' and this object" podUID="9a8feaa1-e0e4-469c-80cd-fda2d9f7fab9" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c2br8" Jul 13 07:09:25.806900 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:25.806872 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a8feaa1-e0e4-469c-80cd-fda2d9f7fab9" path="/var/lib/kubelet/pods/9a8feaa1-e0e4-469c-80cd-fda2d9f7fab9/volumes" Jul 13 07:09:25.807295 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:25.807277 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ddc269fa-145a-479d-b0c2-c30a15829f8d" path="/var/lib/kubelet/pods/ddc269fa-145a-479d-b0c2-c30a15829f8d/volumes" Jul 13 07:09:26.069336 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:26.069230 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7xfkf" event={"ID":"ff395397-599a-49c9-9ef2-0273bb1d8c87","Type":"ContainerStarted","Data":"2439cc30e56579520bb39c3aba68e843bbcc1489770f01dd3af76b429daae040"} Jul 13 07:09:26.069336 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:26.069295 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7xfkf" event={"ID":"ff395397-599a-49c9-9ef2-0273bb1d8c87","Type":"ContainerStarted","Data":"44f196a87c1aa2db2c07d9cb512cd7e56027f745ed45f4ac6af67e7122dff730"} Jul 13 07:09:26.069336 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:26.069321 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7xfkf" Jul 13 07:09:26.070867 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:26.070838 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tzcqk" event={"ID":"f3d5aa6b-20fb-404c-b7cd-72fc73f113b3","Type":"ContainerStarted","Data":"ee9916928b2bc53c4471b7e2774f10dd6055da973ead381dd324db81b2e63826"} Jul 13 07:09:26.071011 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:26.070888 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tzcqk" event={"ID":"f3d5aa6b-20fb-404c-b7cd-72fc73f113b3","Type":"ContainerStarted","Data":"ff73ce4aa3c74c255846bd0d7929fb3eaa7cf4bb5bd52036801aa9cac0f76055"} Jul 13 07:09:26.071011 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:26.070913 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tzcqk" Jul 13 07:09:26.073123 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:26.073096 2586 generic.go:356] "Generic (PLEG): container finished" podID="e121cf45-b562-4c35-a7df-094f60dece96" containerID="cafce630f25a0f7fabe45a4967b11c68ea4430ea368ba5f03af8e19f70f020b7" exitCode=0 Jul 13 07:09:26.073223 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:26.073132 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dbf5bccf7-2tmdw" Jul 13 07:09:26.073223 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:26.073179 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dbf5bccf7-2tmdw" event={"ID":"e121cf45-b562-4c35-a7df-094f60dece96","Type":"ContainerDied","Data":"cafce630f25a0f7fabe45a4967b11c68ea4430ea368ba5f03af8e19f70f020b7"} Jul 13 07:09:26.073319 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:26.073220 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dbf5bccf7-2tmdw" event={"ID":"e121cf45-b562-4c35-a7df-094f60dece96","Type":"ContainerDied","Data":"2121ae0c0825898b6c9d1a6d3c8759d6e5754a4df1dece671a965fb9f6f87917"} Jul 13 07:09:26.073319 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:26.073248 2586 scope.go:117] "RemoveContainer" containerID="cafce630f25a0f7fabe45a4967b11c68ea4430ea368ba5f03af8e19f70f020b7" Jul 13 07:09:26.082932 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:26.082756 2586 scope.go:117] "RemoveContainer" containerID="cafce630f25a0f7fabe45a4967b11c68ea4430ea368ba5f03af8e19f70f020b7" Jul 13 07:09:26.083639 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:09:26.083607 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cafce630f25a0f7fabe45a4967b11c68ea4430ea368ba5f03af8e19f70f020b7\": container with ID starting with cafce630f25a0f7fabe45a4967b11c68ea4430ea368ba5f03af8e19f70f020b7 not found: ID does not exist" containerID="cafce630f25a0f7fabe45a4967b11c68ea4430ea368ba5f03af8e19f70f020b7" Jul 13 07:09:26.083928 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:26.083649 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cafce630f25a0f7fabe45a4967b11c68ea4430ea368ba5f03af8e19f70f020b7"} err="failed to get container status \"cafce630f25a0f7fabe45a4967b11c68ea4430ea368ba5f03af8e19f70f020b7\": rpc error: code = NotFound desc = could not find container \"cafce630f25a0f7fabe45a4967b11c68ea4430ea368ba5f03af8e19f70f020b7\": container with ID starting with cafce630f25a0f7fabe45a4967b11c68ea4430ea368ba5f03af8e19f70f020b7 not found: ID does not exist" Jul 13 07:09:26.087632 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:26.087572 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7xfkf" podStartSLOduration=2.087552511 podStartE2EDuration="2.087552511s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:09:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:09:26.083383019 +0000 UTC m=+440.918946852" watchObservedRunningTime="2026-07-13 07:09:26.087552511 +0000 UTC m=+440.923116344" Jul 13 07:09:26.102248 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:26.102211 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7dbf5bccf7-2tmdw"] Jul 13 07:09:26.107697 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:26.107649 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7dbf5bccf7-2tmdw"] Jul 13 07:09:26.121460 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:26.121409 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tzcqk" podStartSLOduration=2.121391169 podStartE2EDuration="2.121391169s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:09:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:09:26.120813336 +0000 UTC m=+440.956377172" watchObservedRunningTime="2026-07-13 07:09:26.121391169 +0000 UTC m=+440.956955001" Jul 13 07:09:27.805968 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:27.805887 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e121cf45-b562-4c35-a7df-094f60dece96" path="/var/lib/kubelet/pods/e121cf45-b562-4c35-a7df-094f60dece96/volumes" Jul 13 07:09:37.083084 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:37.083050 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tzcqk" Jul 13 07:09:37.083560 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:37.083236 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7xfkf" Jul 13 07:09:52.492988 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:52.492952 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tzcqk"] Jul 13 07:09:52.493465 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:52.493180 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tzcqk" podUID="f3d5aa6b-20fb-404c-b7cd-72fc73f113b3" containerName="manager" containerID="cri-o://ee9916928b2bc53c4471b7e2774f10dd6055da973ead381dd324db81b2e63826" gracePeriod=10 Jul 13 07:09:52.741866 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:52.741841 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tzcqk" Jul 13 07:09:52.766559 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:52.766498 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5cbpq"] Jul 13 07:09:52.766854 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:52.766840 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3d5aa6b-20fb-404c-b7cd-72fc73f113b3" containerName="manager" Jul 13 07:09:52.766921 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:52.766856 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3d5aa6b-20fb-404c-b7cd-72fc73f113b3" containerName="manager" Jul 13 07:09:52.766921 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:52.766872 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e121cf45-b562-4c35-a7df-094f60dece96" containerName="console" Jul 13 07:09:52.766921 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:52.766878 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="e121cf45-b562-4c35-a7df-094f60dece96" containerName="console" Jul 13 07:09:52.767018 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:52.766937 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="e121cf45-b562-4c35-a7df-094f60dece96" containerName="console" Jul 13 07:09:52.767018 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:52.766946 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3d5aa6b-20fb-404c-b7cd-72fc73f113b3" containerName="manager" Jul 13 07:09:52.768654 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:52.768637 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5cbpq" Jul 13 07:09:52.795600 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:52.795575 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5cbpq"] Jul 13 07:09:52.830651 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:52.830627 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f3d5aa6b-20fb-404c-b7cd-72fc73f113b3-extensions-socket-volume\") pod \"f3d5aa6b-20fb-404c-b7cd-72fc73f113b3\" (UID: \"f3d5aa6b-20fb-404c-b7cd-72fc73f113b3\") " Jul 13 07:09:52.830793 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:52.830658 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t9pbf\" (UniqueName: \"kubernetes.io/projected/f3d5aa6b-20fb-404c-b7cd-72fc73f113b3-kube-api-access-t9pbf\") pod \"f3d5aa6b-20fb-404c-b7cd-72fc73f113b3\" (UID: \"f3d5aa6b-20fb-404c-b7cd-72fc73f113b3\") " Jul 13 07:09:52.831024 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:52.831004 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3d5aa6b-20fb-404c-b7cd-72fc73f113b3-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f3d5aa6b-20fb-404c-b7cd-72fc73f113b3" (UID: "f3d5aa6b-20fb-404c-b7cd-72fc73f113b3"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:09:52.832663 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:52.832646 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3d5aa6b-20fb-404c-b7cd-72fc73f113b3-kube-api-access-t9pbf" (OuterVolumeSpecName: "kube-api-access-t9pbf") pod "f3d5aa6b-20fb-404c-b7cd-72fc73f113b3" (UID: "f3d5aa6b-20fb-404c-b7cd-72fc73f113b3"). InnerVolumeSpecName "kube-api-access-t9pbf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:09:52.931640 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:52.931603 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ht8np\" (UniqueName: \"kubernetes.io/projected/ee5d74d8-923b-49c5-96b3-f3e9492530f0-kube-api-access-ht8np\") pod \"kuadrant-operator-controller-manager-86878c769f-5cbpq\" (UID: \"ee5d74d8-923b-49c5-96b3-f3e9492530f0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5cbpq" Jul 13 07:09:52.931898 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:52.931676 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ee5d74d8-923b-49c5-96b3-f3e9492530f0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-5cbpq\" (UID: \"ee5d74d8-923b-49c5-96b3-f3e9492530f0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5cbpq" Jul 13 07:09:52.931898 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:52.931743 2586 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f3d5aa6b-20fb-404c-b7cd-72fc73f113b3-extensions-socket-volume\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:09:52.931898 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:52.931771 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t9pbf\" (UniqueName: \"kubernetes.io/projected/f3d5aa6b-20fb-404c-b7cd-72fc73f113b3-kube-api-access-t9pbf\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:09:53.033085 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:53.033006 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ht8np\" (UniqueName: \"kubernetes.io/projected/ee5d74d8-923b-49c5-96b3-f3e9492530f0-kube-api-access-ht8np\") pod \"kuadrant-operator-controller-manager-86878c769f-5cbpq\" (UID: \"ee5d74d8-923b-49c5-96b3-f3e9492530f0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5cbpq" Jul 13 07:09:53.033085 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:53.033052 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ee5d74d8-923b-49c5-96b3-f3e9492530f0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-5cbpq\" (UID: \"ee5d74d8-923b-49c5-96b3-f3e9492530f0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5cbpq" Jul 13 07:09:53.033373 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:53.033357 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ee5d74d8-923b-49c5-96b3-f3e9492530f0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-5cbpq\" (UID: \"ee5d74d8-923b-49c5-96b3-f3e9492530f0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5cbpq" Jul 13 07:09:53.043882 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:53.043862 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ht8np\" (UniqueName: \"kubernetes.io/projected/ee5d74d8-923b-49c5-96b3-f3e9492530f0-kube-api-access-ht8np\") pod \"kuadrant-operator-controller-manager-86878c769f-5cbpq\" (UID: \"ee5d74d8-923b-49c5-96b3-f3e9492530f0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5cbpq" Jul 13 07:09:53.077923 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:53.077894 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5cbpq" Jul 13 07:09:53.210602 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:53.210573 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5cbpq"] Jul 13 07:09:53.212078 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:09:53.212051 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee5d74d8_923b_49c5_96b3_f3e9492530f0.slice/crio-4c9f71162f9cca38b4ee94b5f4b972e20d6f6f05ec963f0a4b974853ed1068d1 WatchSource:0}: Error finding container 4c9f71162f9cca38b4ee94b5f4b972e20d6f6f05ec963f0a4b974853ed1068d1: Status 404 returned error can't find the container with id 4c9f71162f9cca38b4ee94b5f4b972e20d6f6f05ec963f0a4b974853ed1068d1 Jul 13 07:09:53.226739 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:53.226714 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5cbpq" event={"ID":"ee5d74d8-923b-49c5-96b3-f3e9492530f0","Type":"ContainerStarted","Data":"4c9f71162f9cca38b4ee94b5f4b972e20d6f6f05ec963f0a4b974853ed1068d1"} Jul 13 07:09:53.228153 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:53.228123 2586 generic.go:356] "Generic (PLEG): container finished" podID="f3d5aa6b-20fb-404c-b7cd-72fc73f113b3" containerID="ee9916928b2bc53c4471b7e2774f10dd6055da973ead381dd324db81b2e63826" exitCode=0 Jul 13 07:09:53.228263 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:53.228165 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tzcqk" Jul 13 07:09:53.228263 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:53.228167 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tzcqk" event={"ID":"f3d5aa6b-20fb-404c-b7cd-72fc73f113b3","Type":"ContainerDied","Data":"ee9916928b2bc53c4471b7e2774f10dd6055da973ead381dd324db81b2e63826"} Jul 13 07:09:53.228263 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:53.228227 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tzcqk" event={"ID":"f3d5aa6b-20fb-404c-b7cd-72fc73f113b3","Type":"ContainerDied","Data":"ff73ce4aa3c74c255846bd0d7929fb3eaa7cf4bb5bd52036801aa9cac0f76055"} Jul 13 07:09:53.228263 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:53.228261 2586 scope.go:117] "RemoveContainer" containerID="ee9916928b2bc53c4471b7e2774f10dd6055da973ead381dd324db81b2e63826" Jul 13 07:09:53.252956 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:53.252932 2586 scope.go:117] "RemoveContainer" containerID="ee9916928b2bc53c4471b7e2774f10dd6055da973ead381dd324db81b2e63826" Jul 13 07:09:53.253267 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:09:53.253245 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee9916928b2bc53c4471b7e2774f10dd6055da973ead381dd324db81b2e63826\": container with ID starting with ee9916928b2bc53c4471b7e2774f10dd6055da973ead381dd324db81b2e63826 not found: ID does not exist" containerID="ee9916928b2bc53c4471b7e2774f10dd6055da973ead381dd324db81b2e63826" Jul 13 07:09:53.253363 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:53.253277 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee9916928b2bc53c4471b7e2774f10dd6055da973ead381dd324db81b2e63826"} err="failed to get container status \"ee9916928b2bc53c4471b7e2774f10dd6055da973ead381dd324db81b2e63826\": rpc error: code = NotFound desc = could not find container \"ee9916928b2bc53c4471b7e2774f10dd6055da973ead381dd324db81b2e63826\": container with ID starting with ee9916928b2bc53c4471b7e2774f10dd6055da973ead381dd324db81b2e63826 not found: ID does not exist" Jul 13 07:09:53.263534 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:53.263505 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tzcqk"] Jul 13 07:09:53.270854 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:53.270827 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tzcqk"] Jul 13 07:09:53.804815 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:53.804778 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3d5aa6b-20fb-404c-b7cd-72fc73f113b3" path="/var/lib/kubelet/pods/f3d5aa6b-20fb-404c-b7cd-72fc73f113b3/volumes" Jul 13 07:09:54.235419 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:54.235383 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5cbpq" event={"ID":"ee5d74d8-923b-49c5-96b3-f3e9492530f0","Type":"ContainerStarted","Data":"78a443177f1b523a13ba12bb190104cedc2074f6acb0a39cbdaa3a1577648bd6"} Jul 13 07:09:54.235628 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:54.235457 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5cbpq" Jul 13 07:09:54.254313 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:09:54.254253 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5cbpq" podStartSLOduration=2.254238085 podStartE2EDuration="2.254238085s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:09:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:09:54.252918667 +0000 UTC m=+469.088482498" watchObservedRunningTime="2026-07-13 07:09:54.254238085 +0000 UTC m=+469.089801916" Jul 13 07:10:05.244612 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:05.244582 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5cbpq" Jul 13 07:10:12.686338 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:12.686299 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:10:12.691393 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:12.691366 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-2rksc" Jul 13 07:10:12.693444 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:12.693416 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 13 07:10:12.693614 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:12.693488 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-n6874\"" Jul 13 07:10:12.702780 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:12.702746 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:10:12.726341 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:12.726305 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:10:12.795520 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:12.795485 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fm85s\" (UniqueName: \"kubernetes.io/projected/a405d03d-88ae-493b-b1e0-d7ca25f52246-kube-api-access-fm85s\") pod \"limitador-limitador-54d6d988d5-2rksc\" (UID: \"a405d03d-88ae-493b-b1e0-d7ca25f52246\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-2rksc" Jul 13 07:10:12.795724 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:12.795560 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a405d03d-88ae-493b-b1e0-d7ca25f52246-config-file\") pod \"limitador-limitador-54d6d988d5-2rksc\" (UID: \"a405d03d-88ae-493b-b1e0-d7ca25f52246\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-2rksc" Jul 13 07:10:12.896552 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:12.896508 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fm85s\" (UniqueName: \"kubernetes.io/projected/a405d03d-88ae-493b-b1e0-d7ca25f52246-kube-api-access-fm85s\") pod \"limitador-limitador-54d6d988d5-2rksc\" (UID: \"a405d03d-88ae-493b-b1e0-d7ca25f52246\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-2rksc" Jul 13 07:10:12.896748 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:12.896591 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a405d03d-88ae-493b-b1e0-d7ca25f52246-config-file\") pod \"limitador-limitador-54d6d988d5-2rksc\" (UID: \"a405d03d-88ae-493b-b1e0-d7ca25f52246\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-2rksc" Jul 13 07:10:12.897211 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:12.897192 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a405d03d-88ae-493b-b1e0-d7ca25f52246-config-file\") pod \"limitador-limitador-54d6d988d5-2rksc\" (UID: \"a405d03d-88ae-493b-b1e0-d7ca25f52246\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-2rksc" Jul 13 07:10:12.904374 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:12.904343 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fm85s\" (UniqueName: \"kubernetes.io/projected/a405d03d-88ae-493b-b1e0-d7ca25f52246-kube-api-access-fm85s\") pod \"limitador-limitador-54d6d988d5-2rksc\" (UID: \"a405d03d-88ae-493b-b1e0-d7ca25f52246\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-2rksc" Jul 13 07:10:13.002742 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:13.002636 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-2rksc" Jul 13 07:10:13.121614 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:13.121582 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-c9gq9"] Jul 13 07:10:13.125213 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:13.125194 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-c9gq9" Jul 13 07:10:13.127843 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:13.127800 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-d5rk4\"" Jul 13 07:10:13.133633 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:13.133595 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-c9gq9"] Jul 13 07:10:13.146033 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:13.146005 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:10:13.148059 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:10:13.148028 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda405d03d_88ae_493b_b1e0_d7ca25f52246.slice/crio-cf64fa90a388bf1c0f8fce69110cf7143f9cc9c09832e6abfc727a67476bc73a WatchSource:0}: Error finding container cf64fa90a388bf1c0f8fce69110cf7143f9cc9c09832e6abfc727a67476bc73a: Status 404 returned error can't find the container with id cf64fa90a388bf1c0f8fce69110cf7143f9cc9c09832e6abfc727a67476bc73a Jul 13 07:10:13.199123 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:13.199082 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7vv7\" (UniqueName: \"kubernetes.io/projected/5a7101c1-c76b-4443-8202-62a68ed62d1f-kube-api-access-r7vv7\") pod \"authorino-58cd955675-c9gq9\" (UID: \"5a7101c1-c76b-4443-8202-62a68ed62d1f\") " pod="kuadrant-system/authorino-58cd955675-c9gq9" Jul 13 07:10:13.300248 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:13.300154 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7vv7\" (UniqueName: \"kubernetes.io/projected/5a7101c1-c76b-4443-8202-62a68ed62d1f-kube-api-access-r7vv7\") pod \"authorino-58cd955675-c9gq9\" (UID: \"5a7101c1-c76b-4443-8202-62a68ed62d1f\") " pod="kuadrant-system/authorino-58cd955675-c9gq9" Jul 13 07:10:13.307603 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:13.307573 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7vv7\" (UniqueName: \"kubernetes.io/projected/5a7101c1-c76b-4443-8202-62a68ed62d1f-kube-api-access-r7vv7\") pod \"authorino-58cd955675-c9gq9\" (UID: \"5a7101c1-c76b-4443-8202-62a68ed62d1f\") " pod="kuadrant-system/authorino-58cd955675-c9gq9" Jul 13 07:10:13.339892 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:13.339856 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-2rksc" event={"ID":"a405d03d-88ae-493b-b1e0-d7ca25f52246","Type":"ContainerStarted","Data":"cf64fa90a388bf1c0f8fce69110cf7143f9cc9c09832e6abfc727a67476bc73a"} Jul 13 07:10:13.377661 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:13.377623 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-xdnm2"] Jul 13 07:10:13.381241 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:13.381214 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-xdnm2" Jul 13 07:10:13.386728 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:13.386673 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-xdnm2"] Jul 13 07:10:13.400888 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:13.400857 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75bcj\" (UniqueName: \"kubernetes.io/projected/7df48fbf-e057-47f1-aa4a-f993586e3ec8-kube-api-access-75bcj\") pod \"authorino-75d556bd77-xdnm2\" (UID: \"7df48fbf-e057-47f1-aa4a-f993586e3ec8\") " pod="kuadrant-system/authorino-75d556bd77-xdnm2" Jul 13 07:10:13.437131 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:13.437090 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-c9gq9" Jul 13 07:10:13.501594 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:13.501495 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75bcj\" (UniqueName: \"kubernetes.io/projected/7df48fbf-e057-47f1-aa4a-f993586e3ec8-kube-api-access-75bcj\") pod \"authorino-75d556bd77-xdnm2\" (UID: \"7df48fbf-e057-47f1-aa4a-f993586e3ec8\") " pod="kuadrant-system/authorino-75d556bd77-xdnm2" Jul 13 07:10:13.519370 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:13.519340 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75bcj\" (UniqueName: \"kubernetes.io/projected/7df48fbf-e057-47f1-aa4a-f993586e3ec8-kube-api-access-75bcj\") pod \"authorino-75d556bd77-xdnm2\" (UID: \"7df48fbf-e057-47f1-aa4a-f993586e3ec8\") " pod="kuadrant-system/authorino-75d556bd77-xdnm2" Jul 13 07:10:13.594341 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:13.594316 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-c9gq9"] Jul 13 07:10:13.596115 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:10:13.596090 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5a7101c1_c76b_4443_8202_62a68ed62d1f.slice/crio-86f3d15845c6550d3642badb0f3b9b7cb0b6c47c095dc4d72d6066c52973d9a6 WatchSource:0}: Error finding container 86f3d15845c6550d3642badb0f3b9b7cb0b6c47c095dc4d72d6066c52973d9a6: Status 404 returned error can't find the container with id 86f3d15845c6550d3642badb0f3b9b7cb0b6c47c095dc4d72d6066c52973d9a6 Jul 13 07:10:13.692822 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:13.692776 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-xdnm2" Jul 13 07:10:13.860834 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:13.860742 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-xdnm2"] Jul 13 07:10:13.865166 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:10:13.865127 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7df48fbf_e057_47f1_aa4a_f993586e3ec8.slice/crio-a2a6bc8d9a0c30faa15ed360ca5124c0de1b8e702580464ed23351bd393d7e69 WatchSource:0}: Error finding container a2a6bc8d9a0c30faa15ed360ca5124c0de1b8e702580464ed23351bd393d7e69: Status 404 returned error can't find the container with id a2a6bc8d9a0c30faa15ed360ca5124c0de1b8e702580464ed23351bd393d7e69 Jul 13 07:10:14.351893 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:14.351852 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-xdnm2" event={"ID":"7df48fbf-e057-47f1-aa4a-f993586e3ec8","Type":"ContainerStarted","Data":"a2a6bc8d9a0c30faa15ed360ca5124c0de1b8e702580464ed23351bd393d7e69"} Jul 13 07:10:14.356513 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:14.356471 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-c9gq9" event={"ID":"5a7101c1-c76b-4443-8202-62a68ed62d1f","Type":"ContainerStarted","Data":"86f3d15845c6550d3642badb0f3b9b7cb0b6c47c095dc4d72d6066c52973d9a6"} Jul 13 07:10:18.389146 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:18.389095 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-xdnm2" event={"ID":"7df48fbf-e057-47f1-aa4a-f993586e3ec8","Type":"ContainerStarted","Data":"a401864a877a26fee09a3036a6348084562d697e1fd8657678cb4bf545cc480a"} Jul 13 07:10:18.391589 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:18.391555 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-c9gq9" event={"ID":"5a7101c1-c76b-4443-8202-62a68ed62d1f","Type":"ContainerStarted","Data":"bb7d8564b6d7f031796415408235f007e2b6545d43bac55be3a70ed107e4587c"} Jul 13 07:10:18.394089 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:18.394055 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-2rksc" event={"ID":"a405d03d-88ae-493b-b1e0-d7ca25f52246","Type":"ContainerStarted","Data":"5b462a9251066ec1ba2dea630a9afb71cc3fd9b1b9b8b2d9750c4df82d8dfc24"} Jul 13 07:10:18.394241 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:18.394215 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-2rksc" Jul 13 07:10:18.406006 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:18.405955 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-xdnm2" podStartSLOduration=1.038068225 podStartE2EDuration="5.405938907s" totalImagesPullingTime="4.367870682s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:10:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:10:18.404261558 +0000 UTC m=+493.239825391" watchObservedRunningTime="2026-07-13 07:10:18.405938907 +0000 UTC m=+493.241502737" Jul 13 07:10:18.420602 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:18.420546 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-2rksc" podStartSLOduration=1.27925895 podStartE2EDuration="6.420525251s" totalImagesPullingTime="5.141266301s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:10:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:10:18.417875096 +0000 UTC m=+493.253438918" watchObservedRunningTime="2026-07-13 07:10:18.420525251 +0000 UTC m=+493.256089083" Jul 13 07:10:18.428277 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:18.428248 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-c9gq9"] Jul 13 07:10:18.438950 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:18.438900 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-c9gq9" podStartSLOduration=0.802140718 podStartE2EDuration="5.438889903s" totalImagesPullingTime="4.636749185s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:10:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:10:18.437933082 +0000 UTC m=+493.273496913" watchObservedRunningTime="2026-07-13 07:10:18.438889903 +0000 UTC m=+493.274453735" Jul 13 07:10:20.405003 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:20.404961 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-c9gq9" podUID="5a7101c1-c76b-4443-8202-62a68ed62d1f" containerName="authorino" containerID="cri-o://bb7d8564b6d7f031796415408235f007e2b6545d43bac55be3a70ed107e4587c" gracePeriod=30 Jul 13 07:10:20.661830 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:20.661765 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-c9gq9" Jul 13 07:10:20.766530 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:20.766489 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r7vv7\" (UniqueName: \"kubernetes.io/projected/5a7101c1-c76b-4443-8202-62a68ed62d1f-kube-api-access-r7vv7\") pod \"5a7101c1-c76b-4443-8202-62a68ed62d1f\" (UID: \"5a7101c1-c76b-4443-8202-62a68ed62d1f\") " Jul 13 07:10:20.768715 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:20.768673 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a7101c1-c76b-4443-8202-62a68ed62d1f-kube-api-access-r7vv7" (OuterVolumeSpecName: "kube-api-access-r7vv7") pod "5a7101c1-c76b-4443-8202-62a68ed62d1f" (UID: "5a7101c1-c76b-4443-8202-62a68ed62d1f"). InnerVolumeSpecName "kube-api-access-r7vv7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:10:20.867322 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:20.867288 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r7vv7\" (UniqueName: \"kubernetes.io/projected/5a7101c1-c76b-4443-8202-62a68ed62d1f-kube-api-access-r7vv7\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:10:21.411960 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:21.411924 2586 generic.go:356] "Generic (PLEG): container finished" podID="5a7101c1-c76b-4443-8202-62a68ed62d1f" containerID="bb7d8564b6d7f031796415408235f007e2b6545d43bac55be3a70ed107e4587c" exitCode=0 Jul 13 07:10:21.412391 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:21.411964 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-c9gq9" Jul 13 07:10:21.412391 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:21.412001 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-c9gq9" event={"ID":"5a7101c1-c76b-4443-8202-62a68ed62d1f","Type":"ContainerDied","Data":"bb7d8564b6d7f031796415408235f007e2b6545d43bac55be3a70ed107e4587c"} Jul 13 07:10:21.412391 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:21.412043 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-c9gq9" event={"ID":"5a7101c1-c76b-4443-8202-62a68ed62d1f","Type":"ContainerDied","Data":"86f3d15845c6550d3642badb0f3b9b7cb0b6c47c095dc4d72d6066c52973d9a6"} Jul 13 07:10:21.412391 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:21.412060 2586 scope.go:117] "RemoveContainer" containerID="bb7d8564b6d7f031796415408235f007e2b6545d43bac55be3a70ed107e4587c" Jul 13 07:10:21.422518 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:21.422500 2586 scope.go:117] "RemoveContainer" containerID="bb7d8564b6d7f031796415408235f007e2b6545d43bac55be3a70ed107e4587c" Jul 13 07:10:21.422848 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:10:21.422826 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb7d8564b6d7f031796415408235f007e2b6545d43bac55be3a70ed107e4587c\": container with ID starting with bb7d8564b6d7f031796415408235f007e2b6545d43bac55be3a70ed107e4587c not found: ID does not exist" containerID="bb7d8564b6d7f031796415408235f007e2b6545d43bac55be3a70ed107e4587c" Jul 13 07:10:21.422893 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:21.422859 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb7d8564b6d7f031796415408235f007e2b6545d43bac55be3a70ed107e4587c"} err="failed to get container status \"bb7d8564b6d7f031796415408235f007e2b6545d43bac55be3a70ed107e4587c\": rpc error: code = NotFound desc = could not find container \"bb7d8564b6d7f031796415408235f007e2b6545d43bac55be3a70ed107e4587c\": container with ID starting with bb7d8564b6d7f031796415408235f007e2b6545d43bac55be3a70ed107e4587c not found: ID does not exist" Jul 13 07:10:21.438078 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:21.438048 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-c9gq9"] Jul 13 07:10:21.443347 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:21.443318 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-c9gq9"] Jul 13 07:10:21.805468 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:21.805417 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a7101c1-c76b-4443-8202-62a68ed62d1f" path="/var/lib/kubelet/pods/5a7101c1-c76b-4443-8202-62a68ed62d1f/volumes" Jul 13 07:10:29.401034 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:29.401002 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-2rksc" Jul 13 07:10:44.307847 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:44.307760 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-wf6kp"] Jul 13 07:10:44.308233 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:44.308128 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a7101c1-c76b-4443-8202-62a68ed62d1f" containerName="authorino" Jul 13 07:10:44.308233 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:44.308144 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a7101c1-c76b-4443-8202-62a68ed62d1f" containerName="authorino" Jul 13 07:10:44.308233 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:44.308225 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a7101c1-c76b-4443-8202-62a68ed62d1f" containerName="authorino" Jul 13 07:10:44.376226 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:44.376181 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-wf6kp"] Jul 13 07:10:44.376455 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:44.376304 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-wf6kp" Jul 13 07:10:44.477846 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:44.477806 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nncll\" (UniqueName: \"kubernetes.io/projected/adda321f-165c-4776-ba6c-4c7013a0e4ce-kube-api-access-nncll\") pod \"authorino-5d4b67c446-wf6kp\" (UID: \"adda321f-165c-4776-ba6c-4c7013a0e4ce\") " pod="kuadrant-system/authorino-5d4b67c446-wf6kp" Jul 13 07:10:44.493492 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:44.493453 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-wf6kp"] Jul 13 07:10:44.493777 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:10:44.493751 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-nncll], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-wf6kp" podUID="adda321f-165c-4776-ba6c-4c7013a0e4ce" Jul 13 07:10:44.532290 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:44.532257 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-wf6kp" Jul 13 07:10:44.561950 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:44.561867 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-wf6kp" Jul 13 07:10:44.579231 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:44.579189 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nncll\" (UniqueName: \"kubernetes.io/projected/adda321f-165c-4776-ba6c-4c7013a0e4ce-kube-api-access-nncll\") pod \"authorino-5d4b67c446-wf6kp\" (UID: \"adda321f-165c-4776-ba6c-4c7013a0e4ce\") " pod="kuadrant-system/authorino-5d4b67c446-wf6kp" Jul 13 07:10:44.589611 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:44.589584 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nncll\" (UniqueName: \"kubernetes.io/projected/adda321f-165c-4776-ba6c-4c7013a0e4ce-kube-api-access-nncll\") pod \"authorino-5d4b67c446-wf6kp\" (UID: \"adda321f-165c-4776-ba6c-4c7013a0e4ce\") " pod="kuadrant-system/authorino-5d4b67c446-wf6kp" Jul 13 07:10:44.606216 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:44.606179 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-548bc6d586-vpkhq"] Jul 13 07:10:44.630768 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:44.630726 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-548bc6d586-vpkhq"] Jul 13 07:10:44.630955 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:44.630840 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-548bc6d586-vpkhq" Jul 13 07:10:44.632978 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:44.632956 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 13 07:10:44.680615 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:44.680577 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nncll\" (UniqueName: \"kubernetes.io/projected/adda321f-165c-4776-ba6c-4c7013a0e4ce-kube-api-access-nncll\") pod \"adda321f-165c-4776-ba6c-4c7013a0e4ce\" (UID: \"adda321f-165c-4776-ba6c-4c7013a0e4ce\") " Jul 13 07:10:44.682868 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:44.682841 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/adda321f-165c-4776-ba6c-4c7013a0e4ce-kube-api-access-nncll" (OuterVolumeSpecName: "kube-api-access-nncll") pod "adda321f-165c-4776-ba6c-4c7013a0e4ce" (UID: "adda321f-165c-4776-ba6c-4c7013a0e4ce"). InnerVolumeSpecName "kube-api-access-nncll". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:10:44.782111 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:44.782071 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85xvt\" (UniqueName: \"kubernetes.io/projected/c40d2768-2409-4c45-99dc-ef0f02ae8370-kube-api-access-85xvt\") pod \"authorino-548bc6d586-vpkhq\" (UID: \"c40d2768-2409-4c45-99dc-ef0f02ae8370\") " pod="kuadrant-system/authorino-548bc6d586-vpkhq" Jul 13 07:10:44.782314 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:44.782201 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c40d2768-2409-4c45-99dc-ef0f02ae8370-tls-cert\") pod \"authorino-548bc6d586-vpkhq\" (UID: \"c40d2768-2409-4c45-99dc-ef0f02ae8370\") " pod="kuadrant-system/authorino-548bc6d586-vpkhq" Jul 13 07:10:44.782314 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:44.782246 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nncll\" (UniqueName: \"kubernetes.io/projected/adda321f-165c-4776-ba6c-4c7013a0e4ce-kube-api-access-nncll\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:10:44.883549 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:44.883444 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c40d2768-2409-4c45-99dc-ef0f02ae8370-tls-cert\") pod \"authorino-548bc6d586-vpkhq\" (UID: \"c40d2768-2409-4c45-99dc-ef0f02ae8370\") " pod="kuadrant-system/authorino-548bc6d586-vpkhq" Jul 13 07:10:44.883549 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:44.883493 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85xvt\" (UniqueName: \"kubernetes.io/projected/c40d2768-2409-4c45-99dc-ef0f02ae8370-kube-api-access-85xvt\") pod \"authorino-548bc6d586-vpkhq\" (UID: \"c40d2768-2409-4c45-99dc-ef0f02ae8370\") " pod="kuadrant-system/authorino-548bc6d586-vpkhq" Jul 13 07:10:44.886009 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:44.885983 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c40d2768-2409-4c45-99dc-ef0f02ae8370-tls-cert\") pod \"authorino-548bc6d586-vpkhq\" (UID: \"c40d2768-2409-4c45-99dc-ef0f02ae8370\") " pod="kuadrant-system/authorino-548bc6d586-vpkhq" Jul 13 07:10:44.891697 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:44.891658 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-85xvt\" (UniqueName: \"kubernetes.io/projected/c40d2768-2409-4c45-99dc-ef0f02ae8370-kube-api-access-85xvt\") pod \"authorino-548bc6d586-vpkhq\" (UID: \"c40d2768-2409-4c45-99dc-ef0f02ae8370\") " pod="kuadrant-system/authorino-548bc6d586-vpkhq" Jul 13 07:10:44.940519 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:44.940475 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-548bc6d586-vpkhq" Jul 13 07:10:45.096433 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:45.096399 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-548bc6d586-vpkhq"] Jul 13 07:10:45.097857 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:10:45.097831 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc40d2768_2409_4c45_99dc_ef0f02ae8370.slice/crio-124b98a9c9c46c3b3dac8fb15b148bbc7a9c64215f5013a6c74171fe7da7afdb WatchSource:0}: Error finding container 124b98a9c9c46c3b3dac8fb15b148bbc7a9c64215f5013a6c74171fe7da7afdb: Status 404 returned error can't find the container with id 124b98a9c9c46c3b3dac8fb15b148bbc7a9c64215f5013a6c74171fe7da7afdb Jul 13 07:10:45.539741 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:45.539707 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-548bc6d586-vpkhq" event={"ID":"c40d2768-2409-4c45-99dc-ef0f02ae8370","Type":"ContainerStarted","Data":"124b98a9c9c46c3b3dac8fb15b148bbc7a9c64215f5013a6c74171fe7da7afdb"} Jul 13 07:10:45.539741 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:45.539730 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-wf6kp" Jul 13 07:10:45.572391 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:45.572361 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-wf6kp"] Jul 13 07:10:45.574281 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:45.574251 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-wf6kp"] Jul 13 07:10:45.805229 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:45.805151 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="adda321f-165c-4776-ba6c-4c7013a0e4ce" path="/var/lib/kubelet/pods/adda321f-165c-4776-ba6c-4c7013a0e4ce/volumes" Jul 13 07:10:46.546310 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:46.546275 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-548bc6d586-vpkhq" event={"ID":"c40d2768-2409-4c45-99dc-ef0f02ae8370","Type":"ContainerStarted","Data":"0570ba86b862228c53e2fc8a3b2c33801e4d8251f1f16243fb063f0004d62e8b"} Jul 13 07:10:46.559249 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:46.559204 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-548bc6d586-vpkhq" podStartSLOduration=2.124427725 podStartE2EDuration="2.559189864s" totalImagesPullingTime="434.762139ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:10:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:10:46.558845517 +0000 UTC m=+521.394409353" watchObservedRunningTime="2026-07-13 07:10:46.559189864 +0000 UTC m=+521.394753694" Jul 13 07:10:46.582907 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:46.582874 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-xdnm2"] Jul 13 07:10:46.583110 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:46.583086 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-xdnm2" podUID="7df48fbf-e057-47f1-aa4a-f993586e3ec8" containerName="authorino" containerID="cri-o://a401864a877a26fee09a3036a6348084562d697e1fd8657678cb4bf545cc480a" gracePeriod=30 Jul 13 07:10:46.839812 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:46.839788 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-xdnm2" Jul 13 07:10:47.002165 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:47.002134 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-75bcj\" (UniqueName: \"kubernetes.io/projected/7df48fbf-e057-47f1-aa4a-f993586e3ec8-kube-api-access-75bcj\") pod \"7df48fbf-e057-47f1-aa4a-f993586e3ec8\" (UID: \"7df48fbf-e057-47f1-aa4a-f993586e3ec8\") " Jul 13 07:10:47.004188 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:47.004160 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7df48fbf-e057-47f1-aa4a-f993586e3ec8-kube-api-access-75bcj" (OuterVolumeSpecName: "kube-api-access-75bcj") pod "7df48fbf-e057-47f1-aa4a-f993586e3ec8" (UID: "7df48fbf-e057-47f1-aa4a-f993586e3ec8"). InnerVolumeSpecName "kube-api-access-75bcj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:10:47.103282 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:47.103201 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-75bcj\" (UniqueName: \"kubernetes.io/projected/7df48fbf-e057-47f1-aa4a-f993586e3ec8-kube-api-access-75bcj\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:10:47.553864 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:47.553830 2586 generic.go:356] "Generic (PLEG): container finished" podID="7df48fbf-e057-47f1-aa4a-f993586e3ec8" containerID="a401864a877a26fee09a3036a6348084562d697e1fd8657678cb4bf545cc480a" exitCode=0 Jul 13 07:10:47.554328 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:47.553870 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-xdnm2" event={"ID":"7df48fbf-e057-47f1-aa4a-f993586e3ec8","Type":"ContainerDied","Data":"a401864a877a26fee09a3036a6348084562d697e1fd8657678cb4bf545cc480a"} Jul 13 07:10:47.554328 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:47.553928 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-xdnm2" event={"ID":"7df48fbf-e057-47f1-aa4a-f993586e3ec8","Type":"ContainerDied","Data":"a2a6bc8d9a0c30faa15ed360ca5124c0de1b8e702580464ed23351bd393d7e69"} Jul 13 07:10:47.554328 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:47.553954 2586 scope.go:117] "RemoveContainer" containerID="a401864a877a26fee09a3036a6348084562d697e1fd8657678cb4bf545cc480a" Jul 13 07:10:47.554328 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:47.553886 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-xdnm2" Jul 13 07:10:47.563097 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:47.563081 2586 scope.go:117] "RemoveContainer" containerID="a401864a877a26fee09a3036a6348084562d697e1fd8657678cb4bf545cc480a" Jul 13 07:10:47.563345 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:10:47.563326 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a401864a877a26fee09a3036a6348084562d697e1fd8657678cb4bf545cc480a\": container with ID starting with a401864a877a26fee09a3036a6348084562d697e1fd8657678cb4bf545cc480a not found: ID does not exist" containerID="a401864a877a26fee09a3036a6348084562d697e1fd8657678cb4bf545cc480a" Jul 13 07:10:47.563396 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:47.563356 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a401864a877a26fee09a3036a6348084562d697e1fd8657678cb4bf545cc480a"} err="failed to get container status \"a401864a877a26fee09a3036a6348084562d697e1fd8657678cb4bf545cc480a\": rpc error: code = NotFound desc = could not find container \"a401864a877a26fee09a3036a6348084562d697e1fd8657678cb4bf545cc480a\": container with ID starting with a401864a877a26fee09a3036a6348084562d697e1fd8657678cb4bf545cc480a not found: ID does not exist" Jul 13 07:10:47.573846 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:47.573827 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-xdnm2"] Jul 13 07:10:47.577567 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:47.577549 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-xdnm2"] Jul 13 07:10:47.805261 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:10:47.805183 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7df48fbf-e057-47f1-aa4a-f993586e3ec8" path="/var/lib/kubelet/pods/7df48fbf-e057-47f1-aa4a-f993586e3ec8/volumes" Jul 13 07:11:25.668653 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:11:25.668504 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:11:49.750561 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:11:49.750519 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:11:59.966177 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:11:59.966137 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-hc2sg"] Jul 13 07:11:59.970122 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:11:59.967091 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7df48fbf-e057-47f1-aa4a-f993586e3ec8" containerName="authorino" Jul 13 07:11:59.970122 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:11:59.967143 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="7df48fbf-e057-47f1-aa4a-f993586e3ec8" containerName="authorino" Jul 13 07:11:59.970122 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:11:59.967403 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="7df48fbf-e057-47f1-aa4a-f993586e3ec8" containerName="authorino" Jul 13 07:11:59.974180 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:11:59.974157 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hc2sg" Jul 13 07:11:59.981437 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:11:59.981415 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 13 07:11:59.981545 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:11:59.981415 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-2zk6k\"" Jul 13 07:11:59.981545 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:11:59.981490 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 13 07:11:59.981545 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:11:59.981494 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 13 07:11:59.986176 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:11:59.986151 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-hc2sg"] Jul 13 07:12:00.096794 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:00.096760 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkpd9\" (UniqueName: \"kubernetes.io/projected/72534e7c-99e2-4679-a199-6eb856bcf30f-kube-api-access-xkpd9\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hc2sg\" (UID: \"72534e7c-99e2-4679-a199-6eb856bcf30f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hc2sg" Jul 13 07:12:00.096972 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:00.096811 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72534e7c-99e2-4679-a199-6eb856bcf30f-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hc2sg\" (UID: \"72534e7c-99e2-4679-a199-6eb856bcf30f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hc2sg" Jul 13 07:12:00.096972 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:00.096830 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/72534e7c-99e2-4679-a199-6eb856bcf30f-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hc2sg\" (UID: \"72534e7c-99e2-4679-a199-6eb856bcf30f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hc2sg" Jul 13 07:12:00.096972 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:00.096944 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/72534e7c-99e2-4679-a199-6eb856bcf30f-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hc2sg\" (UID: \"72534e7c-99e2-4679-a199-6eb856bcf30f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hc2sg" Jul 13 07:12:00.097100 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:00.096986 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72534e7c-99e2-4679-a199-6eb856bcf30f-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hc2sg\" (UID: \"72534e7c-99e2-4679-a199-6eb856bcf30f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hc2sg" Jul 13 07:12:00.097100 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:00.097011 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/72534e7c-99e2-4679-a199-6eb856bcf30f-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hc2sg\" (UID: \"72534e7c-99e2-4679-a199-6eb856bcf30f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hc2sg" Jul 13 07:12:00.197815 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:00.197778 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/72534e7c-99e2-4679-a199-6eb856bcf30f-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hc2sg\" (UID: \"72534e7c-99e2-4679-a199-6eb856bcf30f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hc2sg" Jul 13 07:12:00.197985 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:00.197827 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkpd9\" (UniqueName: \"kubernetes.io/projected/72534e7c-99e2-4679-a199-6eb856bcf30f-kube-api-access-xkpd9\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hc2sg\" (UID: \"72534e7c-99e2-4679-a199-6eb856bcf30f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hc2sg" Jul 13 07:12:00.197985 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:00.197869 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72534e7c-99e2-4679-a199-6eb856bcf30f-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hc2sg\" (UID: \"72534e7c-99e2-4679-a199-6eb856bcf30f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hc2sg" Jul 13 07:12:00.197985 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:00.197887 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/72534e7c-99e2-4679-a199-6eb856bcf30f-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hc2sg\" (UID: \"72534e7c-99e2-4679-a199-6eb856bcf30f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hc2sg" Jul 13 07:12:00.197985 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:00.197932 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/72534e7c-99e2-4679-a199-6eb856bcf30f-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hc2sg\" (UID: \"72534e7c-99e2-4679-a199-6eb856bcf30f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hc2sg" Jul 13 07:12:00.198135 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:00.198041 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72534e7c-99e2-4679-a199-6eb856bcf30f-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hc2sg\" (UID: \"72534e7c-99e2-4679-a199-6eb856bcf30f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hc2sg" Jul 13 07:12:00.198270 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:00.198245 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/72534e7c-99e2-4679-a199-6eb856bcf30f-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hc2sg\" (UID: \"72534e7c-99e2-4679-a199-6eb856bcf30f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hc2sg" Jul 13 07:12:00.198340 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:00.198312 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72534e7c-99e2-4679-a199-6eb856bcf30f-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hc2sg\" (UID: \"72534e7c-99e2-4679-a199-6eb856bcf30f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hc2sg" Jul 13 07:12:00.198376 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:00.198363 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/72534e7c-99e2-4679-a199-6eb856bcf30f-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hc2sg\" (UID: \"72534e7c-99e2-4679-a199-6eb856bcf30f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hc2sg" Jul 13 07:12:00.200158 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:00.200139 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/72534e7c-99e2-4679-a199-6eb856bcf30f-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hc2sg\" (UID: \"72534e7c-99e2-4679-a199-6eb856bcf30f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hc2sg" Jul 13 07:12:00.200403 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:00.200387 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72534e7c-99e2-4679-a199-6eb856bcf30f-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hc2sg\" (UID: \"72534e7c-99e2-4679-a199-6eb856bcf30f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hc2sg" Jul 13 07:12:00.207941 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:00.207918 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkpd9\" (UniqueName: \"kubernetes.io/projected/72534e7c-99e2-4679-a199-6eb856bcf30f-kube-api-access-xkpd9\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hc2sg\" (UID: \"72534e7c-99e2-4679-a199-6eb856bcf30f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hc2sg" Jul 13 07:12:00.285729 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:00.285621 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hc2sg" Jul 13 07:12:00.435025 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:00.434997 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-hc2sg"] Jul 13 07:12:00.436858 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:12:00.436815 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72534e7c_99e2_4679_a199_6eb856bcf30f.slice/crio-9d115ce65be07ea487f18fb75e8d244df53b1ff7f77dac3dd89ef5fb7320520b WatchSource:0}: Error finding container 9d115ce65be07ea487f18fb75e8d244df53b1ff7f77dac3dd89ef5fb7320520b: Status 404 returned error can't find the container with id 9d115ce65be07ea487f18fb75e8d244df53b1ff7f77dac3dd89ef5fb7320520b Jul 13 07:12:00.947260 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:00.947182 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hc2sg" event={"ID":"72534e7c-99e2-4679-a199-6eb856bcf30f","Type":"ContainerStarted","Data":"9d115ce65be07ea487f18fb75e8d244df53b1ff7f77dac3dd89ef5fb7320520b"} Jul 13 07:12:01.168035 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:01.167993 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:12:01.954022 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:01.953984 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hc2sg" event={"ID":"72534e7c-99e2-4679-a199-6eb856bcf30f","Type":"ContainerStarted","Data":"e0994efad3de071e3f58bbc2fad2d35b7a403d610a4e891a91f6b8a17bb7d210"} Jul 13 07:12:01.972391 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:01.972343 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hc2sg" podStartSLOduration=1.831929448 podStartE2EDuration="2.972328608s" totalImagesPullingTime="1.14039916s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:11:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:12:01.971546189 +0000 UTC m=+596.807110019" watchObservedRunningTime="2026-07-13 07:12:01.972328608 +0000 UTC m=+596.807892438" Jul 13 07:12:05.659148 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:05.659116 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-r878q_293edc99-2691-4915-ba03-7407d88b89b4/console-operator/1.log" Jul 13 07:12:05.659553 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:05.659536 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-r878q_293edc99-2691-4915-ba03-7407d88b89b4/console-operator/1.log" Jul 13 07:12:05.662968 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:05.662945 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7b964_3d71897a-4b2e-4e4e-9525-ccd89bd18458/ovn-acl-logging/0.log" Jul 13 07:12:05.663577 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:05.663561 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7b964_3d71897a-4b2e-4e4e-9525-ccd89bd18458/ovn-acl-logging/0.log" Jul 13 07:12:08.446169 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:08.446137 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:12:10.286342 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:10.286298 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hc2sg" Jul 13 07:12:10.286828 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:10.286385 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hc2sg" Jul 13 07:12:10.299553 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:10.299523 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hc2sg" Jul 13 07:12:11.015152 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:11.015122 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hc2sg" Jul 13 07:12:15.061635 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:15.061595 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:12:28.455014 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:28.454935 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-6zwph"] Jul 13 07:12:28.460218 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:28.460174 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6zwph" Jul 13 07:12:28.466214 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:28.466172 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 13 07:12:28.470922 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:28.470898 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-6zwph"] Jul 13 07:12:28.544279 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:28.544241 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81753bcc-9288-4af8-b919-86c11dfa654c-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6zwph\" (UID: \"81753bcc-9288-4af8-b919-86c11dfa654c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6zwph" Jul 13 07:12:28.544279 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:28.544282 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81753bcc-9288-4af8-b919-86c11dfa654c-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6zwph\" (UID: \"81753bcc-9288-4af8-b919-86c11dfa654c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6zwph" Jul 13 07:12:28.544476 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:28.544300 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81753bcc-9288-4af8-b919-86c11dfa654c-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6zwph\" (UID: \"81753bcc-9288-4af8-b919-86c11dfa654c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6zwph" Jul 13 07:12:28.544476 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:28.544348 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81753bcc-9288-4af8-b919-86c11dfa654c-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6zwph\" (UID: \"81753bcc-9288-4af8-b919-86c11dfa654c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6zwph" Jul 13 07:12:28.544476 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:28.544364 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81753bcc-9288-4af8-b919-86c11dfa654c-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6zwph\" (UID: \"81753bcc-9288-4af8-b919-86c11dfa654c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6zwph" Jul 13 07:12:28.544476 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:28.544385 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjxjv\" (UniqueName: \"kubernetes.io/projected/81753bcc-9288-4af8-b919-86c11dfa654c-kube-api-access-gjxjv\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6zwph\" (UID: \"81753bcc-9288-4af8-b919-86c11dfa654c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6zwph" Jul 13 07:12:28.645584 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:28.645539 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81753bcc-9288-4af8-b919-86c11dfa654c-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6zwph\" (UID: \"81753bcc-9288-4af8-b919-86c11dfa654c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6zwph" Jul 13 07:12:28.645825 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:28.645604 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81753bcc-9288-4af8-b919-86c11dfa654c-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6zwph\" (UID: \"81753bcc-9288-4af8-b919-86c11dfa654c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6zwph" Jul 13 07:12:28.645825 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:28.645635 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81753bcc-9288-4af8-b919-86c11dfa654c-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6zwph\" (UID: \"81753bcc-9288-4af8-b919-86c11dfa654c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6zwph" Jul 13 07:12:28.645825 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:28.645675 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81753bcc-9288-4af8-b919-86c11dfa654c-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6zwph\" (UID: \"81753bcc-9288-4af8-b919-86c11dfa654c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6zwph" Jul 13 07:12:28.645825 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:28.645733 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81753bcc-9288-4af8-b919-86c11dfa654c-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6zwph\" (UID: \"81753bcc-9288-4af8-b919-86c11dfa654c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6zwph" Jul 13 07:12:28.645825 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:28.645764 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjxjv\" (UniqueName: \"kubernetes.io/projected/81753bcc-9288-4af8-b919-86c11dfa654c-kube-api-access-gjxjv\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6zwph\" (UID: \"81753bcc-9288-4af8-b919-86c11dfa654c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6zwph" Jul 13 07:12:28.646082 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:28.646020 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81753bcc-9288-4af8-b919-86c11dfa654c-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6zwph\" (UID: \"81753bcc-9288-4af8-b919-86c11dfa654c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6zwph" Jul 13 07:12:28.646140 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:28.646090 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81753bcc-9288-4af8-b919-86c11dfa654c-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6zwph\" (UID: \"81753bcc-9288-4af8-b919-86c11dfa654c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6zwph" Jul 13 07:12:28.646140 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:28.646114 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81753bcc-9288-4af8-b919-86c11dfa654c-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6zwph\" (UID: \"81753bcc-9288-4af8-b919-86c11dfa654c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6zwph" Jul 13 07:12:28.648501 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:28.648471 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81753bcc-9288-4af8-b919-86c11dfa654c-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6zwph\" (UID: \"81753bcc-9288-4af8-b919-86c11dfa654c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6zwph" Jul 13 07:12:28.648736 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:28.648720 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81753bcc-9288-4af8-b919-86c11dfa654c-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6zwph\" (UID: \"81753bcc-9288-4af8-b919-86c11dfa654c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6zwph" Jul 13 07:12:28.655344 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:28.655321 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjxjv\" (UniqueName: \"kubernetes.io/projected/81753bcc-9288-4af8-b919-86c11dfa654c-kube-api-access-gjxjv\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6zwph\" (UID: \"81753bcc-9288-4af8-b919-86c11dfa654c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6zwph" Jul 13 07:12:28.773968 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:28.773887 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6zwph" Jul 13 07:12:28.910857 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:28.910832 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-6zwph"] Jul 13 07:12:28.913218 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:12:28.913194 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81753bcc_9288_4af8_b919_86c11dfa654c.slice/crio-0e719a893e8abacb7e95f7e5331eac70c68f6ab4144bf3157944788c13ee28bf WatchSource:0}: Error finding container 0e719a893e8abacb7e95f7e5331eac70c68f6ab4144bf3157944788c13ee28bf: Status 404 returned error can't find the container with id 0e719a893e8abacb7e95f7e5331eac70c68f6ab4144bf3157944788c13ee28bf Jul 13 07:12:28.914430 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:28.914413 2586 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 07:12:29.110513 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:29.110484 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6zwph" event={"ID":"81753bcc-9288-4af8-b919-86c11dfa654c","Type":"ContainerStarted","Data":"0e719a893e8abacb7e95f7e5331eac70c68f6ab4144bf3157944788c13ee28bf"} Jul 13 07:12:30.116921 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:30.116885 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6zwph" event={"ID":"81753bcc-9288-4af8-b919-86c11dfa654c","Type":"ContainerStarted","Data":"2fdf9d19e02c96aab0e4bafee90cad5cd015103acf2a9a6b3194fff2d91c90ac"} Jul 13 07:12:30.135981 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:30.135933 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6zwph" podStartSLOduration=1.960161569 podStartE2EDuration="2.135915975s" totalImagesPullingTime="175.754406ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:12:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:12:30.132700127 +0000 UTC m=+624.968263949" watchObservedRunningTime="2026-07-13 07:12:30.135915975 +0000 UTC m=+624.971479805" Jul 13 07:12:30.854553 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:30.854519 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:12:38.774624 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:38.774589 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6zwph" Jul 13 07:12:38.775166 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:38.774644 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6zwph" Jul 13 07:12:38.788632 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:38.788609 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6zwph" Jul 13 07:12:39.177457 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:39.177368 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6zwph" Jul 13 07:12:53.753925 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:53.753887 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k5fkk"] Jul 13 07:12:53.760289 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:53.760267 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k5fkk" Jul 13 07:12:53.762587 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:53.762564 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 13 07:12:53.766368 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:53.766343 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k5fkk"] Jul 13 07:12:53.874092 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:53.874057 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrg26\" (UniqueName: \"kubernetes.io/projected/d3639c44-bb55-4d7f-a0b3-323e86441bb1-kube-api-access-nrg26\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k5fkk\" (UID: \"d3639c44-bb55-4d7f-a0b3-323e86441bb1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k5fkk" Jul 13 07:12:53.874256 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:53.874101 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3639c44-bb55-4d7f-a0b3-323e86441bb1-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k5fkk\" (UID: \"d3639c44-bb55-4d7f-a0b3-323e86441bb1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k5fkk" Jul 13 07:12:53.874256 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:53.874174 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3639c44-bb55-4d7f-a0b3-323e86441bb1-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k5fkk\" (UID: \"d3639c44-bb55-4d7f-a0b3-323e86441bb1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k5fkk" Jul 13 07:12:53.874256 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:53.874192 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3639c44-bb55-4d7f-a0b3-323e86441bb1-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k5fkk\" (UID: \"d3639c44-bb55-4d7f-a0b3-323e86441bb1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k5fkk" Jul 13 07:12:53.874256 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:53.874208 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3639c44-bb55-4d7f-a0b3-323e86441bb1-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k5fkk\" (UID: \"d3639c44-bb55-4d7f-a0b3-323e86441bb1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k5fkk" Jul 13 07:12:53.874256 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:53.874223 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3639c44-bb55-4d7f-a0b3-323e86441bb1-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k5fkk\" (UID: \"d3639c44-bb55-4d7f-a0b3-323e86441bb1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k5fkk" Jul 13 07:12:53.975195 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:53.975156 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrg26\" (UniqueName: \"kubernetes.io/projected/d3639c44-bb55-4d7f-a0b3-323e86441bb1-kube-api-access-nrg26\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k5fkk\" (UID: \"d3639c44-bb55-4d7f-a0b3-323e86441bb1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k5fkk" Jul 13 07:12:53.975421 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:53.975224 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3639c44-bb55-4d7f-a0b3-323e86441bb1-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k5fkk\" (UID: \"d3639c44-bb55-4d7f-a0b3-323e86441bb1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k5fkk" Jul 13 07:12:53.975551 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:53.975507 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3639c44-bb55-4d7f-a0b3-323e86441bb1-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k5fkk\" (UID: \"d3639c44-bb55-4d7f-a0b3-323e86441bb1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k5fkk" Jul 13 07:12:53.975624 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:53.975572 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3639c44-bb55-4d7f-a0b3-323e86441bb1-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k5fkk\" (UID: \"d3639c44-bb55-4d7f-a0b3-323e86441bb1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k5fkk" Jul 13 07:12:53.975624 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:53.975603 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3639c44-bb55-4d7f-a0b3-323e86441bb1-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k5fkk\" (UID: \"d3639c44-bb55-4d7f-a0b3-323e86441bb1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k5fkk" Jul 13 07:12:53.975624 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:53.975614 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3639c44-bb55-4d7f-a0b3-323e86441bb1-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k5fkk\" (UID: \"d3639c44-bb55-4d7f-a0b3-323e86441bb1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k5fkk" Jul 13 07:12:53.975817 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:53.975640 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3639c44-bb55-4d7f-a0b3-323e86441bb1-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k5fkk\" (UID: \"d3639c44-bb55-4d7f-a0b3-323e86441bb1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k5fkk" Jul 13 07:12:53.975871 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:53.975813 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3639c44-bb55-4d7f-a0b3-323e86441bb1-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k5fkk\" (UID: \"d3639c44-bb55-4d7f-a0b3-323e86441bb1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k5fkk" Jul 13 07:12:53.976204 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:53.976180 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3639c44-bb55-4d7f-a0b3-323e86441bb1-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k5fkk\" (UID: \"d3639c44-bb55-4d7f-a0b3-323e86441bb1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k5fkk" Jul 13 07:12:53.977901 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:53.977881 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3639c44-bb55-4d7f-a0b3-323e86441bb1-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k5fkk\" (UID: \"d3639c44-bb55-4d7f-a0b3-323e86441bb1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k5fkk" Jul 13 07:12:53.978145 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:53.978127 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3639c44-bb55-4d7f-a0b3-323e86441bb1-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k5fkk\" (UID: \"d3639c44-bb55-4d7f-a0b3-323e86441bb1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k5fkk" Jul 13 07:12:53.983737 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:53.983713 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrg26\" (UniqueName: \"kubernetes.io/projected/d3639c44-bb55-4d7f-a0b3-323e86441bb1-kube-api-access-nrg26\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k5fkk\" (UID: \"d3639c44-bb55-4d7f-a0b3-323e86441bb1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k5fkk" Jul 13 07:12:54.072009 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:54.071913 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k5fkk" Jul 13 07:12:54.214039 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:54.214005 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k5fkk"] Jul 13 07:12:54.215439 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:12:54.215412 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd3639c44_bb55_4d7f_a0b3_323e86441bb1.slice/crio-64c2a07442403318e3b79c0c2b910a0f41f489c997540a677d04e2361fd81cc5 WatchSource:0}: Error finding container 64c2a07442403318e3b79c0c2b910a0f41f489c997540a677d04e2361fd81cc5: Status 404 returned error can't find the container with id 64c2a07442403318e3b79c0c2b910a0f41f489c997540a677d04e2361fd81cc5 Jul 13 07:12:54.256108 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:54.256072 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k5fkk" event={"ID":"d3639c44-bb55-4d7f-a0b3-323e86441bb1","Type":"ContainerStarted","Data":"64c2a07442403318e3b79c0c2b910a0f41f489c997540a677d04e2361fd81cc5"} Jul 13 07:12:55.263426 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:55.263387 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k5fkk" event={"ID":"d3639c44-bb55-4d7f-a0b3-323e86441bb1","Type":"ContainerStarted","Data":"86170e3924fa07db575d6ce8912d75d769b0458717311dcfc293615090e94979"} Jul 13 07:12:55.284298 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:55.284229 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k5fkk" podStartSLOduration=2.100761067 podStartE2EDuration="2.284207274s" totalImagesPullingTime="183.446207ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:12:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:12:55.280629922 +0000 UTC m=+650.116193750" watchObservedRunningTime="2026-07-13 07:12:55.284207274 +0000 UTC m=+650.119771106" Jul 13 07:12:56.445834 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:12:56.445799 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:13:04.072532 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:13:04.072494 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k5fkk" Jul 13 07:13:04.072970 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:13:04.072547 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k5fkk" Jul 13 07:13:04.085329 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:13:04.085304 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k5fkk" Jul 13 07:13:04.327440 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:13:04.327359 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k5fkk" Jul 13 07:13:43.470489 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:13:43.470452 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-64cb9bbcff-xcvzz"] Jul 13 07:13:43.474103 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:13:43.474082 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-64cb9bbcff-xcvzz" Jul 13 07:13:43.480981 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:13:43.480938 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-64cb9bbcff-xcvzz"] Jul 13 07:13:43.589798 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:13:43.589762 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e5df704d-8a0d-486f-b395-8ff096d648de-tls-cert\") pod \"authorino-64cb9bbcff-xcvzz\" (UID: \"e5df704d-8a0d-486f-b395-8ff096d648de\") " pod="kuadrant-system/authorino-64cb9bbcff-xcvzz" Jul 13 07:13:43.589995 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:13:43.589831 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22ct7\" (UniqueName: \"kubernetes.io/projected/e5df704d-8a0d-486f-b395-8ff096d648de-kube-api-access-22ct7\") pod \"authorino-64cb9bbcff-xcvzz\" (UID: \"e5df704d-8a0d-486f-b395-8ff096d648de\") " pod="kuadrant-system/authorino-64cb9bbcff-xcvzz" Jul 13 07:13:43.691141 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:13:43.691094 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e5df704d-8a0d-486f-b395-8ff096d648de-tls-cert\") pod \"authorino-64cb9bbcff-xcvzz\" (UID: \"e5df704d-8a0d-486f-b395-8ff096d648de\") " pod="kuadrant-system/authorino-64cb9bbcff-xcvzz" Jul 13 07:13:43.691316 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:13:43.691195 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22ct7\" (UniqueName: \"kubernetes.io/projected/e5df704d-8a0d-486f-b395-8ff096d648de-kube-api-access-22ct7\") pod \"authorino-64cb9bbcff-xcvzz\" (UID: \"e5df704d-8a0d-486f-b395-8ff096d648de\") " pod="kuadrant-system/authorino-64cb9bbcff-xcvzz" Jul 13 07:13:43.693744 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:13:43.693717 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e5df704d-8a0d-486f-b395-8ff096d648de-tls-cert\") pod \"authorino-64cb9bbcff-xcvzz\" (UID: \"e5df704d-8a0d-486f-b395-8ff096d648de\") " pod="kuadrant-system/authorino-64cb9bbcff-xcvzz" Jul 13 07:13:43.698277 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:13:43.698247 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22ct7\" (UniqueName: \"kubernetes.io/projected/e5df704d-8a0d-486f-b395-8ff096d648de-kube-api-access-22ct7\") pod \"authorino-64cb9bbcff-xcvzz\" (UID: \"e5df704d-8a0d-486f-b395-8ff096d648de\") " pod="kuadrant-system/authorino-64cb9bbcff-xcvzz" Jul 13 07:13:43.784502 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:13:43.784414 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-64cb9bbcff-xcvzz" Jul 13 07:13:43.934858 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:13:43.934830 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-64cb9bbcff-xcvzz"] Jul 13 07:13:43.936451 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:13:43.936421 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode5df704d_8a0d_486f_b395_8ff096d648de.slice/crio-5d48b054f932c4b4465e98865c7fb0175c525e817209e622b1b8ae6aeeff3e54 WatchSource:0}: Error finding container 5d48b054f932c4b4465e98865c7fb0175c525e817209e622b1b8ae6aeeff3e54: Status 404 returned error can't find the container with id 5d48b054f932c4b4465e98865c7fb0175c525e817209e622b1b8ae6aeeff3e54 Jul 13 07:13:44.547570 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:13:44.547528 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-64cb9bbcff-xcvzz" event={"ID":"e5df704d-8a0d-486f-b395-8ff096d648de","Type":"ContainerStarted","Data":"9f3f45cee2dd0884df12273ff3b5608a0db6052ea0e4a0bae2c1ae4321163e6c"} Jul 13 07:13:44.547570 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:13:44.547578 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-64cb9bbcff-xcvzz" event={"ID":"e5df704d-8a0d-486f-b395-8ff096d648de","Type":"ContainerStarted","Data":"5d48b054f932c4b4465e98865c7fb0175c525e817209e622b1b8ae6aeeff3e54"} Jul 13 07:13:44.563499 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:13:44.563441 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-64cb9bbcff-xcvzz" podStartSLOduration=1.12246512 podStartE2EDuration="1.563422886s" totalImagesPullingTime="440.957766ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:13:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:13:44.561592529 +0000 UTC m=+699.397156373" watchObservedRunningTime="2026-07-13 07:13:44.563422886 +0000 UTC m=+699.398986716" Jul 13 07:13:44.597694 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:13:44.597644 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-548bc6d586-vpkhq"] Jul 13 07:13:44.597943 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:13:44.597886 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-548bc6d586-vpkhq" podUID="c40d2768-2409-4c45-99dc-ef0f02ae8370" containerName="authorino" containerID="cri-o://0570ba86b862228c53e2fc8a3b2c33801e4d8251f1f16243fb063f0004d62e8b" gracePeriod=30 Jul 13 07:13:44.865414 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:13:44.865378 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-548bc6d586-vpkhq" Jul 13 07:13:45.002676 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:13:45.002623 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-85xvt\" (UniqueName: \"kubernetes.io/projected/c40d2768-2409-4c45-99dc-ef0f02ae8370-kube-api-access-85xvt\") pod \"c40d2768-2409-4c45-99dc-ef0f02ae8370\" (UID: \"c40d2768-2409-4c45-99dc-ef0f02ae8370\") " Jul 13 07:13:45.002932 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:13:45.002739 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c40d2768-2409-4c45-99dc-ef0f02ae8370-tls-cert\") pod \"c40d2768-2409-4c45-99dc-ef0f02ae8370\" (UID: \"c40d2768-2409-4c45-99dc-ef0f02ae8370\") " Jul 13 07:13:45.004978 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:13:45.004940 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c40d2768-2409-4c45-99dc-ef0f02ae8370-kube-api-access-85xvt" (OuterVolumeSpecName: "kube-api-access-85xvt") pod "c40d2768-2409-4c45-99dc-ef0f02ae8370" (UID: "c40d2768-2409-4c45-99dc-ef0f02ae8370"). InnerVolumeSpecName "kube-api-access-85xvt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:13:45.014199 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:13:45.014166 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c40d2768-2409-4c45-99dc-ef0f02ae8370-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "c40d2768-2409-4c45-99dc-ef0f02ae8370" (UID: "c40d2768-2409-4c45-99dc-ef0f02ae8370"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 07:13:45.104355 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:13:45.104321 2586 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c40d2768-2409-4c45-99dc-ef0f02ae8370-tls-cert\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:13:45.104355 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:13:45.104354 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-85xvt\" (UniqueName: \"kubernetes.io/projected/c40d2768-2409-4c45-99dc-ef0f02ae8370-kube-api-access-85xvt\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:13:45.554326 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:13:45.554291 2586 generic.go:356] "Generic (PLEG): container finished" podID="c40d2768-2409-4c45-99dc-ef0f02ae8370" containerID="0570ba86b862228c53e2fc8a3b2c33801e4d8251f1f16243fb063f0004d62e8b" exitCode=0 Jul 13 07:13:45.554747 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:13:45.554335 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-548bc6d586-vpkhq" Jul 13 07:13:45.554747 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:13:45.554375 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-548bc6d586-vpkhq" event={"ID":"c40d2768-2409-4c45-99dc-ef0f02ae8370","Type":"ContainerDied","Data":"0570ba86b862228c53e2fc8a3b2c33801e4d8251f1f16243fb063f0004d62e8b"} Jul 13 07:13:45.554747 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:13:45.554420 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-548bc6d586-vpkhq" event={"ID":"c40d2768-2409-4c45-99dc-ef0f02ae8370","Type":"ContainerDied","Data":"124b98a9c9c46c3b3dac8fb15b148bbc7a9c64215f5013a6c74171fe7da7afdb"} Jul 13 07:13:45.554747 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:13:45.554441 2586 scope.go:117] "RemoveContainer" containerID="0570ba86b862228c53e2fc8a3b2c33801e4d8251f1f16243fb063f0004d62e8b" Jul 13 07:13:45.565107 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:13:45.565081 2586 scope.go:117] "RemoveContainer" containerID="0570ba86b862228c53e2fc8a3b2c33801e4d8251f1f16243fb063f0004d62e8b" Jul 13 07:13:45.565384 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:13:45.565367 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0570ba86b862228c53e2fc8a3b2c33801e4d8251f1f16243fb063f0004d62e8b\": container with ID starting with 0570ba86b862228c53e2fc8a3b2c33801e4d8251f1f16243fb063f0004d62e8b not found: ID does not exist" containerID="0570ba86b862228c53e2fc8a3b2c33801e4d8251f1f16243fb063f0004d62e8b" Jul 13 07:13:45.565433 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:13:45.565392 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0570ba86b862228c53e2fc8a3b2c33801e4d8251f1f16243fb063f0004d62e8b"} err="failed to get container status \"0570ba86b862228c53e2fc8a3b2c33801e4d8251f1f16243fb063f0004d62e8b\": rpc error: code = NotFound desc = could not find container \"0570ba86b862228c53e2fc8a3b2c33801e4d8251f1f16243fb063f0004d62e8b\": container with ID starting with 0570ba86b862228c53e2fc8a3b2c33801e4d8251f1f16243fb063f0004d62e8b not found: ID does not exist" Jul 13 07:13:45.577939 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:13:45.577908 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-548bc6d586-vpkhq"] Jul 13 07:13:45.587531 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:13:45.587496 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-548bc6d586-vpkhq"] Jul 13 07:13:45.807351 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:13:45.807237 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c40d2768-2409-4c45-99dc-ef0f02ae8370" path="/var/lib/kubelet/pods/c40d2768-2409-4c45-99dc-ef0f02ae8370/volumes" Jul 13 07:14:33.947935 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:14:33.947897 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:14:44.240099 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:14:44.240001 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:14:55.438726 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:14:55.438674 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:15:00.140168 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:15:00.140130 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732115-5k22n"] Jul 13 07:15:00.140538 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:15:00.140504 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c40d2768-2409-4c45-99dc-ef0f02ae8370" containerName="authorino" Jul 13 07:15:00.140538 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:15:00.140515 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="c40d2768-2409-4c45-99dc-ef0f02ae8370" containerName="authorino" Jul 13 07:15:00.140657 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:15:00.140586 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="c40d2768-2409-4c45-99dc-ef0f02ae8370" containerName="authorino" Jul 13 07:15:00.143569 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:15:00.143552 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732115-5k22n" Jul 13 07:15:00.145552 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:15:00.145526 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 13 07:15:00.145675 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:15:00.145639 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 13 07:15:00.146190 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:15:00.146173 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8rh7l\"" Jul 13 07:15:00.149848 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:15:00.149828 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732115-5k22n"] Jul 13 07:15:00.168279 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:15:00.168247 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gplxq\" (UniqueName: \"kubernetes.io/projected/44b8ab33-cb52-40ec-80ba-677f07c3086b-kube-api-access-gplxq\") pod \"maas-api-key-cleanup-29732115-5k22n\" (UID: \"44b8ab33-cb52-40ec-80ba-677f07c3086b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732115-5k22n" Jul 13 07:15:00.269608 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:15:00.269574 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gplxq\" (UniqueName: \"kubernetes.io/projected/44b8ab33-cb52-40ec-80ba-677f07c3086b-kube-api-access-gplxq\") pod \"maas-api-key-cleanup-29732115-5k22n\" (UID: \"44b8ab33-cb52-40ec-80ba-677f07c3086b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732115-5k22n" Jul 13 07:15:00.277952 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:15:00.277923 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gplxq\" (UniqueName: \"kubernetes.io/projected/44b8ab33-cb52-40ec-80ba-677f07c3086b-kube-api-access-gplxq\") pod \"maas-api-key-cleanup-29732115-5k22n\" (UID: \"44b8ab33-cb52-40ec-80ba-677f07c3086b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732115-5k22n" Jul 13 07:15:00.454980 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:15:00.454947 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732115-5k22n" Jul 13 07:15:00.593300 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:15:00.593272 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732115-5k22n"] Jul 13 07:15:00.596331 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:15:00.596300 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44b8ab33_cb52_40ec_80ba_677f07c3086b.slice/crio-7f137970624609027c4b66d1e763c50051f1daef94e2323bf32ca0dd47c41352 WatchSource:0}: Error finding container 7f137970624609027c4b66d1e763c50051f1daef94e2323bf32ca0dd47c41352: Status 404 returned error can't find the container with id 7f137970624609027c4b66d1e763c50051f1daef94e2323bf32ca0dd47c41352 Jul 13 07:15:00.974669 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:15:00.974604 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732115-5k22n" event={"ID":"44b8ab33-cb52-40ec-80ba-677f07c3086b","Type":"ContainerStarted","Data":"7f137970624609027c4b66d1e763c50051f1daef94e2323bf32ca0dd47c41352"} Jul 13 07:15:02.990997 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:15:02.990964 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732115-5k22n" event={"ID":"44b8ab33-cb52-40ec-80ba-677f07c3086b","Type":"ContainerStarted","Data":"42d4ca145bc6cb8bd3b511feb4c66738e4b76068e39e44f9e16f6cbefcdf21ad"} Jul 13 07:15:03.007428 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:15:03.007373 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732115-5k22n" podStartSLOduration=1.017807692 podStartE2EDuration="3.007358667s" totalImagesPullingTime="1.989550975s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:15:03.004540095 +0000 UTC m=+777.840103938" watchObservedRunningTime="2026-07-13 07:15:03.007358667 +0000 UTC m=+777.842922498" Jul 13 07:15:03.240648 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:15:03.240614 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:15:14.838147 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:15:14.838107 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:15:41.939719 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:15:41.939604 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:15:54.642724 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:15:54.642671 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:16:00.011162 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:16:00.011120 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732115-5k22n"] Jul 13 07:16:00.011646 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:16:00.011332 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732115-5k22n" podUID="44b8ab33-cb52-40ec-80ba-677f07c3086b" containerName="cleanup" containerID="cri-o://42d4ca145bc6cb8bd3b511feb4c66738e4b76068e39e44f9e16f6cbefcdf21ad" gracePeriod=30 Jul 13 07:16:30.215294 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:16:30.215262 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732115-5k22n" Jul 13 07:16:30.382494 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:16:30.382393 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gplxq\" (UniqueName: \"kubernetes.io/projected/44b8ab33-cb52-40ec-80ba-677f07c3086b-kube-api-access-gplxq\") pod \"44b8ab33-cb52-40ec-80ba-677f07c3086b\" (UID: \"44b8ab33-cb52-40ec-80ba-677f07c3086b\") " Jul 13 07:16:30.384725 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:16:30.384671 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44b8ab33-cb52-40ec-80ba-677f07c3086b-kube-api-access-gplxq" (OuterVolumeSpecName: "kube-api-access-gplxq") pod "44b8ab33-cb52-40ec-80ba-677f07c3086b" (UID: "44b8ab33-cb52-40ec-80ba-677f07c3086b"). InnerVolumeSpecName "kube-api-access-gplxq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:16:30.480406 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:16:30.480366 2586 generic.go:356] "Generic (PLEG): container finished" podID="44b8ab33-cb52-40ec-80ba-677f07c3086b" containerID="42d4ca145bc6cb8bd3b511feb4c66738e4b76068e39e44f9e16f6cbefcdf21ad" exitCode=137 Jul 13 07:16:30.480570 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:16:30.480397 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732115-5k22n" event={"ID":"44b8ab33-cb52-40ec-80ba-677f07c3086b","Type":"ContainerDied","Data":"42d4ca145bc6cb8bd3b511feb4c66738e4b76068e39e44f9e16f6cbefcdf21ad"} Jul 13 07:16:30.480570 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:16:30.480476 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732115-5k22n" event={"ID":"44b8ab33-cb52-40ec-80ba-677f07c3086b","Type":"ContainerDied","Data":"7f137970624609027c4b66d1e763c50051f1daef94e2323bf32ca0dd47c41352"} Jul 13 07:16:30.480570 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:16:30.480416 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732115-5k22n" Jul 13 07:16:30.480570 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:16:30.480510 2586 scope.go:117] "RemoveContainer" containerID="42d4ca145bc6cb8bd3b511feb4c66738e4b76068e39e44f9e16f6cbefcdf21ad" Jul 13 07:16:30.483406 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:16:30.483371 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gplxq\" (UniqueName: \"kubernetes.io/projected/44b8ab33-cb52-40ec-80ba-677f07c3086b-kube-api-access-gplxq\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:16:30.490625 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:16:30.490599 2586 scope.go:117] "RemoveContainer" containerID="42d4ca145bc6cb8bd3b511feb4c66738e4b76068e39e44f9e16f6cbefcdf21ad" Jul 13 07:16:30.490988 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:16:30.490964 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42d4ca145bc6cb8bd3b511feb4c66738e4b76068e39e44f9e16f6cbefcdf21ad\": container with ID starting with 42d4ca145bc6cb8bd3b511feb4c66738e4b76068e39e44f9e16f6cbefcdf21ad not found: ID does not exist" containerID="42d4ca145bc6cb8bd3b511feb4c66738e4b76068e39e44f9e16f6cbefcdf21ad" Jul 13 07:16:30.491059 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:16:30.491001 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42d4ca145bc6cb8bd3b511feb4c66738e4b76068e39e44f9e16f6cbefcdf21ad"} err="failed to get container status \"42d4ca145bc6cb8bd3b511feb4c66738e4b76068e39e44f9e16f6cbefcdf21ad\": rpc error: code = NotFound desc = could not find container \"42d4ca145bc6cb8bd3b511feb4c66738e4b76068e39e44f9e16f6cbefcdf21ad\": container with ID starting with 42d4ca145bc6cb8bd3b511feb4c66738e4b76068e39e44f9e16f6cbefcdf21ad not found: ID does not exist" Jul 13 07:16:30.502769 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:16:30.502736 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732115-5k22n"] Jul 13 07:16:30.506734 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:16:30.506708 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732115-5k22n"] Jul 13 07:16:31.259940 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:16:31.259903 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:16:31.805855 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:16:31.805818 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44b8ab33-cb52-40ec-80ba-677f07c3086b" path="/var/lib/kubelet/pods/44b8ab33-cb52-40ec-80ba-677f07c3086b/volumes" Jul 13 07:16:40.547673 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:16:40.547633 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:16:50.049884 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:16:50.049843 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:16:58.147022 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:16:58.146944 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:17:05.702542 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:17:05.702510 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-r878q_293edc99-2691-4915-ba03-7407d88b89b4/console-operator/1.log" Jul 13 07:17:05.707262 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:17:05.707239 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-r878q_293edc99-2691-4915-ba03-7407d88b89b4/console-operator/1.log" Jul 13 07:17:05.707408 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:17:05.707318 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7b964_3d71897a-4b2e-4e4e-9525-ccd89bd18458/ovn-acl-logging/0.log" Jul 13 07:17:05.711964 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:17:05.711948 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7b964_3d71897a-4b2e-4e4e-9525-ccd89bd18458/ovn-acl-logging/0.log" Jul 13 07:17:08.247524 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:17:08.247490 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:17:15.248254 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:17:15.248214 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:17:30.452970 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:17:30.452932 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:17:40.243526 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:17:40.243477 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:18:02.537844 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:18:02.537800 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:18:14.644746 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:18:14.644711 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:18:38.160942 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:18:38.160852 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:18:49.105272 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:18:49.105230 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:19:05.070060 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:19:05.070021 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:19:17.450965 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:19:17.450926 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:19:33.148762 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:19:33.148723 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:19:42.861713 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:19:42.861144 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:20:11.047907 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:20:11.047870 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:20:12.228869 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:20:12.228831 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5cbpq"] Jul 13 07:20:12.229292 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:20:12.229071 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5cbpq" podUID="ee5d74d8-923b-49c5-96b3-f3e9492530f0" containerName="manager" containerID="cri-o://78a443177f1b523a13ba12bb190104cedc2074f6acb0a39cbdaa3a1577648bd6" gracePeriod=10 Jul 13 07:20:13.737182 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:20:13.737091 2586 generic.go:356] "Generic (PLEG): container finished" podID="ee5d74d8-923b-49c5-96b3-f3e9492530f0" containerID="78a443177f1b523a13ba12bb190104cedc2074f6acb0a39cbdaa3a1577648bd6" exitCode=0 Jul 13 07:20:13.737520 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:20:13.737163 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5cbpq" event={"ID":"ee5d74d8-923b-49c5-96b3-f3e9492530f0","Type":"ContainerDied","Data":"78a443177f1b523a13ba12bb190104cedc2074f6acb0a39cbdaa3a1577648bd6"} Jul 13 07:20:14.082745 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:20:14.082716 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5cbpq" Jul 13 07:20:14.165898 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:20:14.165868 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ee5d74d8-923b-49c5-96b3-f3e9492530f0-extensions-socket-volume\") pod \"ee5d74d8-923b-49c5-96b3-f3e9492530f0\" (UID: \"ee5d74d8-923b-49c5-96b3-f3e9492530f0\") " Jul 13 07:20:14.166068 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:20:14.165927 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ht8np\" (UniqueName: \"kubernetes.io/projected/ee5d74d8-923b-49c5-96b3-f3e9492530f0-kube-api-access-ht8np\") pod \"ee5d74d8-923b-49c5-96b3-f3e9492530f0\" (UID: \"ee5d74d8-923b-49c5-96b3-f3e9492530f0\") " Jul 13 07:20:14.166274 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:20:14.166248 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee5d74d8-923b-49c5-96b3-f3e9492530f0-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ee5d74d8-923b-49c5-96b3-f3e9492530f0" (UID: "ee5d74d8-923b-49c5-96b3-f3e9492530f0"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:20:14.167858 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:20:14.167836 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee5d74d8-923b-49c5-96b3-f3e9492530f0-kube-api-access-ht8np" (OuterVolumeSpecName: "kube-api-access-ht8np") pod "ee5d74d8-923b-49c5-96b3-f3e9492530f0" (UID: "ee5d74d8-923b-49c5-96b3-f3e9492530f0"). InnerVolumeSpecName "kube-api-access-ht8np". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:20:14.267328 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:20:14.267260 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ht8np\" (UniqueName: \"kubernetes.io/projected/ee5d74d8-923b-49c5-96b3-f3e9492530f0-kube-api-access-ht8np\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:20:14.267328 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:20:14.267286 2586 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ee5d74d8-923b-49c5-96b3-f3e9492530f0-extensions-socket-volume\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:20:14.743760 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:20:14.743727 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5cbpq" event={"ID":"ee5d74d8-923b-49c5-96b3-f3e9492530f0","Type":"ContainerDied","Data":"4c9f71162f9cca38b4ee94b5f4b972e20d6f6f05ec963f0a4b974853ed1068d1"} Jul 13 07:20:14.744241 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:20:14.743789 2586 scope.go:117] "RemoveContainer" containerID="78a443177f1b523a13ba12bb190104cedc2074f6acb0a39cbdaa3a1577648bd6" Jul 13 07:20:14.744241 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:20:14.743810 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5cbpq" Jul 13 07:20:14.765802 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:20:14.765776 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5cbpq"] Jul 13 07:20:14.769178 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:20:14.769157 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5cbpq"] Jul 13 07:20:15.804782 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:20:15.804750 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee5d74d8-923b-49c5-96b3-f3e9492530f0" path="/var/lib/kubelet/pods/ee5d74d8-923b-49c5-96b3-f3e9492530f0/volumes" Jul 13 07:21:13.361894 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:21:13.361859 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nwwwh"] Jul 13 07:21:13.362297 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:21:13.362210 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44b8ab33-cb52-40ec-80ba-677f07c3086b" containerName="cleanup" Jul 13 07:21:13.362297 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:21:13.362221 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="44b8ab33-cb52-40ec-80ba-677f07c3086b" containerName="cleanup" Jul 13 07:21:13.362297 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:21:13.362241 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee5d74d8-923b-49c5-96b3-f3e9492530f0" containerName="manager" Jul 13 07:21:13.362297 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:21:13.362249 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee5d74d8-923b-49c5-96b3-f3e9492530f0" containerName="manager" Jul 13 07:21:13.362436 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:21:13.362334 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="44b8ab33-cb52-40ec-80ba-677f07c3086b" containerName="cleanup" Jul 13 07:21:13.362436 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:21:13.362346 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee5d74d8-923b-49c5-96b3-f3e9492530f0" containerName="manager" Jul 13 07:21:13.365917 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:21:13.365901 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nwwwh" Jul 13 07:21:13.368013 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:21:13.367996 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-rhv55\"" Jul 13 07:21:13.369421 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:21:13.369406 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b47e79da-5d27-4566-babe-1e2ce244a33b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-nwwwh\" (UID: \"b47e79da-5d27-4566-babe-1e2ce244a33b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nwwwh" Jul 13 07:21:13.369492 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:21:13.369465 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdwxz\" (UniqueName: \"kubernetes.io/projected/b47e79da-5d27-4566-babe-1e2ce244a33b-kube-api-access-hdwxz\") pod \"kuadrant-operator-controller-manager-86878c769f-nwwwh\" (UID: \"b47e79da-5d27-4566-babe-1e2ce244a33b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nwwwh" Jul 13 07:21:13.376563 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:21:13.376538 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nwwwh"] Jul 13 07:21:13.470231 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:21:13.470196 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdwxz\" (UniqueName: \"kubernetes.io/projected/b47e79da-5d27-4566-babe-1e2ce244a33b-kube-api-access-hdwxz\") pod \"kuadrant-operator-controller-manager-86878c769f-nwwwh\" (UID: \"b47e79da-5d27-4566-babe-1e2ce244a33b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nwwwh" Jul 13 07:21:13.470407 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:21:13.470245 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b47e79da-5d27-4566-babe-1e2ce244a33b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-nwwwh\" (UID: \"b47e79da-5d27-4566-babe-1e2ce244a33b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nwwwh" Jul 13 07:21:13.470571 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:21:13.470555 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b47e79da-5d27-4566-babe-1e2ce244a33b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-nwwwh\" (UID: \"b47e79da-5d27-4566-babe-1e2ce244a33b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nwwwh" Jul 13 07:21:13.481083 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:21:13.481062 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdwxz\" (UniqueName: \"kubernetes.io/projected/b47e79da-5d27-4566-babe-1e2ce244a33b-kube-api-access-hdwxz\") pod \"kuadrant-operator-controller-manager-86878c769f-nwwwh\" (UID: \"b47e79da-5d27-4566-babe-1e2ce244a33b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nwwwh" Jul 13 07:21:13.676852 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:21:13.676765 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nwwwh" Jul 13 07:21:13.817887 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:21:13.817856 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nwwwh"] Jul 13 07:21:13.819102 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:21:13.819078 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb47e79da_5d27_4566_babe_1e2ce244a33b.slice/crio-b2517d200354cb9d563ae6828843403fa83c2753161348d850f629ecbcbda2a3 WatchSource:0}: Error finding container b2517d200354cb9d563ae6828843403fa83c2753161348d850f629ecbcbda2a3: Status 404 returned error can't find the container with id b2517d200354cb9d563ae6828843403fa83c2753161348d850f629ecbcbda2a3 Jul 13 07:21:14.055431 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:21:14.055398 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nwwwh" event={"ID":"b47e79da-5d27-4566-babe-1e2ce244a33b","Type":"ContainerStarted","Data":"c781e111568820a7697ba29e88690a1ee3d8b123d483652fff7dbc75fac8ccdc"} Jul 13 07:21:14.055605 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:21:14.055450 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nwwwh" event={"ID":"b47e79da-5d27-4566-babe-1e2ce244a33b","Type":"ContainerStarted","Data":"b2517d200354cb9d563ae6828843403fa83c2753161348d850f629ecbcbda2a3"} Jul 13 07:21:14.055605 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:21:14.055505 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nwwwh" Jul 13 07:21:14.076546 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:21:14.076494 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nwwwh" podStartSLOduration=1.076474234 podStartE2EDuration="1.076474234s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:21:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:21:14.07525869 +0000 UTC m=+1148.910822516" watchObservedRunningTime="2026-07-13 07:21:14.076474234 +0000 UTC m=+1148.912038069" Jul 13 07:21:25.064077 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:21:25.064046 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nwwwh" Jul 13 07:21:41.731984 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:21:41.731898 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:21:56.156656 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:21:56.156614 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:22:05.740427 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:22:05.740396 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-r878q_293edc99-2691-4915-ba03-7407d88b89b4/console-operator/1.log" Jul 13 07:22:05.744570 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:22:05.744546 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7b964_3d71897a-4b2e-4e4e-9525-ccd89bd18458/ovn-acl-logging/0.log" Jul 13 07:22:05.745494 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:22:05.745474 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-r878q_293edc99-2691-4915-ba03-7407d88b89b4/console-operator/1.log" Jul 13 07:22:05.749701 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:22:05.749665 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7b964_3d71897a-4b2e-4e4e-9525-ccd89bd18458/ovn-acl-logging/0.log" Jul 13 07:22:07.145689 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:22:07.145657 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:22:18.937829 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:22:18.937788 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:22:29.342399 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:22:29.342360 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:22:38.242899 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:22:38.242864 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:22:49.739815 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:22:49.739777 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:22:59.740358 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:22:59.740281 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:23:07.339705 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:23:07.339645 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:23:16.743579 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:23:16.743544 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:23:23.042821 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:23:23.042782 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:23:45.643690 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:23:45.643637 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:23:54.445531 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:23:54.445493 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:24:04.550977 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:24:04.550936 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:24:17.202109 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:24:17.202075 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:24:33.151533 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:24:33.151437 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:24:42.640441 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:24:42.640403 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:24:52.140233 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:24:52.140200 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:24:59.949447 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:24:59.949412 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:25:10.148539 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:25:10.148498 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:25:20.142171 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:25:20.142133 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:25:34.142661 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:25:34.142626 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:25:48.244555 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:25:48.244521 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:25:58.242993 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:25:58.242910 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:26:04.041834 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:26:04.041798 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:26:16.942402 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:26:16.942370 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:26:24.739307 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:26:24.739269 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:26:35.351856 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:26:35.351820 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:26:47.240831 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:26:47.240791 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:26:56.535354 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:26:56.535318 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:27:05.774741 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:05.774713 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-r878q_293edc99-2691-4915-ba03-7407d88b89b4/console-operator/1.log" Jul 13 07:27:05.778709 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:05.778673 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7b964_3d71897a-4b2e-4e4e-9525-ccd89bd18458/ovn-acl-logging/0.log" Jul 13 07:27:05.782716 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:05.782699 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-r878q_293edc99-2691-4915-ba03-7407d88b89b4/console-operator/1.log" Jul 13 07:27:05.787038 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:05.787020 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7b964_3d71897a-4b2e-4e4e-9525-ccd89bd18458/ovn-acl-logging/0.log" Jul 13 07:27:09.049337 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:09.049298 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d86b647c-6ddf5df794-9pm2l"] Jul 13 07:27:09.053277 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:09.053256 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d86b647c-6ddf5df794-9pm2l" Jul 13 07:27:09.055548 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:09.055521 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-d86b647c-serving-cert\"" Jul 13 07:27:09.055713 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:09.055669 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 13 07:27:09.056142 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:09.056126 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 13 07:27:09.056226 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:09.056158 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8rh7l\"" Jul 13 07:27:09.063443 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:09.063421 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d86b647c-6ddf5df794-9pm2l"] Jul 13 07:27:09.124020 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:09.123979 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rr676\" (UniqueName: \"kubernetes.io/projected/21e61daf-2b2c-4361-ad86-6d09b7001681-kube-api-access-rr676\") pod \"maas-api-e2e-ait-d86b647c-6ddf5df794-9pm2l\" (UID: \"21e61daf-2b2c-4361-ad86-6d09b7001681\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d86b647c-6ddf5df794-9pm2l" Jul 13 07:27:09.124188 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:09.124044 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/21e61daf-2b2c-4361-ad86-6d09b7001681-openshift-service-ca\") pod \"maas-api-e2e-ait-d86b647c-6ddf5df794-9pm2l\" (UID: \"21e61daf-2b2c-4361-ad86-6d09b7001681\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d86b647c-6ddf5df794-9pm2l" Jul 13 07:27:09.124188 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:09.124172 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21e61daf-2b2c-4361-ad86-6d09b7001681-maas-api-tls\") pod \"maas-api-e2e-ait-d86b647c-6ddf5df794-9pm2l\" (UID: \"21e61daf-2b2c-4361-ad86-6d09b7001681\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d86b647c-6ddf5df794-9pm2l" Jul 13 07:27:09.225498 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:09.225460 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21e61daf-2b2c-4361-ad86-6d09b7001681-maas-api-tls\") pod \"maas-api-e2e-ait-d86b647c-6ddf5df794-9pm2l\" (UID: \"21e61daf-2b2c-4361-ad86-6d09b7001681\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d86b647c-6ddf5df794-9pm2l" Jul 13 07:27:09.225673 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:09.225508 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rr676\" (UniqueName: \"kubernetes.io/projected/21e61daf-2b2c-4361-ad86-6d09b7001681-kube-api-access-rr676\") pod \"maas-api-e2e-ait-d86b647c-6ddf5df794-9pm2l\" (UID: \"21e61daf-2b2c-4361-ad86-6d09b7001681\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d86b647c-6ddf5df794-9pm2l" Jul 13 07:27:09.225673 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:09.225551 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/21e61daf-2b2c-4361-ad86-6d09b7001681-openshift-service-ca\") pod \"maas-api-e2e-ait-d86b647c-6ddf5df794-9pm2l\" (UID: \"21e61daf-2b2c-4361-ad86-6d09b7001681\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d86b647c-6ddf5df794-9pm2l" Jul 13 07:27:09.225673 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:27:09.225599 2586 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-d86b647c-serving-cert: secret "maas-api-e2e-ait-d86b647c-serving-cert" not found Jul 13 07:27:09.225673 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:27:09.225675 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21e61daf-2b2c-4361-ad86-6d09b7001681-maas-api-tls podName:21e61daf-2b2c-4361-ad86-6d09b7001681 nodeName:}" failed. No retries permitted until 2026-07-13 07:27:09.725657362 +0000 UTC m=+1504.561221171 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/21e61daf-2b2c-4361-ad86-6d09b7001681-maas-api-tls") pod "maas-api-e2e-ait-d86b647c-6ddf5df794-9pm2l" (UID: "21e61daf-2b2c-4361-ad86-6d09b7001681") : secret "maas-api-e2e-ait-d86b647c-serving-cert" not found Jul 13 07:27:09.226260 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:09.226241 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/21e61daf-2b2c-4361-ad86-6d09b7001681-openshift-service-ca\") pod \"maas-api-e2e-ait-d86b647c-6ddf5df794-9pm2l\" (UID: \"21e61daf-2b2c-4361-ad86-6d09b7001681\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d86b647c-6ddf5df794-9pm2l" Jul 13 07:27:09.234590 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:09.234570 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rr676\" (UniqueName: \"kubernetes.io/projected/21e61daf-2b2c-4361-ad86-6d09b7001681-kube-api-access-rr676\") pod \"maas-api-e2e-ait-d86b647c-6ddf5df794-9pm2l\" (UID: \"21e61daf-2b2c-4361-ad86-6d09b7001681\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d86b647c-6ddf5df794-9pm2l" Jul 13 07:27:09.730540 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:09.730499 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21e61daf-2b2c-4361-ad86-6d09b7001681-maas-api-tls\") pod \"maas-api-e2e-ait-d86b647c-6ddf5df794-9pm2l\" (UID: \"21e61daf-2b2c-4361-ad86-6d09b7001681\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d86b647c-6ddf5df794-9pm2l" Jul 13 07:27:09.732877 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:09.732858 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21e61daf-2b2c-4361-ad86-6d09b7001681-maas-api-tls\") pod \"maas-api-e2e-ait-d86b647c-6ddf5df794-9pm2l\" (UID: \"21e61daf-2b2c-4361-ad86-6d09b7001681\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d86b647c-6ddf5df794-9pm2l" Jul 13 07:27:09.965972 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:09.965924 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d86b647c-6ddf5df794-9pm2l" Jul 13 07:27:10.112707 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:10.112662 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d86b647c-6ddf5df794-9pm2l"] Jul 13 07:27:10.113502 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:27:10.113478 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21e61daf_2b2c_4361_ad86_6d09b7001681.slice/crio-5c2fcd7fd0ffcbb719d92653e8d4c6c26c1fbfbd9a5ba8fc3a07ad2e50b04315 WatchSource:0}: Error finding container 5c2fcd7fd0ffcbb719d92653e8d4c6c26c1fbfbd9a5ba8fc3a07ad2e50b04315: Status 404 returned error can't find the container with id 5c2fcd7fd0ffcbb719d92653e8d4c6c26c1fbfbd9a5ba8fc3a07ad2e50b04315 Jul 13 07:27:10.114863 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:10.114844 2586 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 07:27:11.084903 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:11.084803 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d86b647c-6ddf5df794-9pm2l" event={"ID":"21e61daf-2b2c-4361-ad86-6d09b7001681","Type":"ContainerStarted","Data":"5c2fcd7fd0ffcbb719d92653e8d4c6c26c1fbfbd9a5ba8fc3a07ad2e50b04315"} Jul 13 07:27:11.944595 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:11.944526 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:27:13.098696 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:13.098652 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d86b647c-6ddf5df794-9pm2l" event={"ID":"21e61daf-2b2c-4361-ad86-6d09b7001681","Type":"ContainerStarted","Data":"99d9aaa0d759e89613443b294def80df66adfba4a6afc93aadd57715426f2dbe"} Jul 13 07:27:13.099087 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:13.098775 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d86b647c-6ddf5df794-9pm2l" Jul 13 07:27:13.113513 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:13.113371 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d86b647c-6ddf5df794-9pm2l" podStartSLOduration=2.175286638 podStartE2EDuration="4.113355772s" totalImagesPullingTime="1.938069134s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:27:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:27:13.112942954 +0000 UTC m=+1507.948506779" watchObservedRunningTime="2026-07-13 07:27:13.113355772 +0000 UTC m=+1507.948919602" Jul 13 07:27:14.450734 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:14.450702 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d86b647c-6ddf5df794-9pm2l"] Jul 13 07:27:15.108516 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:15.108466 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d86b647c-6ddf5df794-9pm2l" podUID="21e61daf-2b2c-4361-ad86-6d09b7001681" containerName="maas-api" containerID="cri-o://99d9aaa0d759e89613443b294def80df66adfba4a6afc93aadd57715426f2dbe" gracePeriod=30 Jul 13 07:27:15.358205 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:15.358179 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d86b647c-6ddf5df794-9pm2l" Jul 13 07:27:15.483977 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:15.483938 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21e61daf-2b2c-4361-ad86-6d09b7001681-maas-api-tls\") pod \"21e61daf-2b2c-4361-ad86-6d09b7001681\" (UID: \"21e61daf-2b2c-4361-ad86-6d09b7001681\") " Jul 13 07:27:15.484356 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:15.484013 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rr676\" (UniqueName: \"kubernetes.io/projected/21e61daf-2b2c-4361-ad86-6d09b7001681-kube-api-access-rr676\") pod \"21e61daf-2b2c-4361-ad86-6d09b7001681\" (UID: \"21e61daf-2b2c-4361-ad86-6d09b7001681\") " Jul 13 07:27:15.484356 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:15.484058 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/21e61daf-2b2c-4361-ad86-6d09b7001681-openshift-service-ca\") pod \"21e61daf-2b2c-4361-ad86-6d09b7001681\" (UID: \"21e61daf-2b2c-4361-ad86-6d09b7001681\") " Jul 13 07:27:15.484497 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:15.484471 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21e61daf-2b2c-4361-ad86-6d09b7001681-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "21e61daf-2b2c-4361-ad86-6d09b7001681" (UID: "21e61daf-2b2c-4361-ad86-6d09b7001681"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 07:27:15.486158 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:15.486133 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21e61daf-2b2c-4361-ad86-6d09b7001681-kube-api-access-rr676" (OuterVolumeSpecName: "kube-api-access-rr676") pod "21e61daf-2b2c-4361-ad86-6d09b7001681" (UID: "21e61daf-2b2c-4361-ad86-6d09b7001681"). InnerVolumeSpecName "kube-api-access-rr676". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:27:15.486158 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:15.486144 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21e61daf-2b2c-4361-ad86-6d09b7001681-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "21e61daf-2b2c-4361-ad86-6d09b7001681" (UID: "21e61daf-2b2c-4361-ad86-6d09b7001681"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 07:27:15.585568 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:15.585519 2586 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21e61daf-2b2c-4361-ad86-6d09b7001681-maas-api-tls\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:27:15.585568 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:15.585559 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rr676\" (UniqueName: \"kubernetes.io/projected/21e61daf-2b2c-4361-ad86-6d09b7001681-kube-api-access-rr676\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:27:15.585568 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:15.585569 2586 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/21e61daf-2b2c-4361-ad86-6d09b7001681-openshift-service-ca\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:27:16.115180 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:16.115084 2586 generic.go:356] "Generic (PLEG): container finished" podID="21e61daf-2b2c-4361-ad86-6d09b7001681" containerID="99d9aaa0d759e89613443b294def80df66adfba4a6afc93aadd57715426f2dbe" exitCode=0 Jul 13 07:27:16.115180 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:16.115130 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d86b647c-6ddf5df794-9pm2l" Jul 13 07:27:16.115180 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:16.115171 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d86b647c-6ddf5df794-9pm2l" event={"ID":"21e61daf-2b2c-4361-ad86-6d09b7001681","Type":"ContainerDied","Data":"99d9aaa0d759e89613443b294def80df66adfba4a6afc93aadd57715426f2dbe"} Jul 13 07:27:16.115424 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:16.115219 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d86b647c-6ddf5df794-9pm2l" event={"ID":"21e61daf-2b2c-4361-ad86-6d09b7001681","Type":"ContainerDied","Data":"5c2fcd7fd0ffcbb719d92653e8d4c6c26c1fbfbd9a5ba8fc3a07ad2e50b04315"} Jul 13 07:27:16.115424 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:16.115242 2586 scope.go:117] "RemoveContainer" containerID="99d9aaa0d759e89613443b294def80df66adfba4a6afc93aadd57715426f2dbe" Jul 13 07:27:16.124726 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:16.124709 2586 scope.go:117] "RemoveContainer" containerID="99d9aaa0d759e89613443b294def80df66adfba4a6afc93aadd57715426f2dbe" Jul 13 07:27:16.124997 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:27:16.124978 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99d9aaa0d759e89613443b294def80df66adfba4a6afc93aadd57715426f2dbe\": container with ID starting with 99d9aaa0d759e89613443b294def80df66adfba4a6afc93aadd57715426f2dbe not found: ID does not exist" containerID="99d9aaa0d759e89613443b294def80df66adfba4a6afc93aadd57715426f2dbe" Jul 13 07:27:16.125067 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:16.125008 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99d9aaa0d759e89613443b294def80df66adfba4a6afc93aadd57715426f2dbe"} err="failed to get container status \"99d9aaa0d759e89613443b294def80df66adfba4a6afc93aadd57715426f2dbe\": rpc error: code = NotFound desc = could not find container \"99d9aaa0d759e89613443b294def80df66adfba4a6afc93aadd57715426f2dbe\": container with ID starting with 99d9aaa0d759e89613443b294def80df66adfba4a6afc93aadd57715426f2dbe not found: ID does not exist" Jul 13 07:27:16.130497 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:16.130472 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d86b647c-6ddf5df794-9pm2l"] Jul 13 07:27:16.135655 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:16.135631 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d86b647c-6ddf5df794-9pm2l"] Jul 13 07:27:17.811019 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:17.810980 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21e61daf-2b2c-4361-ad86-6d09b7001681" path="/var/lib/kubelet/pods/21e61daf-2b2c-4361-ad86-6d09b7001681/volumes" Jul 13 07:27:21.409425 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:21.409388 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d5942115-54b74466cc-jj92n"] Jul 13 07:27:21.409884 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:21.409786 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21e61daf-2b2c-4361-ad86-6d09b7001681" containerName="maas-api" Jul 13 07:27:21.409884 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:21.409798 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="21e61daf-2b2c-4361-ad86-6d09b7001681" containerName="maas-api" Jul 13 07:27:21.409884 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:21.409871 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="21e61daf-2b2c-4361-ad86-6d09b7001681" containerName="maas-api" Jul 13 07:27:21.413061 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:21.413036 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d5942115-54b74466cc-jj92n" Jul 13 07:27:21.415212 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:21.415184 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-d5942115-serving-cert\"" Jul 13 07:27:21.415347 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:21.415210 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 13 07:27:21.415624 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:21.415608 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8rh7l\"" Jul 13 07:27:21.415703 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:21.415606 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 13 07:27:21.419723 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:21.419676 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d5942115-54b74466cc-jj92n"] Jul 13 07:27:21.436829 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:21.436804 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf22fa9d-d027-45cf-9f3d-5f0a5451dc44-maas-api-tls\") pod \"maas-api-e2e-ait-d5942115-54b74466cc-jj92n\" (UID: \"cf22fa9d-d027-45cf-9f3d-5f0a5451dc44\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d5942115-54b74466cc-jj92n" Jul 13 07:27:21.437020 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:21.436834 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf22fa9d-d027-45cf-9f3d-5f0a5451dc44-openshift-service-ca\") pod \"maas-api-e2e-ait-d5942115-54b74466cc-jj92n\" (UID: \"cf22fa9d-d027-45cf-9f3d-5f0a5451dc44\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d5942115-54b74466cc-jj92n" Jul 13 07:27:21.437020 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:21.436861 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njm9d\" (UniqueName: \"kubernetes.io/projected/cf22fa9d-d027-45cf-9f3d-5f0a5451dc44-kube-api-access-njm9d\") pod \"maas-api-e2e-ait-d5942115-54b74466cc-jj92n\" (UID: \"cf22fa9d-d027-45cf-9f3d-5f0a5451dc44\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d5942115-54b74466cc-jj92n" Jul 13 07:27:21.537819 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:21.537786 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf22fa9d-d027-45cf-9f3d-5f0a5451dc44-maas-api-tls\") pod \"maas-api-e2e-ait-d5942115-54b74466cc-jj92n\" (UID: \"cf22fa9d-d027-45cf-9f3d-5f0a5451dc44\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d5942115-54b74466cc-jj92n" Jul 13 07:27:21.537819 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:21.537822 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf22fa9d-d027-45cf-9f3d-5f0a5451dc44-openshift-service-ca\") pod \"maas-api-e2e-ait-d5942115-54b74466cc-jj92n\" (UID: \"cf22fa9d-d027-45cf-9f3d-5f0a5451dc44\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d5942115-54b74466cc-jj92n" Jul 13 07:27:21.538042 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:21.537857 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njm9d\" (UniqueName: \"kubernetes.io/projected/cf22fa9d-d027-45cf-9f3d-5f0a5451dc44-kube-api-access-njm9d\") pod \"maas-api-e2e-ait-d5942115-54b74466cc-jj92n\" (UID: \"cf22fa9d-d027-45cf-9f3d-5f0a5451dc44\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d5942115-54b74466cc-jj92n" Jul 13 07:27:21.538520 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:21.538497 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf22fa9d-d027-45cf-9f3d-5f0a5451dc44-openshift-service-ca\") pod \"maas-api-e2e-ait-d5942115-54b74466cc-jj92n\" (UID: \"cf22fa9d-d027-45cf-9f3d-5f0a5451dc44\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d5942115-54b74466cc-jj92n" Jul 13 07:27:21.540270 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:21.540251 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf22fa9d-d027-45cf-9f3d-5f0a5451dc44-maas-api-tls\") pod \"maas-api-e2e-ait-d5942115-54b74466cc-jj92n\" (UID: \"cf22fa9d-d027-45cf-9f3d-5f0a5451dc44\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d5942115-54b74466cc-jj92n" Jul 13 07:27:21.545878 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:21.545853 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njm9d\" (UniqueName: \"kubernetes.io/projected/cf22fa9d-d027-45cf-9f3d-5f0a5451dc44-kube-api-access-njm9d\") pod \"maas-api-e2e-ait-d5942115-54b74466cc-jj92n\" (UID: \"cf22fa9d-d027-45cf-9f3d-5f0a5451dc44\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d5942115-54b74466cc-jj92n" Jul 13 07:27:21.725206 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:21.725169 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d5942115-54b74466cc-jj92n" Jul 13 07:27:21.870462 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:21.870418 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d5942115-54b74466cc-jj92n"] Jul 13 07:27:21.870647 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:27:21.870621 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf22fa9d_d027_45cf_9f3d_5f0a5451dc44.slice/crio-e962b62d4e65d3a6fd9a90dfd1e07a48c0d0aa05ada98031fe9892b360a9e26b WatchSource:0}: Error finding container e962b62d4e65d3a6fd9a90dfd1e07a48c0d0aa05ada98031fe9892b360a9e26b: Status 404 returned error can't find the container with id e962b62d4e65d3a6fd9a90dfd1e07a48c0d0aa05ada98031fe9892b360a9e26b Jul 13 07:27:22.152154 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:22.152069 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d5942115-54b74466cc-jj92n" event={"ID":"cf22fa9d-d027-45cf-9f3d-5f0a5451dc44","Type":"ContainerStarted","Data":"e962b62d4e65d3a6fd9a90dfd1e07a48c0d0aa05ada98031fe9892b360a9e26b"} Jul 13 07:27:23.047664 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:23.047624 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:27:23.159834 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:23.159802 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d5942115-54b74466cc-jj92n" event={"ID":"cf22fa9d-d027-45cf-9f3d-5f0a5451dc44","Type":"ContainerStarted","Data":"cc7c49761c4a7a30a47ef43b9013be692093949be0c91010fdee3a5a726d08b8"} Jul 13 07:27:23.160003 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:23.159918 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d5942115-54b74466cc-jj92n" Jul 13 07:27:23.177176 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:23.177129 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d5942115-54b74466cc-jj92n" podStartSLOduration=1.8554594720000002 podStartE2EDuration="2.177113724s" totalImagesPullingTime="321.654252ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:27:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:27:23.175226743 +0000 UTC m=+1518.010790574" watchObservedRunningTime="2026-07-13 07:27:23.177113724 +0000 UTC m=+1518.012677552" Jul 13 07:27:26.775254 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:26.775211 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d5942115-54b74466cc-jj92n"] Jul 13 07:27:26.775769 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:26.775425 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d5942115-54b74466cc-jj92n" podUID="cf22fa9d-d027-45cf-9f3d-5f0a5451dc44" containerName="maas-api" containerID="cri-o://cc7c49761c4a7a30a47ef43b9013be692093949be0c91010fdee3a5a726d08b8" gracePeriod=30 Jul 13 07:27:27.029121 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:27.029064 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d5942115-54b74466cc-jj92n" Jul 13 07:27:27.088651 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:27.088619 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf22fa9d-d027-45cf-9f3d-5f0a5451dc44-maas-api-tls\") pod \"cf22fa9d-d027-45cf-9f3d-5f0a5451dc44\" (UID: \"cf22fa9d-d027-45cf-9f3d-5f0a5451dc44\") " Jul 13 07:27:27.088855 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:27.088765 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf22fa9d-d027-45cf-9f3d-5f0a5451dc44-openshift-service-ca\") pod \"cf22fa9d-d027-45cf-9f3d-5f0a5451dc44\" (UID: \"cf22fa9d-d027-45cf-9f3d-5f0a5451dc44\") " Jul 13 07:27:27.088855 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:27.088793 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-njm9d\" (UniqueName: \"kubernetes.io/projected/cf22fa9d-d027-45cf-9f3d-5f0a5451dc44-kube-api-access-njm9d\") pod \"cf22fa9d-d027-45cf-9f3d-5f0a5451dc44\" (UID: \"cf22fa9d-d027-45cf-9f3d-5f0a5451dc44\") " Jul 13 07:27:27.089150 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:27.089124 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf22fa9d-d027-45cf-9f3d-5f0a5451dc44-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cf22fa9d-d027-45cf-9f3d-5f0a5451dc44" (UID: "cf22fa9d-d027-45cf-9f3d-5f0a5451dc44"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 07:27:27.091083 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:27.091045 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf22fa9d-d027-45cf-9f3d-5f0a5451dc44-kube-api-access-njm9d" (OuterVolumeSpecName: "kube-api-access-njm9d") pod "cf22fa9d-d027-45cf-9f3d-5f0a5451dc44" (UID: "cf22fa9d-d027-45cf-9f3d-5f0a5451dc44"). InnerVolumeSpecName "kube-api-access-njm9d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:27:27.091236 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:27.091220 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf22fa9d-d027-45cf-9f3d-5f0a5451dc44-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cf22fa9d-d027-45cf-9f3d-5f0a5451dc44" (UID: "cf22fa9d-d027-45cf-9f3d-5f0a5451dc44"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 07:27:27.182776 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:27.182737 2586 generic.go:356] "Generic (PLEG): container finished" podID="cf22fa9d-d027-45cf-9f3d-5f0a5451dc44" containerID="cc7c49761c4a7a30a47ef43b9013be692093949be0c91010fdee3a5a726d08b8" exitCode=0 Jul 13 07:27:27.182949 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:27.182796 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d5942115-54b74466cc-jj92n" Jul 13 07:27:27.182949 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:27.182830 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d5942115-54b74466cc-jj92n" event={"ID":"cf22fa9d-d027-45cf-9f3d-5f0a5451dc44","Type":"ContainerDied","Data":"cc7c49761c4a7a30a47ef43b9013be692093949be0c91010fdee3a5a726d08b8"} Jul 13 07:27:27.182949 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:27.182899 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d5942115-54b74466cc-jj92n" event={"ID":"cf22fa9d-d027-45cf-9f3d-5f0a5451dc44","Type":"ContainerDied","Data":"e962b62d4e65d3a6fd9a90dfd1e07a48c0d0aa05ada98031fe9892b360a9e26b"} Jul 13 07:27:27.182949 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:27.182932 2586 scope.go:117] "RemoveContainer" containerID="cc7c49761c4a7a30a47ef43b9013be692093949be0c91010fdee3a5a726d08b8" Jul 13 07:27:27.190238 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:27.190211 2586 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf22fa9d-d027-45cf-9f3d-5f0a5451dc44-openshift-service-ca\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:27:27.190350 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:27.190244 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-njm9d\" (UniqueName: \"kubernetes.io/projected/cf22fa9d-d027-45cf-9f3d-5f0a5451dc44-kube-api-access-njm9d\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:27:27.190350 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:27.190261 2586 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf22fa9d-d027-45cf-9f3d-5f0a5451dc44-maas-api-tls\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:27:27.192997 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:27.192915 2586 scope.go:117] "RemoveContainer" containerID="cc7c49761c4a7a30a47ef43b9013be692093949be0c91010fdee3a5a726d08b8" Jul 13 07:27:27.193218 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:27:27.193194 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc7c49761c4a7a30a47ef43b9013be692093949be0c91010fdee3a5a726d08b8\": container with ID starting with cc7c49761c4a7a30a47ef43b9013be692093949be0c91010fdee3a5a726d08b8 not found: ID does not exist" containerID="cc7c49761c4a7a30a47ef43b9013be692093949be0c91010fdee3a5a726d08b8" Jul 13 07:27:27.193306 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:27.193256 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc7c49761c4a7a30a47ef43b9013be692093949be0c91010fdee3a5a726d08b8"} err="failed to get container status \"cc7c49761c4a7a30a47ef43b9013be692093949be0c91010fdee3a5a726d08b8\": rpc error: code = NotFound desc = could not find container \"cc7c49761c4a7a30a47ef43b9013be692093949be0c91010fdee3a5a726d08b8\": container with ID starting with cc7c49761c4a7a30a47ef43b9013be692093949be0c91010fdee3a5a726d08b8 not found: ID does not exist" Jul 13 07:27:27.205938 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:27.205915 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d5942115-54b74466cc-jj92n"] Jul 13 07:27:27.208186 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:27.208165 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d5942115-54b74466cc-jj92n"] Jul 13 07:27:27.806067 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:27.805993 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf22fa9d-d027-45cf-9f3d-5f0a5451dc44" path="/var/lib/kubelet/pods/cf22fa9d-d027-45cf-9f3d-5f0a5451dc44/volumes" Jul 13 07:27:42.642472 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:42.642441 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:27:51.249011 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:27:51.248969 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:29:14.950990 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:29:14.950950 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:29:23.940811 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:29:23.940769 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:29:35.241797 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:29:35.241766 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:29:46.446515 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:29:46.446475 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:30:00.134974 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:30:00.134935 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732130-bqkxp"] Jul 13 07:30:00.135376 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:30:00.135333 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf22fa9d-d027-45cf-9f3d-5f0a5451dc44" containerName="maas-api" Jul 13 07:30:00.135376 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:30:00.135346 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf22fa9d-d027-45cf-9f3d-5f0a5451dc44" containerName="maas-api" Jul 13 07:30:00.135449 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:30:00.135422 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf22fa9d-d027-45cf-9f3d-5f0a5451dc44" containerName="maas-api" Jul 13 07:30:00.138755 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:30:00.138737 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732130-bqkxp" Jul 13 07:30:00.140875 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:30:00.140848 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 13 07:30:00.141392 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:30:00.141325 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 13 07:30:00.141531 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:30:00.141329 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8rh7l\"" Jul 13 07:30:00.145244 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:30:00.145213 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732130-bqkxp"] Jul 13 07:30:00.241701 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:30:00.241634 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6b9rw\" (UniqueName: \"kubernetes.io/projected/82ffc719-9f6c-4ce3-9b21-a43fb1ad33d5-kube-api-access-6b9rw\") pod \"maas-api-key-cleanup-29732130-bqkxp\" (UID: \"82ffc719-9f6c-4ce3-9b21-a43fb1ad33d5\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732130-bqkxp" Jul 13 07:30:00.342597 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:30:00.342554 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6b9rw\" (UniqueName: \"kubernetes.io/projected/82ffc719-9f6c-4ce3-9b21-a43fb1ad33d5-kube-api-access-6b9rw\") pod \"maas-api-key-cleanup-29732130-bqkxp\" (UID: \"82ffc719-9f6c-4ce3-9b21-a43fb1ad33d5\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732130-bqkxp" Jul 13 07:30:00.350168 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:30:00.350138 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6b9rw\" (UniqueName: \"kubernetes.io/projected/82ffc719-9f6c-4ce3-9b21-a43fb1ad33d5-kube-api-access-6b9rw\") pod \"maas-api-key-cleanup-29732130-bqkxp\" (UID: \"82ffc719-9f6c-4ce3-9b21-a43fb1ad33d5\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732130-bqkxp" Jul 13 07:30:00.450488 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:30:00.450455 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732130-bqkxp" Jul 13 07:30:00.589317 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:30:00.589287 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732130-bqkxp"] Jul 13 07:30:00.590890 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:30:00.590857 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82ffc719_9f6c_4ce3_9b21_a43fb1ad33d5.slice/crio-df5f0786340a4880e323cc9efc3c6fbbebb1c38a269eac327c47f26dd78965d3 WatchSource:0}: Error finding container df5f0786340a4880e323cc9efc3c6fbbebb1c38a269eac327c47f26dd78965d3: Status 404 returned error can't find the container with id df5f0786340a4880e323cc9efc3c6fbbebb1c38a269eac327c47f26dd78965d3 Jul 13 07:30:01.018425 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:30:01.018391 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732130-bqkxp" event={"ID":"82ffc719-9f6c-4ce3-9b21-a43fb1ad33d5","Type":"ContainerStarted","Data":"891c300ee6c7dd58037a6fcf87d459520e17154e7ac044d402ed7a326cad04f9"} Jul 13 07:30:01.018641 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:30:01.018452 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732130-bqkxp" event={"ID":"82ffc719-9f6c-4ce3-9b21-a43fb1ad33d5","Type":"ContainerStarted","Data":"df5f0786340a4880e323cc9efc3c6fbbebb1c38a269eac327c47f26dd78965d3"} Jul 13 07:30:01.034693 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:30:01.034634 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732130-bqkxp" podStartSLOduration=1.034616735 podStartE2EDuration="1.034616735s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:30:01.034015656 +0000 UTC m=+1675.869579487" watchObservedRunningTime="2026-07-13 07:30:01.034616735 +0000 UTC m=+1675.870180566" Jul 13 07:30:14.242588 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:30:14.242551 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:30:18.035202 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:30:18.035157 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-2735700e/e2e-lifecycle-model-2735700e-kserve-74bfd9bd59-vblfz"] Jul 13 07:30:18.038982 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:30:18.038957 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-2735700e/e2e-lifecycle-model-2735700e-kserve-74bfd9bd59-vblfz" Jul 13 07:30:18.041165 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:30:18.041144 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-2735700e\"/\"openshift-service-ca.crt\"" Jul 13 07:30:18.041786 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:30:18.041767 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-2735700e\"/\"default-dockercfg-5xg4p\"" Jul 13 07:30:18.041915 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:30:18.041794 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-2735700e\"/\"e2e-lifecycle-model-2735700e-kserve-self-signed-certs\"" Jul 13 07:30:18.042036 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:30:18.041836 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-2735700e\"/\"kube-root-ca.crt\"" Jul 13 07:30:18.048048 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:30:18.048023 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-2735700e/e2e-lifecycle-model-2735700e-kserve-74bfd9bd59-vblfz"] Jul 13 07:30:18.110368 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:30:18.110333 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42-tls-certs\") pod \"e2e-lifecycle-model-2735700e-kserve-74bfd9bd59-vblfz\" (UID: \"2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42\") " pod="ai-tenant-e2e-mt-2735700e/e2e-lifecycle-model-2735700e-kserve-74bfd9bd59-vblfz" Jul 13 07:30:18.110549 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:30:18.110374 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42-dshm\") pod \"e2e-lifecycle-model-2735700e-kserve-74bfd9bd59-vblfz\" (UID: \"2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42\") " pod="ai-tenant-e2e-mt-2735700e/e2e-lifecycle-model-2735700e-kserve-74bfd9bd59-vblfz" Jul 13 07:30:18.110549 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:30:18.110442 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42-tmp-dir\") pod \"e2e-lifecycle-model-2735700e-kserve-74bfd9bd59-vblfz\" (UID: \"2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42\") " pod="ai-tenant-e2e-mt-2735700e/e2e-lifecycle-model-2735700e-kserve-74bfd9bd59-vblfz" Jul 13 07:30:18.110549 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:30:18.110499 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49bbj\" (UniqueName: \"kubernetes.io/projected/2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42-kube-api-access-49bbj\") pod \"e2e-lifecycle-model-2735700e-kserve-74bfd9bd59-vblfz\" (UID: \"2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42\") " pod="ai-tenant-e2e-mt-2735700e/e2e-lifecycle-model-2735700e-kserve-74bfd9bd59-vblfz" Jul 13 07:30:18.110549 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:30:18.110520 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42-home\") pod \"e2e-lifecycle-model-2735700e-kserve-74bfd9bd59-vblfz\" (UID: \"2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42\") " pod="ai-tenant-e2e-mt-2735700e/e2e-lifecycle-model-2735700e-kserve-74bfd9bd59-vblfz" Jul 13 07:30:18.110726 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:30:18.110633 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42-model-cache\") pod \"e2e-lifecycle-model-2735700e-kserve-74bfd9bd59-vblfz\" (UID: \"2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42\") " pod="ai-tenant-e2e-mt-2735700e/e2e-lifecycle-model-2735700e-kserve-74bfd9bd59-vblfz" Jul 13 07:30:18.211828 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:30:18.211797 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42-tls-certs\") pod \"e2e-lifecycle-model-2735700e-kserve-74bfd9bd59-vblfz\" (UID: \"2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42\") " pod="ai-tenant-e2e-mt-2735700e/e2e-lifecycle-model-2735700e-kserve-74bfd9bd59-vblfz" Jul 13 07:30:18.212023 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:30:18.211833 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42-dshm\") pod \"e2e-lifecycle-model-2735700e-kserve-74bfd9bd59-vblfz\" (UID: \"2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42\") " pod="ai-tenant-e2e-mt-2735700e/e2e-lifecycle-model-2735700e-kserve-74bfd9bd59-vblfz" Jul 13 07:30:18.212023 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:30:18.211858 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42-tmp-dir\") pod \"e2e-lifecycle-model-2735700e-kserve-74bfd9bd59-vblfz\" (UID: \"2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42\") " pod="ai-tenant-e2e-mt-2735700e/e2e-lifecycle-model-2735700e-kserve-74bfd9bd59-vblfz" Jul 13 07:30:18.212023 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:30:18.211900 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49bbj\" (UniqueName: \"kubernetes.io/projected/2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42-kube-api-access-49bbj\") pod \"e2e-lifecycle-model-2735700e-kserve-74bfd9bd59-vblfz\" (UID: \"2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42\") " pod="ai-tenant-e2e-mt-2735700e/e2e-lifecycle-model-2735700e-kserve-74bfd9bd59-vblfz" Jul 13 07:30:18.212023 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:30:18.211932 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42-home\") pod \"e2e-lifecycle-model-2735700e-kserve-74bfd9bd59-vblfz\" (UID: \"2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42\") " pod="ai-tenant-e2e-mt-2735700e/e2e-lifecycle-model-2735700e-kserve-74bfd9bd59-vblfz" Jul 13 07:30:18.212240 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:30:18.212025 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42-model-cache\") pod \"e2e-lifecycle-model-2735700e-kserve-74bfd9bd59-vblfz\" (UID: \"2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42\") " pod="ai-tenant-e2e-mt-2735700e/e2e-lifecycle-model-2735700e-kserve-74bfd9bd59-vblfz" Jul 13 07:30:18.212449 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:30:18.212421 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42-model-cache\") pod \"e2e-lifecycle-model-2735700e-kserve-74bfd9bd59-vblfz\" (UID: \"2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42\") " pod="ai-tenant-e2e-mt-2735700e/e2e-lifecycle-model-2735700e-kserve-74bfd9bd59-vblfz" Jul 13 07:30:18.212449 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:30:18.212428 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42-tmp-dir\") pod \"e2e-lifecycle-model-2735700e-kserve-74bfd9bd59-vblfz\" (UID: \"2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42\") " pod="ai-tenant-e2e-mt-2735700e/e2e-lifecycle-model-2735700e-kserve-74bfd9bd59-vblfz" Jul 13 07:30:18.212607 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:30:18.212485 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42-home\") pod \"e2e-lifecycle-model-2735700e-kserve-74bfd9bd59-vblfz\" (UID: \"2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42\") " pod="ai-tenant-e2e-mt-2735700e/e2e-lifecycle-model-2735700e-kserve-74bfd9bd59-vblfz" Jul 13 07:30:18.214197 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:30:18.214176 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42-dshm\") pod \"e2e-lifecycle-model-2735700e-kserve-74bfd9bd59-vblfz\" (UID: \"2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42\") " pod="ai-tenant-e2e-mt-2735700e/e2e-lifecycle-model-2735700e-kserve-74bfd9bd59-vblfz" Jul 13 07:30:18.214452 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:30:18.214436 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42-tls-certs\") pod \"e2e-lifecycle-model-2735700e-kserve-74bfd9bd59-vblfz\" (UID: \"2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42\") " pod="ai-tenant-e2e-mt-2735700e/e2e-lifecycle-model-2735700e-kserve-74bfd9bd59-vblfz" Jul 13 07:30:18.218750 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:30:18.218731 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-49bbj\" (UniqueName: \"kubernetes.io/projected/2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42-kube-api-access-49bbj\") pod \"e2e-lifecycle-model-2735700e-kserve-74bfd9bd59-vblfz\" (UID: \"2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42\") " pod="ai-tenant-e2e-mt-2735700e/e2e-lifecycle-model-2735700e-kserve-74bfd9bd59-vblfz" Jul 13 07:30:18.350657 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:30:18.350564 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-2735700e/e2e-lifecycle-model-2735700e-kserve-74bfd9bd59-vblfz" Jul 13 07:30:18.490548 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:30:18.490516 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-2735700e/e2e-lifecycle-model-2735700e-kserve-74bfd9bd59-vblfz"] Jul 13 07:30:18.492134 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:30:18.492109 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2bf8c6a3_05c0_46ac_8eb4_f6ff791f0f42.slice/crio-648275896bc58ca58ff350aa579894562de052190e0180c9f640bde4546e4d30 WatchSource:0}: Error finding container 648275896bc58ca58ff350aa579894562de052190e0180c9f640bde4546e4d30: Status 404 returned error can't find the container with id 648275896bc58ca58ff350aa579894562de052190e0180c9f640bde4546e4d30 Jul 13 07:30:19.120377 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:30:19.120338 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-2735700e/e2e-lifecycle-model-2735700e-kserve-74bfd9bd59-vblfz" event={"ID":"2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42","Type":"ContainerStarted","Data":"985737c003f33ccdb56e2d7e5a7e3b5d9657879e5ae48e682440a0ec4de163b0"} Jul 13 07:30:19.120377 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:30:19.120381 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-2735700e/e2e-lifecycle-model-2735700e-kserve-74bfd9bd59-vblfz" event={"ID":"2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42","Type":"ContainerStarted","Data":"648275896bc58ca58ff350aa579894562de052190e0180c9f640bde4546e4d30"} Jul 13 07:30:19.138369 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:30:19.138318 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-2735700e/e2e-lifecycle-model-2735700e-kserve-74bfd9bd59-vblfz" podStartSLOduration=1.138302816 podStartE2EDuration="1.138302816s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:30:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:30:19.136658544 +0000 UTC m=+1693.972222376" watchObservedRunningTime="2026-07-13 07:30:19.138302816 +0000 UTC m=+1693.973866647" Jul 13 07:30:28.249658 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:30:28.249559 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:30:28.351446 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:30:28.351405 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-2735700e/e2e-lifecycle-model-2735700e-kserve-74bfd9bd59-vblfz" Jul 13 07:30:28.351446 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:30:28.351460 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-2735700e/e2e-lifecycle-model-2735700e-kserve-74bfd9bd59-vblfz" Jul 13 07:30:28.364204 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:30:28.364178 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-2735700e/e2e-lifecycle-model-2735700e-kserve-74bfd9bd59-vblfz" Jul 13 07:30:29.192126 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:30:29.192094 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-2735700e/e2e-lifecycle-model-2735700e-kserve-74bfd9bd59-vblfz" Jul 13 07:30:45.146344 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:30:45.146305 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:30:58.822431 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:30:58.822396 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-2735700e/e2e-lifecycle-model-2735700e-kserve-74bfd9bd59-vblfz"] Jul 13 07:30:59.733962 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:30:59.733927 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:31:00.010180 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:31:00.010089 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732130-bqkxp"] Jul 13 07:31:00.010559 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:31:00.010293 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732130-bqkxp" podUID="82ffc719-9f6c-4ce3-9b21-a43fb1ad33d5" containerName="cleanup" containerID="cri-o://891c300ee6c7dd58037a6fcf87d459520e17154e7ac044d402ed7a326cad04f9" gracePeriod=30 Jul 13 07:31:13.842999 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:31:13.842940 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-2735700e/e2e-lifecycle-model-2735700e-kserve-74bfd9bd59-vblfz" podUID="2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42" containerName="main" containerID="cri-o://985737c003f33ccdb56e2d7e5a7e3b5d9657879e5ae48e682440a0ec4de163b0" gracePeriod=45 Jul 13 07:31:14.094590 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:31:14.094533 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-2735700e/e2e-lifecycle-model-2735700e-kserve-74bfd9bd59-vblfz" Jul 13 07:31:14.174288 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:31:14.174258 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42-home\") pod \"2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42\" (UID: \"2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42\") " Jul 13 07:31:14.174457 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:31:14.174315 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42-tls-certs\") pod \"2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42\" (UID: \"2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42\") " Jul 13 07:31:14.174457 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:31:14.174332 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42-tmp-dir\") pod \"2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42\" (UID: \"2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42\") " Jul 13 07:31:14.174457 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:31:14.174351 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-49bbj\" (UniqueName: \"kubernetes.io/projected/2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42-kube-api-access-49bbj\") pod \"2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42\" (UID: \"2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42\") " Jul 13 07:31:14.174457 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:31:14.174444 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42-model-cache\") pod \"2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42\" (UID: \"2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42\") " Jul 13 07:31:14.174716 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:31:14.174473 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42-dshm\") pod \"2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42\" (UID: \"2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42\") " Jul 13 07:31:14.174716 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:31:14.174517 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42-home" (OuterVolumeSpecName: "home") pod "2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42" (UID: "2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:31:14.174716 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:31:14.174617 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42" (UID: "2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:31:14.174900 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:31:14.174728 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42-model-cache" (OuterVolumeSpecName: "model-cache") pod "2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42" (UID: "2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:31:14.174900 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:31:14.174786 2586 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42-home\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:31:14.174900 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:31:14.174798 2586 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42-tmp-dir\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:31:14.176558 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:31:14.176531 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42" (UID: "2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 07:31:14.176558 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:31:14.176540 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42-dshm" (OuterVolumeSpecName: "dshm") pod "2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42" (UID: "2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:31:14.176731 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:31:14.176568 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42-kube-api-access-49bbj" (OuterVolumeSpecName: "kube-api-access-49bbj") pod "2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42" (UID: "2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42"). InnerVolumeSpecName "kube-api-access-49bbj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:31:14.275625 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:31:14.275580 2586 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42-tls-certs\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:31:14.275625 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:31:14.275621 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-49bbj\" (UniqueName: \"kubernetes.io/projected/2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42-kube-api-access-49bbj\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:31:14.275625 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:31:14.275632 2586 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42-model-cache\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:31:14.275625 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:31:14.275641 2586 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42-dshm\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:31:14.435168 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:31:14.435138 2586 generic.go:356] "Generic (PLEG): container finished" podID="2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42" containerID="985737c003f33ccdb56e2d7e5a7e3b5d9657879e5ae48e682440a0ec4de163b0" exitCode=0 Jul 13 07:31:14.435355 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:31:14.435174 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-2735700e/e2e-lifecycle-model-2735700e-kserve-74bfd9bd59-vblfz" Jul 13 07:31:14.435355 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:31:14.435216 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-2735700e/e2e-lifecycle-model-2735700e-kserve-74bfd9bd59-vblfz" event={"ID":"2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42","Type":"ContainerDied","Data":"985737c003f33ccdb56e2d7e5a7e3b5d9657879e5ae48e682440a0ec4de163b0"} Jul 13 07:31:14.435355 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:31:14.435259 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-2735700e/e2e-lifecycle-model-2735700e-kserve-74bfd9bd59-vblfz" event={"ID":"2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42","Type":"ContainerDied","Data":"648275896bc58ca58ff350aa579894562de052190e0180c9f640bde4546e4d30"} Jul 13 07:31:14.435355 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:31:14.435275 2586 scope.go:117] "RemoveContainer" containerID="985737c003f33ccdb56e2d7e5a7e3b5d9657879e5ae48e682440a0ec4de163b0" Jul 13 07:31:14.445093 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:31:14.445074 2586 scope.go:117] "RemoveContainer" containerID="985737c003f33ccdb56e2d7e5a7e3b5d9657879e5ae48e682440a0ec4de163b0" Jul 13 07:31:14.445342 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:31:14.445326 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"985737c003f33ccdb56e2d7e5a7e3b5d9657879e5ae48e682440a0ec4de163b0\": container with ID starting with 985737c003f33ccdb56e2d7e5a7e3b5d9657879e5ae48e682440a0ec4de163b0 not found: ID does not exist" containerID="985737c003f33ccdb56e2d7e5a7e3b5d9657879e5ae48e682440a0ec4de163b0" Jul 13 07:31:14.445389 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:31:14.445350 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"985737c003f33ccdb56e2d7e5a7e3b5d9657879e5ae48e682440a0ec4de163b0"} err="failed to get container status \"985737c003f33ccdb56e2d7e5a7e3b5d9657879e5ae48e682440a0ec4de163b0\": rpc error: code = NotFound desc = could not find container \"985737c003f33ccdb56e2d7e5a7e3b5d9657879e5ae48e682440a0ec4de163b0\": container with ID starting with 985737c003f33ccdb56e2d7e5a7e3b5d9657879e5ae48e682440a0ec4de163b0 not found: ID does not exist" Jul 13 07:31:14.459603 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:31:14.459581 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-2735700e/e2e-lifecycle-model-2735700e-kserve-74bfd9bd59-vblfz"] Jul 13 07:31:14.463140 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:31:14.463120 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-2735700e/e2e-lifecycle-model-2735700e-kserve-74bfd9bd59-vblfz"] Jul 13 07:31:15.805546 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:31:15.805510 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42" path="/var/lib/kubelet/pods/2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42/volumes" Jul 13 07:31:30.213853 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:31:30.213824 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732130-bqkxp" Jul 13 07:31:30.325239 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:31:30.325152 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6b9rw\" (UniqueName: \"kubernetes.io/projected/82ffc719-9f6c-4ce3-9b21-a43fb1ad33d5-kube-api-access-6b9rw\") pod \"82ffc719-9f6c-4ce3-9b21-a43fb1ad33d5\" (UID: \"82ffc719-9f6c-4ce3-9b21-a43fb1ad33d5\") " Jul 13 07:31:30.327186 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:31:30.327159 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82ffc719-9f6c-4ce3-9b21-a43fb1ad33d5-kube-api-access-6b9rw" (OuterVolumeSpecName: "kube-api-access-6b9rw") pod "82ffc719-9f6c-4ce3-9b21-a43fb1ad33d5" (UID: "82ffc719-9f6c-4ce3-9b21-a43fb1ad33d5"). InnerVolumeSpecName "kube-api-access-6b9rw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:31:30.426555 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:31:30.426519 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6b9rw\" (UniqueName: \"kubernetes.io/projected/82ffc719-9f6c-4ce3-9b21-a43fb1ad33d5-kube-api-access-6b9rw\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:31:30.536334 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:31:30.536294 2586 generic.go:356] "Generic (PLEG): container finished" podID="82ffc719-9f6c-4ce3-9b21-a43fb1ad33d5" containerID="891c300ee6c7dd58037a6fcf87d459520e17154e7ac044d402ed7a326cad04f9" exitCode=137 Jul 13 07:31:30.536334 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:31:30.536337 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732130-bqkxp" Jul 13 07:31:30.536533 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:31:30.536378 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732130-bqkxp" event={"ID":"82ffc719-9f6c-4ce3-9b21-a43fb1ad33d5","Type":"ContainerDied","Data":"891c300ee6c7dd58037a6fcf87d459520e17154e7ac044d402ed7a326cad04f9"} Jul 13 07:31:30.536533 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:31:30.536429 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732130-bqkxp" event={"ID":"82ffc719-9f6c-4ce3-9b21-a43fb1ad33d5","Type":"ContainerDied","Data":"df5f0786340a4880e323cc9efc3c6fbbebb1c38a269eac327c47f26dd78965d3"} Jul 13 07:31:30.536533 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:31:30.536451 2586 scope.go:117] "RemoveContainer" containerID="891c300ee6c7dd58037a6fcf87d459520e17154e7ac044d402ed7a326cad04f9" Jul 13 07:31:30.545541 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:31:30.545525 2586 scope.go:117] "RemoveContainer" containerID="891c300ee6c7dd58037a6fcf87d459520e17154e7ac044d402ed7a326cad04f9" Jul 13 07:31:30.545844 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:31:30.545826 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"891c300ee6c7dd58037a6fcf87d459520e17154e7ac044d402ed7a326cad04f9\": container with ID starting with 891c300ee6c7dd58037a6fcf87d459520e17154e7ac044d402ed7a326cad04f9 not found: ID does not exist" containerID="891c300ee6c7dd58037a6fcf87d459520e17154e7ac044d402ed7a326cad04f9" Jul 13 07:31:30.545908 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:31:30.545851 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"891c300ee6c7dd58037a6fcf87d459520e17154e7ac044d402ed7a326cad04f9"} err="failed to get container status \"891c300ee6c7dd58037a6fcf87d459520e17154e7ac044d402ed7a326cad04f9\": rpc error: code = NotFound desc = could not find container \"891c300ee6c7dd58037a6fcf87d459520e17154e7ac044d402ed7a326cad04f9\": container with ID starting with 891c300ee6c7dd58037a6fcf87d459520e17154e7ac044d402ed7a326cad04f9 not found: ID does not exist" Jul 13 07:31:30.557424 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:31:30.557403 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732130-bqkxp"] Jul 13 07:31:30.560844 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:31:30.560825 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732130-bqkxp"] Jul 13 07:31:31.805846 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:31:31.805814 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82ffc719-9f6c-4ce3-9b21-a43fb1ad33d5" path="/var/lib/kubelet/pods/82ffc719-9f6c-4ce3-9b21-a43fb1ad33d5/volumes" Jul 13 07:31:38.235019 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:31:38.234982 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:32:03.048063 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:32:03.047969 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:32:05.813751 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:32:05.813723 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-r878q_293edc99-2691-4915-ba03-7407d88b89b4/console-operator/1.log" Jul 13 07:32:05.818141 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:32:05.818116 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7b964_3d71897a-4b2e-4e4e-9525-ccd89bd18458/ovn-acl-logging/0.log" Jul 13 07:32:05.828698 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:32:05.828649 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-r878q_293edc99-2691-4915-ba03-7407d88b89b4/console-operator/1.log" Jul 13 07:32:05.833422 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:32:05.833399 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7b964_3d71897a-4b2e-4e4e-9525-ccd89bd18458/ovn-acl-logging/0.log" Jul 13 07:32:16.762872 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:32:16.762835 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:32:58.143285 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:32:58.143251 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-8ccb17-857c469885-59tfr"] Jul 13 07:32:58.143735 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:32:58.143627 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82ffc719-9f6c-4ce3-9b21-a43fb1ad33d5" containerName="cleanup" Jul 13 07:32:58.143735 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:32:58.143638 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="82ffc719-9f6c-4ce3-9b21-a43fb1ad33d5" containerName="cleanup" Jul 13 07:32:58.143735 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:32:58.143657 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42" containerName="main" Jul 13 07:32:58.143735 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:32:58.143663 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42" containerName="main" Jul 13 07:32:58.143877 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:32:58.143759 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="82ffc719-9f6c-4ce3-9b21-a43fb1ad33d5" containerName="cleanup" Jul 13 07:32:58.143877 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:32:58.143774 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bf8c6a3-05c0-46ac-8eb4-f6ff791f0f42" containerName="main" Jul 13 07:32:58.146910 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:32:58.146894 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8ccb17-857c469885-59tfr" Jul 13 07:32:58.169882 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:32:58.169853 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8rh7l\"" Jul 13 07:32:58.170652 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:32:58.170634 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 13 07:32:58.170877 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:32:58.170860 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-8ccb17-serving-cert\"" Jul 13 07:32:58.171043 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:32:58.171014 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m85fz\" (UniqueName: \"kubernetes.io/projected/ea4f1126-8ac7-4f29-931e-a0ba11c61cd1-kube-api-access-m85fz\") pod \"maas-api-e2e-inf-a-8ccb17-857c469885-59tfr\" (UID: \"ea4f1126-8ac7-4f29-931e-a0ba11c61cd1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8ccb17-857c469885-59tfr" Jul 13 07:32:58.171114 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:32:58.171075 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea4f1126-8ac7-4f29-931e-a0ba11c61cd1-openshift-service-ca\") pod \"maas-api-e2e-inf-a-8ccb17-857c469885-59tfr\" (UID: \"ea4f1126-8ac7-4f29-931e-a0ba11c61cd1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8ccb17-857c469885-59tfr" Jul 13 07:32:58.171173 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:32:58.171121 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea4f1126-8ac7-4f29-931e-a0ba11c61cd1-maas-api-tls\") pod \"maas-api-e2e-inf-a-8ccb17-857c469885-59tfr\" (UID: \"ea4f1126-8ac7-4f29-931e-a0ba11c61cd1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8ccb17-857c469885-59tfr" Jul 13 07:32:58.174372 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:32:58.174352 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 13 07:32:58.224429 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:32:58.224400 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-8ccb17-857c469885-59tfr"] Jul 13 07:32:58.272579 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:32:58.272547 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea4f1126-8ac7-4f29-931e-a0ba11c61cd1-openshift-service-ca\") pod \"maas-api-e2e-inf-a-8ccb17-857c469885-59tfr\" (UID: \"ea4f1126-8ac7-4f29-931e-a0ba11c61cd1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8ccb17-857c469885-59tfr" Jul 13 07:32:58.272756 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:32:58.272590 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea4f1126-8ac7-4f29-931e-a0ba11c61cd1-maas-api-tls\") pod \"maas-api-e2e-inf-a-8ccb17-857c469885-59tfr\" (UID: \"ea4f1126-8ac7-4f29-931e-a0ba11c61cd1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8ccb17-857c469885-59tfr" Jul 13 07:32:58.272756 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:32:58.272647 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m85fz\" (UniqueName: \"kubernetes.io/projected/ea4f1126-8ac7-4f29-931e-a0ba11c61cd1-kube-api-access-m85fz\") pod \"maas-api-e2e-inf-a-8ccb17-857c469885-59tfr\" (UID: \"ea4f1126-8ac7-4f29-931e-a0ba11c61cd1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8ccb17-857c469885-59tfr" Jul 13 07:32:58.272839 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:32:58.272790 2586 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-8ccb17-serving-cert: secret "maas-api-e2e-inf-a-8ccb17-serving-cert" not found Jul 13 07:32:58.272878 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:32:58.272863 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea4f1126-8ac7-4f29-931e-a0ba11c61cd1-maas-api-tls podName:ea4f1126-8ac7-4f29-931e-a0ba11c61cd1 nodeName:}" failed. No retries permitted until 2026-07-13 07:32:58.772844566 +0000 UTC m=+1853.608408394 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ea4f1126-8ac7-4f29-931e-a0ba11c61cd1-maas-api-tls") pod "maas-api-e2e-inf-a-8ccb17-857c469885-59tfr" (UID: "ea4f1126-8ac7-4f29-931e-a0ba11c61cd1") : secret "maas-api-e2e-inf-a-8ccb17-serving-cert" not found Jul 13 07:32:58.273228 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:32:58.273210 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea4f1126-8ac7-4f29-931e-a0ba11c61cd1-openshift-service-ca\") pod \"maas-api-e2e-inf-a-8ccb17-857c469885-59tfr\" (UID: \"ea4f1126-8ac7-4f29-931e-a0ba11c61cd1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8ccb17-857c469885-59tfr" Jul 13 07:32:58.292715 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:32:58.292665 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m85fz\" (UniqueName: \"kubernetes.io/projected/ea4f1126-8ac7-4f29-931e-a0ba11c61cd1-kube-api-access-m85fz\") pod \"maas-api-e2e-inf-a-8ccb17-857c469885-59tfr\" (UID: \"ea4f1126-8ac7-4f29-931e-a0ba11c61cd1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8ccb17-857c469885-59tfr" Jul 13 07:32:58.778209 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:32:58.778174 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea4f1126-8ac7-4f29-931e-a0ba11c61cd1-maas-api-tls\") pod \"maas-api-e2e-inf-a-8ccb17-857c469885-59tfr\" (UID: \"ea4f1126-8ac7-4f29-931e-a0ba11c61cd1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8ccb17-857c469885-59tfr" Jul 13 07:32:58.780702 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:32:58.780654 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea4f1126-8ac7-4f29-931e-a0ba11c61cd1-maas-api-tls\") pod \"maas-api-e2e-inf-a-8ccb17-857c469885-59tfr\" (UID: \"ea4f1126-8ac7-4f29-931e-a0ba11c61cd1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8ccb17-857c469885-59tfr" Jul 13 07:32:59.056818 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:32:59.056716 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8ccb17-857c469885-59tfr" Jul 13 07:32:59.215694 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:32:59.215657 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-8ccb17-857c469885-59tfr"] Jul 13 07:32:59.217764 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:32:59.217737 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea4f1126_8ac7_4f29_931e_a0ba11c61cd1.slice/crio-69d167e266856a765927aa489c8847e68e2dca0fb95b9900184ed43828efd29b WatchSource:0}: Error finding container 69d167e266856a765927aa489c8847e68e2dca0fb95b9900184ed43828efd29b: Status 404 returned error can't find the container with id 69d167e266856a765927aa489c8847e68e2dca0fb95b9900184ed43828efd29b Jul 13 07:32:59.219295 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:32:59.219272 2586 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 07:33:00.034921 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:33:00.034887 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8ccb17-857c469885-59tfr" event={"ID":"ea4f1126-8ac7-4f29-931e-a0ba11c61cd1","Type":"ContainerStarted","Data":"9d1b0a36ce8c09ba5fe1c6ef536df40d06402ae7ddb9d5819df9f0bb66d0a3a7"} Jul 13 07:33:00.035144 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:33:00.034949 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8ccb17-857c469885-59tfr" event={"ID":"ea4f1126-8ac7-4f29-931e-a0ba11c61cd1","Type":"ContainerStarted","Data":"69d167e266856a765927aa489c8847e68e2dca0fb95b9900184ed43828efd29b"} Jul 13 07:33:00.035144 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:33:00.035008 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8ccb17-857c469885-59tfr" Jul 13 07:33:00.061926 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:33:00.061864 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8ccb17-857c469885-59tfr" podStartSLOduration=1.628561994 podStartE2EDuration="2.061849076s" totalImagesPullingTime="433.287082ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:32:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:33:00.061459907 +0000 UTC m=+1854.897023738" watchObservedRunningTime="2026-07-13 07:33:00.061849076 +0000 UTC m=+1854.897412907" Jul 13 07:33:06.045400 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:33:06.045368 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8ccb17-857c469885-59tfr" Jul 13 07:33:06.260556 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:33:06.260523 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:33:17.540843 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:33:17.540807 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:33:28.254298 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:33:28.254218 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-8ccb17/test-model-8ccb17-kserve-6ccbcddc77-tf92p"] Jul 13 07:33:28.257960 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:33:28.257942 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-8ccb17/test-model-8ccb17-kserve-6ccbcddc77-tf92p" Jul 13 07:33:28.260853 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:33:28.260827 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-8ccb17\"/\"test-model-8ccb17-kserve-self-signed-certs\"" Jul 13 07:33:28.260971 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:33:28.260854 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-8ccb17\"/\"kube-root-ca.crt\"" Jul 13 07:33:28.260971 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:33:28.260858 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-8ccb17\"/\"default-dockercfg-kgk7p\"" Jul 13 07:33:28.260971 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:33:28.260858 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-8ccb17\"/\"openshift-service-ca.crt\"" Jul 13 07:33:28.266607 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:33:28.266581 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-8ccb17/test-model-8ccb17-kserve-6ccbcddc77-tf92p"] Jul 13 07:33:28.347906 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:33:28.347857 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e745e21b-8a8f-4d0f-8da7-fac3f846c0f1-dshm\") pod \"test-model-8ccb17-kserve-6ccbcddc77-tf92p\" (UID: \"e745e21b-8a8f-4d0f-8da7-fac3f846c0f1\") " pod="ai-tenant-e2e-inf-a-8ccb17/test-model-8ccb17-kserve-6ccbcddc77-tf92p" Jul 13 07:33:28.348077 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:33:28.347958 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e745e21b-8a8f-4d0f-8da7-fac3f846c0f1-tls-certs\") pod \"test-model-8ccb17-kserve-6ccbcddc77-tf92p\" (UID: \"e745e21b-8a8f-4d0f-8da7-fac3f846c0f1\") " pod="ai-tenant-e2e-inf-a-8ccb17/test-model-8ccb17-kserve-6ccbcddc77-tf92p" Jul 13 07:33:28.348077 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:33:28.348008 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9w54\" (UniqueName: \"kubernetes.io/projected/e745e21b-8a8f-4d0f-8da7-fac3f846c0f1-kube-api-access-p9w54\") pod \"test-model-8ccb17-kserve-6ccbcddc77-tf92p\" (UID: \"e745e21b-8a8f-4d0f-8da7-fac3f846c0f1\") " pod="ai-tenant-e2e-inf-a-8ccb17/test-model-8ccb17-kserve-6ccbcddc77-tf92p" Jul 13 07:33:28.348077 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:33:28.348034 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e745e21b-8a8f-4d0f-8da7-fac3f846c0f1-home\") pod \"test-model-8ccb17-kserve-6ccbcddc77-tf92p\" (UID: \"e745e21b-8a8f-4d0f-8da7-fac3f846c0f1\") " pod="ai-tenant-e2e-inf-a-8ccb17/test-model-8ccb17-kserve-6ccbcddc77-tf92p" Jul 13 07:33:28.348077 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:33:28.348051 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e745e21b-8a8f-4d0f-8da7-fac3f846c0f1-model-cache\") pod \"test-model-8ccb17-kserve-6ccbcddc77-tf92p\" (UID: \"e745e21b-8a8f-4d0f-8da7-fac3f846c0f1\") " pod="ai-tenant-e2e-inf-a-8ccb17/test-model-8ccb17-kserve-6ccbcddc77-tf92p" Jul 13 07:33:28.348294 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:33:28.348175 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e745e21b-8a8f-4d0f-8da7-fac3f846c0f1-tmp-dir\") pod \"test-model-8ccb17-kserve-6ccbcddc77-tf92p\" (UID: \"e745e21b-8a8f-4d0f-8da7-fac3f846c0f1\") " pod="ai-tenant-e2e-inf-a-8ccb17/test-model-8ccb17-kserve-6ccbcddc77-tf92p" Jul 13 07:33:28.448919 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:33:28.448882 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e745e21b-8a8f-4d0f-8da7-fac3f846c0f1-tmp-dir\") pod \"test-model-8ccb17-kserve-6ccbcddc77-tf92p\" (UID: \"e745e21b-8a8f-4d0f-8da7-fac3f846c0f1\") " pod="ai-tenant-e2e-inf-a-8ccb17/test-model-8ccb17-kserve-6ccbcddc77-tf92p" Jul 13 07:33:28.449097 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:33:28.448936 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e745e21b-8a8f-4d0f-8da7-fac3f846c0f1-dshm\") pod \"test-model-8ccb17-kserve-6ccbcddc77-tf92p\" (UID: \"e745e21b-8a8f-4d0f-8da7-fac3f846c0f1\") " pod="ai-tenant-e2e-inf-a-8ccb17/test-model-8ccb17-kserve-6ccbcddc77-tf92p" Jul 13 07:33:28.449097 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:33:28.448973 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e745e21b-8a8f-4d0f-8da7-fac3f846c0f1-tls-certs\") pod \"test-model-8ccb17-kserve-6ccbcddc77-tf92p\" (UID: \"e745e21b-8a8f-4d0f-8da7-fac3f846c0f1\") " pod="ai-tenant-e2e-inf-a-8ccb17/test-model-8ccb17-kserve-6ccbcddc77-tf92p" Jul 13 07:33:28.449097 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:33:28.449006 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9w54\" (UniqueName: \"kubernetes.io/projected/e745e21b-8a8f-4d0f-8da7-fac3f846c0f1-kube-api-access-p9w54\") pod \"test-model-8ccb17-kserve-6ccbcddc77-tf92p\" (UID: \"e745e21b-8a8f-4d0f-8da7-fac3f846c0f1\") " pod="ai-tenant-e2e-inf-a-8ccb17/test-model-8ccb17-kserve-6ccbcddc77-tf92p" Jul 13 07:33:28.449097 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:33:28.449032 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e745e21b-8a8f-4d0f-8da7-fac3f846c0f1-home\") pod \"test-model-8ccb17-kserve-6ccbcddc77-tf92p\" (UID: \"e745e21b-8a8f-4d0f-8da7-fac3f846c0f1\") " pod="ai-tenant-e2e-inf-a-8ccb17/test-model-8ccb17-kserve-6ccbcddc77-tf92p" Jul 13 07:33:28.449097 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:33:28.449056 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e745e21b-8a8f-4d0f-8da7-fac3f846c0f1-model-cache\") pod \"test-model-8ccb17-kserve-6ccbcddc77-tf92p\" (UID: \"e745e21b-8a8f-4d0f-8da7-fac3f846c0f1\") " pod="ai-tenant-e2e-inf-a-8ccb17/test-model-8ccb17-kserve-6ccbcddc77-tf92p" Jul 13 07:33:28.449357 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:33:28.449332 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e745e21b-8a8f-4d0f-8da7-fac3f846c0f1-tmp-dir\") pod \"test-model-8ccb17-kserve-6ccbcddc77-tf92p\" (UID: \"e745e21b-8a8f-4d0f-8da7-fac3f846c0f1\") " pod="ai-tenant-e2e-inf-a-8ccb17/test-model-8ccb17-kserve-6ccbcddc77-tf92p" Jul 13 07:33:28.449420 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:33:28.449407 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e745e21b-8a8f-4d0f-8da7-fac3f846c0f1-home\") pod \"test-model-8ccb17-kserve-6ccbcddc77-tf92p\" (UID: \"e745e21b-8a8f-4d0f-8da7-fac3f846c0f1\") " pod="ai-tenant-e2e-inf-a-8ccb17/test-model-8ccb17-kserve-6ccbcddc77-tf92p" Jul 13 07:33:28.449476 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:33:28.449430 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e745e21b-8a8f-4d0f-8da7-fac3f846c0f1-model-cache\") pod \"test-model-8ccb17-kserve-6ccbcddc77-tf92p\" (UID: \"e745e21b-8a8f-4d0f-8da7-fac3f846c0f1\") " pod="ai-tenant-e2e-inf-a-8ccb17/test-model-8ccb17-kserve-6ccbcddc77-tf92p" Jul 13 07:33:28.451275 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:33:28.451251 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e745e21b-8a8f-4d0f-8da7-fac3f846c0f1-dshm\") pod \"test-model-8ccb17-kserve-6ccbcddc77-tf92p\" (UID: \"e745e21b-8a8f-4d0f-8da7-fac3f846c0f1\") " pod="ai-tenant-e2e-inf-a-8ccb17/test-model-8ccb17-kserve-6ccbcddc77-tf92p" Jul 13 07:33:28.451557 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:33:28.451537 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e745e21b-8a8f-4d0f-8da7-fac3f846c0f1-tls-certs\") pod \"test-model-8ccb17-kserve-6ccbcddc77-tf92p\" (UID: \"e745e21b-8a8f-4d0f-8da7-fac3f846c0f1\") " pod="ai-tenant-e2e-inf-a-8ccb17/test-model-8ccb17-kserve-6ccbcddc77-tf92p" Jul 13 07:33:28.457005 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:33:28.456983 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9w54\" (UniqueName: \"kubernetes.io/projected/e745e21b-8a8f-4d0f-8da7-fac3f846c0f1-kube-api-access-p9w54\") pod \"test-model-8ccb17-kserve-6ccbcddc77-tf92p\" (UID: \"e745e21b-8a8f-4d0f-8da7-fac3f846c0f1\") " pod="ai-tenant-e2e-inf-a-8ccb17/test-model-8ccb17-kserve-6ccbcddc77-tf92p" Jul 13 07:33:28.567747 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:33:28.567644 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-8ccb17/test-model-8ccb17-kserve-6ccbcddc77-tf92p" Jul 13 07:33:28.725483 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:33:28.725448 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-8ccb17/test-model-8ccb17-kserve-6ccbcddc77-tf92p"] Jul 13 07:33:28.730521 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:33:28.730493 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode745e21b_8a8f_4d0f_8da7_fac3f846c0f1.slice/crio-5b76f8b171a51342b7c3dd5d05fd72b7f7830b6a2081243727ced26464e21acd WatchSource:0}: Error finding container 5b76f8b171a51342b7c3dd5d05fd72b7f7830b6a2081243727ced26464e21acd: Status 404 returned error can't find the container with id 5b76f8b171a51342b7c3dd5d05fd72b7f7830b6a2081243727ced26464e21acd Jul 13 07:33:29.202553 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:33:29.202512 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-8ccb17/test-model-8ccb17-kserve-6ccbcddc77-tf92p" event={"ID":"e745e21b-8a8f-4d0f-8da7-fac3f846c0f1","Type":"ContainerStarted","Data":"b6ad1e61c0ddeb76f3c94885e1fc5f56b6ed20b0dcfde015901332b3f3a9652c"} Jul 13 07:33:29.202746 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:33:29.202565 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-8ccb17/test-model-8ccb17-kserve-6ccbcddc77-tf92p" event={"ID":"e745e21b-8a8f-4d0f-8da7-fac3f846c0f1","Type":"ContainerStarted","Data":"5b76f8b171a51342b7c3dd5d05fd72b7f7830b6a2081243727ced26464e21acd"} Jul 13 07:33:29.223220 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:33:29.223171 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-8ccb17/test-model-8ccb17-kserve-6ccbcddc77-tf92p" podStartSLOduration=1.223156061 podStartE2EDuration="1.223156061s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:33:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:33:29.220873266 +0000 UTC m=+1884.056437098" watchObservedRunningTime="2026-07-13 07:33:29.223156061 +0000 UTC m=+1884.058719892" Jul 13 07:33:31.649849 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:33:31.649816 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:33:38.568631 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:33:38.568591 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-8ccb17/test-model-8ccb17-kserve-6ccbcddc77-tf92p" Jul 13 07:33:38.569054 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:33:38.568700 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-8ccb17/test-model-8ccb17-kserve-6ccbcddc77-tf92p" Jul 13 07:33:38.581447 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:33:38.581420 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-8ccb17/test-model-8ccb17-kserve-6ccbcddc77-tf92p" Jul 13 07:33:39.271385 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:33:39.271356 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-8ccb17/test-model-8ccb17-kserve-6ccbcddc77-tf92p" Jul 13 07:33:43.345926 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:33:43.345891 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:34:14.647512 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:14.647475 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-3d88d9/test-model-3d88d9-kserve-775cdc4fc-qb8df"] Jul 13 07:34:14.651564 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:14.651545 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-3d88d9/test-model-3d88d9-kserve-775cdc4fc-qb8df" Jul 13 07:34:14.653958 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:14.653933 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-3d88d9\"/\"kube-root-ca.crt\"" Jul 13 07:34:14.654577 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:14.654556 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-3d88d9\"/\"default-dockercfg-2wxqs\"" Jul 13 07:34:14.654725 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:14.654626 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-3d88d9\"/\"test-model-3d88d9-kserve-self-signed-certs\"" Jul 13 07:34:14.654725 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:14.654707 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-3d88d9\"/\"openshift-service-ca.crt\"" Jul 13 07:34:14.660558 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:14.660538 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-3d88d9/test-model-3d88d9-kserve-775cdc4fc-qb8df"] Jul 13 07:34:14.767592 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:14.767552 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a0b7733-6720-46c4-88cb-4576fe9a3734-tls-certs\") pod \"test-model-3d88d9-kserve-775cdc4fc-qb8df\" (UID: \"0a0b7733-6720-46c4-88cb-4576fe9a3734\") " pod="ai-tenant-e2e-inf-b-3d88d9/test-model-3d88d9-kserve-775cdc4fc-qb8df" Jul 13 07:34:14.767831 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:14.767619 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a0b7733-6720-46c4-88cb-4576fe9a3734-model-cache\") pod \"test-model-3d88d9-kserve-775cdc4fc-qb8df\" (UID: \"0a0b7733-6720-46c4-88cb-4576fe9a3734\") " pod="ai-tenant-e2e-inf-b-3d88d9/test-model-3d88d9-kserve-775cdc4fc-qb8df" Jul 13 07:34:14.767831 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:14.767657 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a0b7733-6720-46c4-88cb-4576fe9a3734-dshm\") pod \"test-model-3d88d9-kserve-775cdc4fc-qb8df\" (UID: \"0a0b7733-6720-46c4-88cb-4576fe9a3734\") " pod="ai-tenant-e2e-inf-b-3d88d9/test-model-3d88d9-kserve-775cdc4fc-qb8df" Jul 13 07:34:14.767831 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:14.767710 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a0b7733-6720-46c4-88cb-4576fe9a3734-home\") pod \"test-model-3d88d9-kserve-775cdc4fc-qb8df\" (UID: \"0a0b7733-6720-46c4-88cb-4576fe9a3734\") " pod="ai-tenant-e2e-inf-b-3d88d9/test-model-3d88d9-kserve-775cdc4fc-qb8df" Jul 13 07:34:14.767831 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:14.767737 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a0b7733-6720-46c4-88cb-4576fe9a3734-tmp-dir\") pod \"test-model-3d88d9-kserve-775cdc4fc-qb8df\" (UID: \"0a0b7733-6720-46c4-88cb-4576fe9a3734\") " pod="ai-tenant-e2e-inf-b-3d88d9/test-model-3d88d9-kserve-775cdc4fc-qb8df" Jul 13 07:34:14.767989 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:14.767822 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lp8f\" (UniqueName: \"kubernetes.io/projected/0a0b7733-6720-46c4-88cb-4576fe9a3734-kube-api-access-6lp8f\") pod \"test-model-3d88d9-kserve-775cdc4fc-qb8df\" (UID: \"0a0b7733-6720-46c4-88cb-4576fe9a3734\") " pod="ai-tenant-e2e-inf-b-3d88d9/test-model-3d88d9-kserve-775cdc4fc-qb8df" Jul 13 07:34:14.868461 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:14.868426 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a0b7733-6720-46c4-88cb-4576fe9a3734-model-cache\") pod \"test-model-3d88d9-kserve-775cdc4fc-qb8df\" (UID: \"0a0b7733-6720-46c4-88cb-4576fe9a3734\") " pod="ai-tenant-e2e-inf-b-3d88d9/test-model-3d88d9-kserve-775cdc4fc-qb8df" Jul 13 07:34:14.868657 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:14.868471 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a0b7733-6720-46c4-88cb-4576fe9a3734-dshm\") pod \"test-model-3d88d9-kserve-775cdc4fc-qb8df\" (UID: \"0a0b7733-6720-46c4-88cb-4576fe9a3734\") " pod="ai-tenant-e2e-inf-b-3d88d9/test-model-3d88d9-kserve-775cdc4fc-qb8df" Jul 13 07:34:14.868657 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:14.868490 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a0b7733-6720-46c4-88cb-4576fe9a3734-home\") pod \"test-model-3d88d9-kserve-775cdc4fc-qb8df\" (UID: \"0a0b7733-6720-46c4-88cb-4576fe9a3734\") " pod="ai-tenant-e2e-inf-b-3d88d9/test-model-3d88d9-kserve-775cdc4fc-qb8df" Jul 13 07:34:14.868657 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:14.868509 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a0b7733-6720-46c4-88cb-4576fe9a3734-tmp-dir\") pod \"test-model-3d88d9-kserve-775cdc4fc-qb8df\" (UID: \"0a0b7733-6720-46c4-88cb-4576fe9a3734\") " pod="ai-tenant-e2e-inf-b-3d88d9/test-model-3d88d9-kserve-775cdc4fc-qb8df" Jul 13 07:34:14.868657 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:14.868546 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lp8f\" (UniqueName: \"kubernetes.io/projected/0a0b7733-6720-46c4-88cb-4576fe9a3734-kube-api-access-6lp8f\") pod \"test-model-3d88d9-kserve-775cdc4fc-qb8df\" (UID: \"0a0b7733-6720-46c4-88cb-4576fe9a3734\") " pod="ai-tenant-e2e-inf-b-3d88d9/test-model-3d88d9-kserve-775cdc4fc-qb8df" Jul 13 07:34:14.868657 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:14.868593 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a0b7733-6720-46c4-88cb-4576fe9a3734-tls-certs\") pod \"test-model-3d88d9-kserve-775cdc4fc-qb8df\" (UID: \"0a0b7733-6720-46c4-88cb-4576fe9a3734\") " pod="ai-tenant-e2e-inf-b-3d88d9/test-model-3d88d9-kserve-775cdc4fc-qb8df" Jul 13 07:34:14.869035 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:14.868928 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a0b7733-6720-46c4-88cb-4576fe9a3734-home\") pod \"test-model-3d88d9-kserve-775cdc4fc-qb8df\" (UID: \"0a0b7733-6720-46c4-88cb-4576fe9a3734\") " pod="ai-tenant-e2e-inf-b-3d88d9/test-model-3d88d9-kserve-775cdc4fc-qb8df" Jul 13 07:34:14.869035 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:14.868950 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a0b7733-6720-46c4-88cb-4576fe9a3734-model-cache\") pod \"test-model-3d88d9-kserve-775cdc4fc-qb8df\" (UID: \"0a0b7733-6720-46c4-88cb-4576fe9a3734\") " pod="ai-tenant-e2e-inf-b-3d88d9/test-model-3d88d9-kserve-775cdc4fc-qb8df" Jul 13 07:34:14.869132 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:14.869063 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a0b7733-6720-46c4-88cb-4576fe9a3734-tmp-dir\") pod \"test-model-3d88d9-kserve-775cdc4fc-qb8df\" (UID: \"0a0b7733-6720-46c4-88cb-4576fe9a3734\") " pod="ai-tenant-e2e-inf-b-3d88d9/test-model-3d88d9-kserve-775cdc4fc-qb8df" Jul 13 07:34:14.870883 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:14.870859 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a0b7733-6720-46c4-88cb-4576fe9a3734-dshm\") pod \"test-model-3d88d9-kserve-775cdc4fc-qb8df\" (UID: \"0a0b7733-6720-46c4-88cb-4576fe9a3734\") " pod="ai-tenant-e2e-inf-b-3d88d9/test-model-3d88d9-kserve-775cdc4fc-qb8df" Jul 13 07:34:14.871228 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:14.871209 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a0b7733-6720-46c4-88cb-4576fe9a3734-tls-certs\") pod \"test-model-3d88d9-kserve-775cdc4fc-qb8df\" (UID: \"0a0b7733-6720-46c4-88cb-4576fe9a3734\") " pod="ai-tenant-e2e-inf-b-3d88d9/test-model-3d88d9-kserve-775cdc4fc-qb8df" Jul 13 07:34:14.877190 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:14.877167 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lp8f\" (UniqueName: \"kubernetes.io/projected/0a0b7733-6720-46c4-88cb-4576fe9a3734-kube-api-access-6lp8f\") pod \"test-model-3d88d9-kserve-775cdc4fc-qb8df\" (UID: \"0a0b7733-6720-46c4-88cb-4576fe9a3734\") " pod="ai-tenant-e2e-inf-b-3d88d9/test-model-3d88d9-kserve-775cdc4fc-qb8df" Jul 13 07:34:14.962706 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:14.962645 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-3d88d9/test-model-3d88d9-kserve-775cdc4fc-qb8df" Jul 13 07:34:15.132969 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:15.132811 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-3d88d9/test-model-3d88d9-kserve-775cdc4fc-qb8df"] Jul 13 07:34:15.135879 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:34:15.135850 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0a0b7733_6720_46c4_88cb_4576fe9a3734.slice/crio-467af3a6891da2fd31d6a90477ce9c4578952da131c50db576401b431825dd34 WatchSource:0}: Error finding container 467af3a6891da2fd31d6a90477ce9c4578952da131c50db576401b431825dd34: Status 404 returned error can't find the container with id 467af3a6891da2fd31d6a90477ce9c4578952da131c50db576401b431825dd34 Jul 13 07:34:15.464871 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:15.464835 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-3d88d9/test-model-3d88d9-kserve-775cdc4fc-qb8df" event={"ID":"0a0b7733-6720-46c4-88cb-4576fe9a3734","Type":"ContainerStarted","Data":"0774b25cf470c1bc986fdef4be07f4f1399d4a84a9dc72dbe6e77afbd8b4502e"} Jul 13 07:34:15.465072 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:15.464878 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-3d88d9/test-model-3d88d9-kserve-775cdc4fc-qb8df" event={"ID":"0a0b7733-6720-46c4-88cb-4576fe9a3734","Type":"ContainerStarted","Data":"467af3a6891da2fd31d6a90477ce9c4578952da131c50db576401b431825dd34"} Jul 13 07:34:15.482790 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:15.482741 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-3d88d9/test-model-3d88d9-kserve-775cdc4fc-qb8df" podStartSLOduration=1.482723001 podStartE2EDuration="1.482723001s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:34:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:34:15.48098206 +0000 UTC m=+1930.316545891" watchObservedRunningTime="2026-07-13 07:34:15.482723001 +0000 UTC m=+1930.318286835" Jul 13 07:34:19.143268 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:19.143234 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:34:24.963794 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:24.963751 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-3d88d9/test-model-3d88d9-kserve-775cdc4fc-qb8df" Jul 13 07:34:24.964340 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:24.963827 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-3d88d9/test-model-3d88d9-kserve-775cdc4fc-qb8df" Jul 13 07:34:24.976658 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:24.976634 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-3d88d9/test-model-3d88d9-kserve-775cdc4fc-qb8df" Jul 13 07:34:25.531073 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:25.531034 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-3d88d9/test-model-3d88d9-kserve-775cdc4fc-qb8df" Jul 13 07:34:33.143923 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:33.143885 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:34:44.549331 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:44.549297 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-8ccb17/test-model-8ccb17-kserve-6ccbcddc77-tf92p"] Jul 13 07:34:45.451102 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:45.451069 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-3d88d9/test-model-3d88d9-kserve-775cdc4fc-qb8df"] Jul 13 07:34:46.550528 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:46.550488 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-8ccb17-857c469885-59tfr"] Jul 13 07:34:46.550991 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:46.550759 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8ccb17-857c469885-59tfr" podUID="ea4f1126-8ac7-4f29-931e-a0ba11c61cd1" containerName="maas-api" containerID="cri-o://9d1b0a36ce8c09ba5fe1c6ef536df40d06402ae7ddb9d5819df9f0bb66d0a3a7" gracePeriod=30 Jul 13 07:34:46.817765 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:46.817740 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8ccb17-857c469885-59tfr" Jul 13 07:34:46.963435 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:46.963399 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m85fz\" (UniqueName: \"kubernetes.io/projected/ea4f1126-8ac7-4f29-931e-a0ba11c61cd1-kube-api-access-m85fz\") pod \"ea4f1126-8ac7-4f29-931e-a0ba11c61cd1\" (UID: \"ea4f1126-8ac7-4f29-931e-a0ba11c61cd1\") " Jul 13 07:34:46.963604 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:46.963451 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea4f1126-8ac7-4f29-931e-a0ba11c61cd1-maas-api-tls\") pod \"ea4f1126-8ac7-4f29-931e-a0ba11c61cd1\" (UID: \"ea4f1126-8ac7-4f29-931e-a0ba11c61cd1\") " Jul 13 07:34:46.963604 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:46.963500 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea4f1126-8ac7-4f29-931e-a0ba11c61cd1-openshift-service-ca\") pod \"ea4f1126-8ac7-4f29-931e-a0ba11c61cd1\" (UID: \"ea4f1126-8ac7-4f29-931e-a0ba11c61cd1\") " Jul 13 07:34:46.963949 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:46.963918 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea4f1126-8ac7-4f29-931e-a0ba11c61cd1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ea4f1126-8ac7-4f29-931e-a0ba11c61cd1" (UID: "ea4f1126-8ac7-4f29-931e-a0ba11c61cd1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 07:34:46.965576 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:46.965547 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea4f1126-8ac7-4f29-931e-a0ba11c61cd1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ea4f1126-8ac7-4f29-931e-a0ba11c61cd1" (UID: "ea4f1126-8ac7-4f29-931e-a0ba11c61cd1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 07:34:46.965693 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:46.965602 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea4f1126-8ac7-4f29-931e-a0ba11c61cd1-kube-api-access-m85fz" (OuterVolumeSpecName: "kube-api-access-m85fz") pod "ea4f1126-8ac7-4f29-931e-a0ba11c61cd1" (UID: "ea4f1126-8ac7-4f29-931e-a0ba11c61cd1"). InnerVolumeSpecName "kube-api-access-m85fz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:34:47.065114 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:47.065021 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m85fz\" (UniqueName: \"kubernetes.io/projected/ea4f1126-8ac7-4f29-931e-a0ba11c61cd1-kube-api-access-m85fz\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:34:47.065114 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:47.065055 2586 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea4f1126-8ac7-4f29-931e-a0ba11c61cd1-maas-api-tls\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:34:47.065114 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:47.065065 2586 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea4f1126-8ac7-4f29-931e-a0ba11c61cd1-openshift-service-ca\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:34:47.661852 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:47.661817 2586 generic.go:356] "Generic (PLEG): container finished" podID="ea4f1126-8ac7-4f29-931e-a0ba11c61cd1" containerID="9d1b0a36ce8c09ba5fe1c6ef536df40d06402ae7ddb9d5819df9f0bb66d0a3a7" exitCode=0 Jul 13 07:34:47.662278 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:47.661858 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8ccb17-857c469885-59tfr" event={"ID":"ea4f1126-8ac7-4f29-931e-a0ba11c61cd1","Type":"ContainerDied","Data":"9d1b0a36ce8c09ba5fe1c6ef536df40d06402ae7ddb9d5819df9f0bb66d0a3a7"} Jul 13 07:34:47.662278 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:47.661864 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8ccb17-857c469885-59tfr" Jul 13 07:34:47.662278 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:47.661895 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8ccb17-857c469885-59tfr" event={"ID":"ea4f1126-8ac7-4f29-931e-a0ba11c61cd1","Type":"ContainerDied","Data":"69d167e266856a765927aa489c8847e68e2dca0fb95b9900184ed43828efd29b"} Jul 13 07:34:47.662278 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:47.661918 2586 scope.go:117] "RemoveContainer" containerID="9d1b0a36ce8c09ba5fe1c6ef536df40d06402ae7ddb9d5819df9f0bb66d0a3a7" Jul 13 07:34:47.672176 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:47.672157 2586 scope.go:117] "RemoveContainer" containerID="9d1b0a36ce8c09ba5fe1c6ef536df40d06402ae7ddb9d5819df9f0bb66d0a3a7" Jul 13 07:34:47.672466 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:34:47.672437 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d1b0a36ce8c09ba5fe1c6ef536df40d06402ae7ddb9d5819df9f0bb66d0a3a7\": container with ID starting with 9d1b0a36ce8c09ba5fe1c6ef536df40d06402ae7ddb9d5819df9f0bb66d0a3a7 not found: ID does not exist" containerID="9d1b0a36ce8c09ba5fe1c6ef536df40d06402ae7ddb9d5819df9f0bb66d0a3a7" Jul 13 07:34:47.672543 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:47.672479 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d1b0a36ce8c09ba5fe1c6ef536df40d06402ae7ddb9d5819df9f0bb66d0a3a7"} err="failed to get container status \"9d1b0a36ce8c09ba5fe1c6ef536df40d06402ae7ddb9d5819df9f0bb66d0a3a7\": rpc error: code = NotFound desc = could not find container \"9d1b0a36ce8c09ba5fe1c6ef536df40d06402ae7ddb9d5819df9f0bb66d0a3a7\": container with ID starting with 9d1b0a36ce8c09ba5fe1c6ef536df40d06402ae7ddb9d5819df9f0bb66d0a3a7 not found: ID does not exist" Jul 13 07:34:47.685118 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:47.685093 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-8ccb17-857c469885-59tfr"] Jul 13 07:34:47.690031 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:47.690009 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-8ccb17-857c469885-59tfr"] Jul 13 07:34:47.805339 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:47.805303 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea4f1126-8ac7-4f29-931e-a0ba11c61cd1" path="/var/lib/kubelet/pods/ea4f1126-8ac7-4f29-931e-a0ba11c61cd1/volumes" Jul 13 07:34:50.850061 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:50.850028 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:34:59.569932 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:59.569838 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-8ccb17/test-model-8ccb17-kserve-6ccbcddc77-tf92p" podUID="e745e21b-8a8f-4d0f-8da7-fac3f846c0f1" containerName="main" containerID="cri-o://b6ad1e61c0ddeb76f3c94885e1fc5f56b6ed20b0dcfde015901332b3f3a9652c" gracePeriod=45 Jul 13 07:34:59.734917 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:59.734884 2586 generic.go:356] "Generic (PLEG): container finished" podID="e745e21b-8a8f-4d0f-8da7-fac3f846c0f1" containerID="b6ad1e61c0ddeb76f3c94885e1fc5f56b6ed20b0dcfde015901332b3f3a9652c" exitCode=0 Jul 13 07:34:59.735094 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:59.734914 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-8ccb17/test-model-8ccb17-kserve-6ccbcddc77-tf92p" event={"ID":"e745e21b-8a8f-4d0f-8da7-fac3f846c0f1","Type":"ContainerDied","Data":"b6ad1e61c0ddeb76f3c94885e1fc5f56b6ed20b0dcfde015901332b3f3a9652c"} Jul 13 07:34:59.830002 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:59.829950 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-8ccb17/test-model-8ccb17-kserve-6ccbcddc77-tf92p" Jul 13 07:34:59.993523 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:59.993479 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p9w54\" (UniqueName: \"kubernetes.io/projected/e745e21b-8a8f-4d0f-8da7-fac3f846c0f1-kube-api-access-p9w54\") pod \"e745e21b-8a8f-4d0f-8da7-fac3f846c0f1\" (UID: \"e745e21b-8a8f-4d0f-8da7-fac3f846c0f1\") " Jul 13 07:34:59.993523 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:59.993522 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e745e21b-8a8f-4d0f-8da7-fac3f846c0f1-tls-certs\") pod \"e745e21b-8a8f-4d0f-8da7-fac3f846c0f1\" (UID: \"e745e21b-8a8f-4d0f-8da7-fac3f846c0f1\") " Jul 13 07:34:59.993830 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:59.993570 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e745e21b-8a8f-4d0f-8da7-fac3f846c0f1-model-cache\") pod \"e745e21b-8a8f-4d0f-8da7-fac3f846c0f1\" (UID: \"e745e21b-8a8f-4d0f-8da7-fac3f846c0f1\") " Jul 13 07:34:59.993830 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:59.993623 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e745e21b-8a8f-4d0f-8da7-fac3f846c0f1-home\") pod \"e745e21b-8a8f-4d0f-8da7-fac3f846c0f1\" (UID: \"e745e21b-8a8f-4d0f-8da7-fac3f846c0f1\") " Jul 13 07:34:59.993830 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:59.993668 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e745e21b-8a8f-4d0f-8da7-fac3f846c0f1-dshm\") pod \"e745e21b-8a8f-4d0f-8da7-fac3f846c0f1\" (UID: \"e745e21b-8a8f-4d0f-8da7-fac3f846c0f1\") " Jul 13 07:34:59.993830 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:59.993782 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e745e21b-8a8f-4d0f-8da7-fac3f846c0f1-tmp-dir\") pod \"e745e21b-8a8f-4d0f-8da7-fac3f846c0f1\" (UID: \"e745e21b-8a8f-4d0f-8da7-fac3f846c0f1\") " Jul 13 07:34:59.994045 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:59.993958 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e745e21b-8a8f-4d0f-8da7-fac3f846c0f1-model-cache" (OuterVolumeSpecName: "model-cache") pod "e745e21b-8a8f-4d0f-8da7-fac3f846c0f1" (UID: "e745e21b-8a8f-4d0f-8da7-fac3f846c0f1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:34:59.994045 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:59.993972 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e745e21b-8a8f-4d0f-8da7-fac3f846c0f1-home" (OuterVolumeSpecName: "home") pod "e745e21b-8a8f-4d0f-8da7-fac3f846c0f1" (UID: "e745e21b-8a8f-4d0f-8da7-fac3f846c0f1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:34:59.994207 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:59.994182 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e745e21b-8a8f-4d0f-8da7-fac3f846c0f1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e745e21b-8a8f-4d0f-8da7-fac3f846c0f1" (UID: "e745e21b-8a8f-4d0f-8da7-fac3f846c0f1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:34:59.994375 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:59.994202 2586 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e745e21b-8a8f-4d0f-8da7-fac3f846c0f1-model-cache\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:34:59.994501 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:59.994474 2586 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e745e21b-8a8f-4d0f-8da7-fac3f846c0f1-home\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:34:59.996068 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:59.996035 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e745e21b-8a8f-4d0f-8da7-fac3f846c0f1-dshm" (OuterVolumeSpecName: "dshm") pod "e745e21b-8a8f-4d0f-8da7-fac3f846c0f1" (UID: "e745e21b-8a8f-4d0f-8da7-fac3f846c0f1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:34:59.996068 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:59.996059 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e745e21b-8a8f-4d0f-8da7-fac3f846c0f1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e745e21b-8a8f-4d0f-8da7-fac3f846c0f1" (UID: "e745e21b-8a8f-4d0f-8da7-fac3f846c0f1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 07:34:59.996185 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:34:59.996121 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e745e21b-8a8f-4d0f-8da7-fac3f846c0f1-kube-api-access-p9w54" (OuterVolumeSpecName: "kube-api-access-p9w54") pod "e745e21b-8a8f-4d0f-8da7-fac3f846c0f1" (UID: "e745e21b-8a8f-4d0f-8da7-fac3f846c0f1"). InnerVolumeSpecName "kube-api-access-p9w54". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:35:00.095138 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:00.095042 2586 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e745e21b-8a8f-4d0f-8da7-fac3f846c0f1-dshm\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:35:00.095138 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:00.095083 2586 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e745e21b-8a8f-4d0f-8da7-fac3f846c0f1-tmp-dir\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:35:00.095138 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:00.095094 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p9w54\" (UniqueName: \"kubernetes.io/projected/e745e21b-8a8f-4d0f-8da7-fac3f846c0f1-kube-api-access-p9w54\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:35:00.095138 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:00.095106 2586 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e745e21b-8a8f-4d0f-8da7-fac3f846c0f1-tls-certs\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:35:00.471906 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:00.471865 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-3d88d9/test-model-3d88d9-kserve-775cdc4fc-qb8df" podUID="0a0b7733-6720-46c4-88cb-4576fe9a3734" containerName="main" containerID="cri-o://0774b25cf470c1bc986fdef4be07f4f1399d4a84a9dc72dbe6e77afbd8b4502e" gracePeriod=45 Jul 13 07:35:00.725662 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:00.725606 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-3d88d9/test-model-3d88d9-kserve-775cdc4fc-qb8df" Jul 13 07:35:00.741983 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:00.741911 2586 generic.go:356] "Generic (PLEG): container finished" podID="0a0b7733-6720-46c4-88cb-4576fe9a3734" containerID="0774b25cf470c1bc986fdef4be07f4f1399d4a84a9dc72dbe6e77afbd8b4502e" exitCode=0 Jul 13 07:35:00.741983 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:00.741973 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-3d88d9/test-model-3d88d9-kserve-775cdc4fc-qb8df" Jul 13 07:35:00.742175 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:00.742100 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-3d88d9/test-model-3d88d9-kserve-775cdc4fc-qb8df" event={"ID":"0a0b7733-6720-46c4-88cb-4576fe9a3734","Type":"ContainerDied","Data":"0774b25cf470c1bc986fdef4be07f4f1399d4a84a9dc72dbe6e77afbd8b4502e"} Jul 13 07:35:00.742175 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:00.742139 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-3d88d9/test-model-3d88d9-kserve-775cdc4fc-qb8df" event={"ID":"0a0b7733-6720-46c4-88cb-4576fe9a3734","Type":"ContainerDied","Data":"467af3a6891da2fd31d6a90477ce9c4578952da131c50db576401b431825dd34"} Jul 13 07:35:00.742175 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:00.742161 2586 scope.go:117] "RemoveContainer" containerID="0774b25cf470c1bc986fdef4be07f4f1399d4a84a9dc72dbe6e77afbd8b4502e" Jul 13 07:35:00.743888 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:00.743865 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-8ccb17/test-model-8ccb17-kserve-6ccbcddc77-tf92p" event={"ID":"e745e21b-8a8f-4d0f-8da7-fac3f846c0f1","Type":"ContainerDied","Data":"5b76f8b171a51342b7c3dd5d05fd72b7f7830b6a2081243727ced26464e21acd"} Jul 13 07:35:00.743888 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:00.743885 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-8ccb17/test-model-8ccb17-kserve-6ccbcddc77-tf92p" Jul 13 07:35:00.763157 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:00.763125 2586 scope.go:117] "RemoveContainer" containerID="0774b25cf470c1bc986fdef4be07f4f1399d4a84a9dc72dbe6e77afbd8b4502e" Jul 13 07:35:00.763714 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:35:00.763666 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0774b25cf470c1bc986fdef4be07f4f1399d4a84a9dc72dbe6e77afbd8b4502e\": container with ID starting with 0774b25cf470c1bc986fdef4be07f4f1399d4a84a9dc72dbe6e77afbd8b4502e not found: ID does not exist" containerID="0774b25cf470c1bc986fdef4be07f4f1399d4a84a9dc72dbe6e77afbd8b4502e" Jul 13 07:35:00.763818 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:00.763723 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0774b25cf470c1bc986fdef4be07f4f1399d4a84a9dc72dbe6e77afbd8b4502e"} err="failed to get container status \"0774b25cf470c1bc986fdef4be07f4f1399d4a84a9dc72dbe6e77afbd8b4502e\": rpc error: code = NotFound desc = could not find container \"0774b25cf470c1bc986fdef4be07f4f1399d4a84a9dc72dbe6e77afbd8b4502e\": container with ID starting with 0774b25cf470c1bc986fdef4be07f4f1399d4a84a9dc72dbe6e77afbd8b4502e not found: ID does not exist" Jul 13 07:35:00.763818 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:00.763742 2586 scope.go:117] "RemoveContainer" containerID="b6ad1e61c0ddeb76f3c94885e1fc5f56b6ed20b0dcfde015901332b3f3a9652c" Jul 13 07:35:00.774968 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:00.774915 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-8ccb17/test-model-8ccb17-kserve-6ccbcddc77-tf92p"] Jul 13 07:35:00.777356 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:00.777336 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-8ccb17/test-model-8ccb17-kserve-6ccbcddc77-tf92p"] Jul 13 07:35:00.901784 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:00.901749 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a0b7733-6720-46c4-88cb-4576fe9a3734-home\") pod \"0a0b7733-6720-46c4-88cb-4576fe9a3734\" (UID: \"0a0b7733-6720-46c4-88cb-4576fe9a3734\") " Jul 13 07:35:00.901963 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:00.901822 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a0b7733-6720-46c4-88cb-4576fe9a3734-tls-certs\") pod \"0a0b7733-6720-46c4-88cb-4576fe9a3734\" (UID: \"0a0b7733-6720-46c4-88cb-4576fe9a3734\") " Jul 13 07:35:00.901963 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:00.901870 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a0b7733-6720-46c4-88cb-4576fe9a3734-tmp-dir\") pod \"0a0b7733-6720-46c4-88cb-4576fe9a3734\" (UID: \"0a0b7733-6720-46c4-88cb-4576fe9a3734\") " Jul 13 07:35:00.901963 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:00.901887 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6lp8f\" (UniqueName: \"kubernetes.io/projected/0a0b7733-6720-46c4-88cb-4576fe9a3734-kube-api-access-6lp8f\") pod \"0a0b7733-6720-46c4-88cb-4576fe9a3734\" (UID: \"0a0b7733-6720-46c4-88cb-4576fe9a3734\") " Jul 13 07:35:00.901963 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:00.901908 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a0b7733-6720-46c4-88cb-4576fe9a3734-dshm\") pod \"0a0b7733-6720-46c4-88cb-4576fe9a3734\" (UID: \"0a0b7733-6720-46c4-88cb-4576fe9a3734\") " Jul 13 07:35:00.901963 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:00.901957 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a0b7733-6720-46c4-88cb-4576fe9a3734-model-cache\") pod \"0a0b7733-6720-46c4-88cb-4576fe9a3734\" (UID: \"0a0b7733-6720-46c4-88cb-4576fe9a3734\") " Jul 13 07:35:00.902239 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:00.902022 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a0b7733-6720-46c4-88cb-4576fe9a3734-home" (OuterVolumeSpecName: "home") pod "0a0b7733-6720-46c4-88cb-4576fe9a3734" (UID: "0a0b7733-6720-46c4-88cb-4576fe9a3734"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:35:00.902239 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:00.902184 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a0b7733-6720-46c4-88cb-4576fe9a3734-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0a0b7733-6720-46c4-88cb-4576fe9a3734" (UID: "0a0b7733-6720-46c4-88cb-4576fe9a3734"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:35:00.902239 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:00.902220 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a0b7733-6720-46c4-88cb-4576fe9a3734-model-cache" (OuterVolumeSpecName: "model-cache") pod "0a0b7733-6720-46c4-88cb-4576fe9a3734" (UID: "0a0b7733-6720-46c4-88cb-4576fe9a3734"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:35:00.902383 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:00.902362 2586 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a0b7733-6720-46c4-88cb-4576fe9a3734-home\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:35:00.902383 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:00.902378 2586 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a0b7733-6720-46c4-88cb-4576fe9a3734-tmp-dir\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:35:00.902494 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:00.902387 2586 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a0b7733-6720-46c4-88cb-4576fe9a3734-model-cache\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:35:00.904236 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:00.904209 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a0b7733-6720-46c4-88cb-4576fe9a3734-dshm" (OuterVolumeSpecName: "dshm") pod "0a0b7733-6720-46c4-88cb-4576fe9a3734" (UID: "0a0b7733-6720-46c4-88cb-4576fe9a3734"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 07:35:00.904345 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:00.904255 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a0b7733-6720-46c4-88cb-4576fe9a3734-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0a0b7733-6720-46c4-88cb-4576fe9a3734" (UID: "0a0b7733-6720-46c4-88cb-4576fe9a3734"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 07:35:00.904345 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:00.904267 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a0b7733-6720-46c4-88cb-4576fe9a3734-kube-api-access-6lp8f" (OuterVolumeSpecName: "kube-api-access-6lp8f") pod "0a0b7733-6720-46c4-88cb-4576fe9a3734" (UID: "0a0b7733-6720-46c4-88cb-4576fe9a3734"). InnerVolumeSpecName "kube-api-access-6lp8f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:35:01.002911 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:01.002816 2586 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a0b7733-6720-46c4-88cb-4576fe9a3734-tls-certs\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:35:01.002911 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:01.002853 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6lp8f\" (UniqueName: \"kubernetes.io/projected/0a0b7733-6720-46c4-88cb-4576fe9a3734-kube-api-access-6lp8f\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:35:01.002911 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:01.002864 2586 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a0b7733-6720-46c4-88cb-4576fe9a3734-dshm\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:35:01.064632 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:01.064590 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-3d88d9/test-model-3d88d9-kserve-775cdc4fc-qb8df"] Jul 13 07:35:01.068650 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:01.068624 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-3d88d9/test-model-3d88d9-kserve-775cdc4fc-qb8df"] Jul 13 07:35:01.805802 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:01.805758 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a0b7733-6720-46c4-88cb-4576fe9a3734" path="/var/lib/kubelet/pods/0a0b7733-6720-46c4-88cb-4576fe9a3734/volumes" Jul 13 07:35:01.806353 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:01.806331 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e745e21b-8a8f-4d0f-8da7-fac3f846c0f1" path="/var/lib/kubelet/pods/e745e21b-8a8f-4d0f-8da7-fac3f846c0f1/volumes" Jul 13 07:35:31.434832 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:31.434795 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:35:39.576907 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:39.576872 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1f88a7-78d769fff-n44lq"] Jul 13 07:35:39.577470 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:39.577449 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e745e21b-8a8f-4d0f-8da7-fac3f846c0f1" containerName="main" Jul 13 07:35:39.577552 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:39.577473 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="e745e21b-8a8f-4d0f-8da7-fac3f846c0f1" containerName="main" Jul 13 07:35:39.577552 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:39.577498 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea4f1126-8ac7-4f29-931e-a0ba11c61cd1" containerName="maas-api" Jul 13 07:35:39.577552 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:39.577507 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea4f1126-8ac7-4f29-931e-a0ba11c61cd1" containerName="maas-api" Jul 13 07:35:39.577552 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:39.577523 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a0b7733-6720-46c4-88cb-4576fe9a3734" containerName="main" Jul 13 07:35:39.577552 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:39.577531 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a0b7733-6720-46c4-88cb-4576fe9a3734" containerName="main" Jul 13 07:35:39.577824 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:39.577645 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a0b7733-6720-46c4-88cb-4576fe9a3734" containerName="main" Jul 13 07:35:39.577824 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:39.577662 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea4f1126-8ac7-4f29-931e-a0ba11c61cd1" containerName="maas-api" Jul 13 07:35:39.577824 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:39.577676 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="e745e21b-8a8f-4d0f-8da7-fac3f846c0f1" containerName="main" Jul 13 07:35:39.580707 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:39.580667 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1f88a7-78d769fff-n44lq" Jul 13 07:35:39.582738 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:39.582714 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8rh7l\"" Jul 13 07:35:39.582851 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:39.582741 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-1f88a7-serving-cert\"" Jul 13 07:35:39.583232 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:39.583209 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 13 07:35:39.583414 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:39.583395 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 13 07:35:39.593126 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:39.593100 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1f88a7-78d769fff-n44lq"] Jul 13 07:35:39.716547 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:39.716503 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zm6s5\" (UniqueName: \"kubernetes.io/projected/b73d131a-e4f0-44cd-98cd-c1177b0a445e-kube-api-access-zm6s5\") pod \"maas-api-e2e-api-a-1f88a7-78d769fff-n44lq\" (UID: \"b73d131a-e4f0-44cd-98cd-c1177b0a445e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1f88a7-78d769fff-n44lq" Jul 13 07:35:39.716788 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:39.716561 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b73d131a-e4f0-44cd-98cd-c1177b0a445e-maas-api-tls\") pod \"maas-api-e2e-api-a-1f88a7-78d769fff-n44lq\" (UID: \"b73d131a-e4f0-44cd-98cd-c1177b0a445e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1f88a7-78d769fff-n44lq" Jul 13 07:35:39.716788 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:39.716626 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b73d131a-e4f0-44cd-98cd-c1177b0a445e-openshift-service-ca\") pod \"maas-api-e2e-api-a-1f88a7-78d769fff-n44lq\" (UID: \"b73d131a-e4f0-44cd-98cd-c1177b0a445e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1f88a7-78d769fff-n44lq" Jul 13 07:35:39.818065 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:39.818032 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b73d131a-e4f0-44cd-98cd-c1177b0a445e-openshift-service-ca\") pod \"maas-api-e2e-api-a-1f88a7-78d769fff-n44lq\" (UID: \"b73d131a-e4f0-44cd-98cd-c1177b0a445e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1f88a7-78d769fff-n44lq" Jul 13 07:35:39.818268 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:39.818130 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zm6s5\" (UniqueName: \"kubernetes.io/projected/b73d131a-e4f0-44cd-98cd-c1177b0a445e-kube-api-access-zm6s5\") pod \"maas-api-e2e-api-a-1f88a7-78d769fff-n44lq\" (UID: \"b73d131a-e4f0-44cd-98cd-c1177b0a445e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1f88a7-78d769fff-n44lq" Jul 13 07:35:39.818268 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:39.818168 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b73d131a-e4f0-44cd-98cd-c1177b0a445e-maas-api-tls\") pod \"maas-api-e2e-api-a-1f88a7-78d769fff-n44lq\" (UID: \"b73d131a-e4f0-44cd-98cd-c1177b0a445e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1f88a7-78d769fff-n44lq" Jul 13 07:35:39.818718 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:39.818669 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b73d131a-e4f0-44cd-98cd-c1177b0a445e-openshift-service-ca\") pod \"maas-api-e2e-api-a-1f88a7-78d769fff-n44lq\" (UID: \"b73d131a-e4f0-44cd-98cd-c1177b0a445e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1f88a7-78d769fff-n44lq" Jul 13 07:35:39.820561 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:39.820540 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b73d131a-e4f0-44cd-98cd-c1177b0a445e-maas-api-tls\") pod \"maas-api-e2e-api-a-1f88a7-78d769fff-n44lq\" (UID: \"b73d131a-e4f0-44cd-98cd-c1177b0a445e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1f88a7-78d769fff-n44lq" Jul 13 07:35:39.828497 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:39.828433 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zm6s5\" (UniqueName: \"kubernetes.io/projected/b73d131a-e4f0-44cd-98cd-c1177b0a445e-kube-api-access-zm6s5\") pod \"maas-api-e2e-api-a-1f88a7-78d769fff-n44lq\" (UID: \"b73d131a-e4f0-44cd-98cd-c1177b0a445e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1f88a7-78d769fff-n44lq" Jul 13 07:35:39.892950 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:39.892912 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1f88a7-78d769fff-n44lq" Jul 13 07:35:40.036584 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:40.036561 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1f88a7-78d769fff-n44lq"] Jul 13 07:35:40.038205 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:35:40.038164 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb73d131a_e4f0_44cd_98cd_c1177b0a445e.slice/crio-8f19c9cc67165e58fe0b8d3765b0d627d2f552c989604fb7ee60238f998928d9 WatchSource:0}: Error finding container 8f19c9cc67165e58fe0b8d3765b0d627d2f552c989604fb7ee60238f998928d9: Status 404 returned error can't find the container with id 8f19c9cc67165e58fe0b8d3765b0d627d2f552c989604fb7ee60238f998928d9 Jul 13 07:35:40.969590 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:40.969555 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1f88a7-78d769fff-n44lq" event={"ID":"b73d131a-e4f0-44cd-98cd-c1177b0a445e","Type":"ContainerStarted","Data":"3f2245cba11aa4fa6d13a2dbffef62e977941202496889b58075a0e8b3ea0390"} Jul 13 07:35:40.969962 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:40.969605 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1f88a7-78d769fff-n44lq" event={"ID":"b73d131a-e4f0-44cd-98cd-c1177b0a445e","Type":"ContainerStarted","Data":"8f19c9cc67165e58fe0b8d3765b0d627d2f552c989604fb7ee60238f998928d9"} Jul 13 07:35:40.969962 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:40.969629 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1f88a7-78d769fff-n44lq" Jul 13 07:35:40.989556 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:40.989508 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1f88a7-78d769fff-n44lq" podStartSLOduration=1.529110704 podStartE2EDuration="1.989494526s" totalImagesPullingTime="460.383822ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:35:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:35:40.988073334 +0000 UTC m=+2015.823637156" watchObservedRunningTime="2026-07-13 07:35:40.989494526 +0000 UTC m=+2015.825058357" Jul 13 07:35:45.042389 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:45.042356 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:35:46.981240 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:46.981211 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1f88a7-78d769fff-n44lq" Jul 13 07:35:58.145635 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:35:58.145597 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:36:00.918700 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:36:00.918659 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1f88a7-78d769fff-n44lq"] Jul 13 07:36:00.919142 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:36:00.919008 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1f88a7-78d769fff-n44lq" podUID="b73d131a-e4f0-44cd-98cd-c1177b0a445e" containerName="maas-api" containerID="cri-o://3f2245cba11aa4fa6d13a2dbffef62e977941202496889b58075a0e8b3ea0390" gracePeriod=30 Jul 13 07:36:01.091502 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:36:01.091461 2586 generic.go:356] "Generic (PLEG): container finished" podID="b73d131a-e4f0-44cd-98cd-c1177b0a445e" containerID="3f2245cba11aa4fa6d13a2dbffef62e977941202496889b58075a0e8b3ea0390" exitCode=0 Jul 13 07:36:01.091635 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:36:01.091579 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1f88a7-78d769fff-n44lq" event={"ID":"b73d131a-e4f0-44cd-98cd-c1177b0a445e","Type":"ContainerDied","Data":"3f2245cba11aa4fa6d13a2dbffef62e977941202496889b58075a0e8b3ea0390"} Jul 13 07:36:01.175653 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:36:01.175590 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1f88a7-78d769fff-n44lq" Jul 13 07:36:01.313306 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:36:01.313257 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zm6s5\" (UniqueName: \"kubernetes.io/projected/b73d131a-e4f0-44cd-98cd-c1177b0a445e-kube-api-access-zm6s5\") pod \"b73d131a-e4f0-44cd-98cd-c1177b0a445e\" (UID: \"b73d131a-e4f0-44cd-98cd-c1177b0a445e\") " Jul 13 07:36:01.313306 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:36:01.313315 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b73d131a-e4f0-44cd-98cd-c1177b0a445e-openshift-service-ca\") pod \"b73d131a-e4f0-44cd-98cd-c1177b0a445e\" (UID: \"b73d131a-e4f0-44cd-98cd-c1177b0a445e\") " Jul 13 07:36:01.313306 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:36:01.313336 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b73d131a-e4f0-44cd-98cd-c1177b0a445e-maas-api-tls\") pod \"b73d131a-e4f0-44cd-98cd-c1177b0a445e\" (UID: \"b73d131a-e4f0-44cd-98cd-c1177b0a445e\") " Jul 13 07:36:01.313840 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:36:01.313811 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b73d131a-e4f0-44cd-98cd-c1177b0a445e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b73d131a-e4f0-44cd-98cd-c1177b0a445e" (UID: "b73d131a-e4f0-44cd-98cd-c1177b0a445e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 07:36:01.315547 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:36:01.315522 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b73d131a-e4f0-44cd-98cd-c1177b0a445e-kube-api-access-zm6s5" (OuterVolumeSpecName: "kube-api-access-zm6s5") pod "b73d131a-e4f0-44cd-98cd-c1177b0a445e" (UID: "b73d131a-e4f0-44cd-98cd-c1177b0a445e"). InnerVolumeSpecName "kube-api-access-zm6s5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:36:01.315633 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:36:01.315608 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b73d131a-e4f0-44cd-98cd-c1177b0a445e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b73d131a-e4f0-44cd-98cd-c1177b0a445e" (UID: "b73d131a-e4f0-44cd-98cd-c1177b0a445e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 07:36:01.414835 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:36:01.414797 2586 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b73d131a-e4f0-44cd-98cd-c1177b0a445e-maas-api-tls\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:36:01.414835 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:36:01.414830 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zm6s5\" (UniqueName: \"kubernetes.io/projected/b73d131a-e4f0-44cd-98cd-c1177b0a445e-kube-api-access-zm6s5\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:36:01.414835 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:36:01.414840 2586 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b73d131a-e4f0-44cd-98cd-c1177b0a445e-openshift-service-ca\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:36:02.098902 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:36:02.098817 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1f88a7-78d769fff-n44lq" event={"ID":"b73d131a-e4f0-44cd-98cd-c1177b0a445e","Type":"ContainerDied","Data":"8f19c9cc67165e58fe0b8d3765b0d627d2f552c989604fb7ee60238f998928d9"} Jul 13 07:36:02.098902 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:36:02.098885 2586 scope.go:117] "RemoveContainer" containerID="3f2245cba11aa4fa6d13a2dbffef62e977941202496889b58075a0e8b3ea0390" Jul 13 07:36:02.099387 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:36:02.098829 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1f88a7-78d769fff-n44lq" Jul 13 07:36:02.117015 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:36:02.116987 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1f88a7-78d769fff-n44lq"] Jul 13 07:36:02.121299 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:36:02.121275 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1f88a7-78d769fff-n44lq"] Jul 13 07:36:03.805584 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:36:03.805550 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b73d131a-e4f0-44cd-98cd-c1177b0a445e" path="/var/lib/kubelet/pods/b73d131a-e4f0-44cd-98cd-c1177b0a445e/volumes" Jul 13 07:36:11.049988 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:36:11.049954 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:36:17.857355 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:36:17.857312 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-cc53e7-cd447d5d7-c4xw9"] Jul 13 07:36:17.857833 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:36:17.857789 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b73d131a-e4f0-44cd-98cd-c1177b0a445e" containerName="maas-api" Jul 13 07:36:17.857833 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:36:17.857804 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="b73d131a-e4f0-44cd-98cd-c1177b0a445e" containerName="maas-api" Jul 13 07:36:17.857910 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:36:17.857884 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="b73d131a-e4f0-44cd-98cd-c1177b0a445e" containerName="maas-api" Jul 13 07:36:17.861003 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:36:17.860978 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-cc53e7-cd447d5d7-c4xw9" Jul 13 07:36:17.863358 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:36:17.863337 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8rh7l\"" Jul 13 07:36:17.863358 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:36:17.863351 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 13 07:36:17.863868 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:36:17.863853 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 13 07:36:17.863945 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:36:17.863855 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-cc53e7-serving-cert\"" Jul 13 07:36:17.876655 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:36:17.876632 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-cc53e7-cd447d5d7-c4xw9"] Jul 13 07:36:17.962255 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:36:17.962214 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eeebeb52-dd8d-4cc6-8b2c-9bfa355b4458-maas-api-tls\") pod \"maas-api-e2e-shared-a-cc53e7-cd447d5d7-c4xw9\" (UID: \"eeebeb52-dd8d-4cc6-8b2c-9bfa355b4458\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-cc53e7-cd447d5d7-c4xw9" Jul 13 07:36:17.962255 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:36:17.962258 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qbw4\" (UniqueName: \"kubernetes.io/projected/eeebeb52-dd8d-4cc6-8b2c-9bfa355b4458-kube-api-access-2qbw4\") pod \"maas-api-e2e-shared-a-cc53e7-cd447d5d7-c4xw9\" (UID: \"eeebeb52-dd8d-4cc6-8b2c-9bfa355b4458\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-cc53e7-cd447d5d7-c4xw9" Jul 13 07:36:17.962469 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:36:17.962308 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eeebeb52-dd8d-4cc6-8b2c-9bfa355b4458-openshift-service-ca\") pod \"maas-api-e2e-shared-a-cc53e7-cd447d5d7-c4xw9\" (UID: \"eeebeb52-dd8d-4cc6-8b2c-9bfa355b4458\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-cc53e7-cd447d5d7-c4xw9" Jul 13 07:36:18.063150 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:36:18.063117 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eeebeb52-dd8d-4cc6-8b2c-9bfa355b4458-maas-api-tls\") pod \"maas-api-e2e-shared-a-cc53e7-cd447d5d7-c4xw9\" (UID: \"eeebeb52-dd8d-4cc6-8b2c-9bfa355b4458\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-cc53e7-cd447d5d7-c4xw9" Jul 13 07:36:18.063150 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:36:18.063157 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qbw4\" (UniqueName: \"kubernetes.io/projected/eeebeb52-dd8d-4cc6-8b2c-9bfa355b4458-kube-api-access-2qbw4\") pod \"maas-api-e2e-shared-a-cc53e7-cd447d5d7-c4xw9\" (UID: \"eeebeb52-dd8d-4cc6-8b2c-9bfa355b4458\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-cc53e7-cd447d5d7-c4xw9" Jul 13 07:36:18.063371 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:36:18.063216 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eeebeb52-dd8d-4cc6-8b2c-9bfa355b4458-openshift-service-ca\") pod \"maas-api-e2e-shared-a-cc53e7-cd447d5d7-c4xw9\" (UID: \"eeebeb52-dd8d-4cc6-8b2c-9bfa355b4458\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-cc53e7-cd447d5d7-c4xw9" Jul 13 07:36:18.063967 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:36:18.063949 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eeebeb52-dd8d-4cc6-8b2c-9bfa355b4458-openshift-service-ca\") pod \"maas-api-e2e-shared-a-cc53e7-cd447d5d7-c4xw9\" (UID: \"eeebeb52-dd8d-4cc6-8b2c-9bfa355b4458\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-cc53e7-cd447d5d7-c4xw9" Jul 13 07:36:18.065692 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:36:18.065665 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eeebeb52-dd8d-4cc6-8b2c-9bfa355b4458-maas-api-tls\") pod \"maas-api-e2e-shared-a-cc53e7-cd447d5d7-c4xw9\" (UID: \"eeebeb52-dd8d-4cc6-8b2c-9bfa355b4458\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-cc53e7-cd447d5d7-c4xw9" Jul 13 07:36:18.075980 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:36:18.075956 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qbw4\" (UniqueName: \"kubernetes.io/projected/eeebeb52-dd8d-4cc6-8b2c-9bfa355b4458-kube-api-access-2qbw4\") pod \"maas-api-e2e-shared-a-cc53e7-cd447d5d7-c4xw9\" (UID: \"eeebeb52-dd8d-4cc6-8b2c-9bfa355b4458\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-cc53e7-cd447d5d7-c4xw9" Jul 13 07:36:18.171850 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:36:18.171767 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-cc53e7-cd447d5d7-c4xw9" Jul 13 07:36:18.315700 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:36:18.315661 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-cc53e7-cd447d5d7-c4xw9"] Jul 13 07:36:18.317330 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:36:18.317303 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeeebeb52_dd8d_4cc6_8b2c_9bfa355b4458.slice/crio-cb02e96be808abfb11410e9a7d939ebf77bb3d58ed5978c71ac92febb9b1d5c7 WatchSource:0}: Error finding container cb02e96be808abfb11410e9a7d939ebf77bb3d58ed5978c71ac92febb9b1d5c7: Status 404 returned error can't find the container with id cb02e96be808abfb11410e9a7d939ebf77bb3d58ed5978c71ac92febb9b1d5c7 Jul 13 07:36:19.195755 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:36:19.195713 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-cc53e7-cd447d5d7-c4xw9" event={"ID":"eeebeb52-dd8d-4cc6-8b2c-9bfa355b4458","Type":"ContainerStarted","Data":"e0ae14fc5fe0f14686e8326e2bdbe59b63357154daf4a1580834a7e653a57a12"} Jul 13 07:36:19.196145 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:36:19.195780 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-cc53e7-cd447d5d7-c4xw9" event={"ID":"eeebeb52-dd8d-4cc6-8b2c-9bfa355b4458","Type":"ContainerStarted","Data":"cb02e96be808abfb11410e9a7d939ebf77bb3d58ed5978c71ac92febb9b1d5c7"} Jul 13 07:36:19.196145 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:36:19.195826 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-cc53e7-cd447d5d7-c4xw9" Jul 13 07:36:19.213640 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:36:19.213556 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-cc53e7-cd447d5d7-c4xw9" podStartSLOduration=1.816418531 podStartE2EDuration="2.213538834s" totalImagesPullingTime="397.120303ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:36:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:36:19.211846096 +0000 UTC m=+2054.047409927" watchObservedRunningTime="2026-07-13 07:36:19.213538834 +0000 UTC m=+2054.049102668" Jul 13 07:36:22.741839 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:36:22.741798 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:36:25.207436 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:36:25.207408 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-cc53e7-cd447d5d7-c4xw9" Jul 13 07:36:31.949252 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:36:31.949170 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:37:05.862790 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:37:05.862760 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-r878q_293edc99-2691-4915-ba03-7407d88b89b4/console-operator/1.log" Jul 13 07:37:05.867247 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:37:05.867221 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7b964_3d71897a-4b2e-4e4e-9525-ccd89bd18458/ovn-acl-logging/0.log" Jul 13 07:37:05.875331 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:37:05.875303 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-r878q_293edc99-2691-4915-ba03-7407d88b89b4/console-operator/1.log" Jul 13 07:37:05.878929 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:37:05.878911 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7b964_3d71897a-4b2e-4e4e-9525-ccd89bd18458/ovn-acl-logging/0.log" Jul 13 07:39:35.866939 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:39:35.866851 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-cc53e7-cd447d5d7-c4xw9"] Jul 13 07:39:35.867530 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:39:35.867121 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-cc53e7-cd447d5d7-c4xw9" podUID="eeebeb52-dd8d-4cc6-8b2c-9bfa355b4458" containerName="maas-api" containerID="cri-o://e0ae14fc5fe0f14686e8326e2bdbe59b63357154daf4a1580834a7e653a57a12" gracePeriod=30 Jul 13 07:39:36.122453 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:39:36.122381 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-cc53e7-cd447d5d7-c4xw9" Jul 13 07:39:36.191332 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:39:36.191301 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eeebeb52-dd8d-4cc6-8b2c-9bfa355b4458-openshift-service-ca\") pod \"eeebeb52-dd8d-4cc6-8b2c-9bfa355b4458\" (UID: \"eeebeb52-dd8d-4cc6-8b2c-9bfa355b4458\") " Jul 13 07:39:36.191509 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:39:36.191355 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eeebeb52-dd8d-4cc6-8b2c-9bfa355b4458-maas-api-tls\") pod \"eeebeb52-dd8d-4cc6-8b2c-9bfa355b4458\" (UID: \"eeebeb52-dd8d-4cc6-8b2c-9bfa355b4458\") " Jul 13 07:39:36.191509 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:39:36.191374 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2qbw4\" (UniqueName: \"kubernetes.io/projected/eeebeb52-dd8d-4cc6-8b2c-9bfa355b4458-kube-api-access-2qbw4\") pod \"eeebeb52-dd8d-4cc6-8b2c-9bfa355b4458\" (UID: \"eeebeb52-dd8d-4cc6-8b2c-9bfa355b4458\") " Jul 13 07:39:36.191668 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:39:36.191639 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eeebeb52-dd8d-4cc6-8b2c-9bfa355b4458-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "eeebeb52-dd8d-4cc6-8b2c-9bfa355b4458" (UID: "eeebeb52-dd8d-4cc6-8b2c-9bfa355b4458"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 07:39:36.193487 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:39:36.193458 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eeebeb52-dd8d-4cc6-8b2c-9bfa355b4458-kube-api-access-2qbw4" (OuterVolumeSpecName: "kube-api-access-2qbw4") pod "eeebeb52-dd8d-4cc6-8b2c-9bfa355b4458" (UID: "eeebeb52-dd8d-4cc6-8b2c-9bfa355b4458"). InnerVolumeSpecName "kube-api-access-2qbw4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 07:39:36.193487 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:39:36.193480 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eeebeb52-dd8d-4cc6-8b2c-9bfa355b4458-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "eeebeb52-dd8d-4cc6-8b2c-9bfa355b4458" (UID: "eeebeb52-dd8d-4cc6-8b2c-9bfa355b4458"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 07:39:36.263870 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:39:36.263839 2586 generic.go:356] "Generic (PLEG): container finished" podID="eeebeb52-dd8d-4cc6-8b2c-9bfa355b4458" containerID="e0ae14fc5fe0f14686e8326e2bdbe59b63357154daf4a1580834a7e653a57a12" exitCode=0 Jul 13 07:39:36.264017 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:39:36.263882 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-cc53e7-cd447d5d7-c4xw9" Jul 13 07:39:36.264017 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:39:36.263918 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-cc53e7-cd447d5d7-c4xw9" event={"ID":"eeebeb52-dd8d-4cc6-8b2c-9bfa355b4458","Type":"ContainerDied","Data":"e0ae14fc5fe0f14686e8326e2bdbe59b63357154daf4a1580834a7e653a57a12"} Jul 13 07:39:36.264017 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:39:36.263970 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-cc53e7-cd447d5d7-c4xw9" event={"ID":"eeebeb52-dd8d-4cc6-8b2c-9bfa355b4458","Type":"ContainerDied","Data":"cb02e96be808abfb11410e9a7d939ebf77bb3d58ed5978c71ac92febb9b1d5c7"} Jul 13 07:39:36.264017 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:39:36.263993 2586 scope.go:117] "RemoveContainer" containerID="e0ae14fc5fe0f14686e8326e2bdbe59b63357154daf4a1580834a7e653a57a12" Jul 13 07:39:36.274119 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:39:36.274097 2586 scope.go:117] "RemoveContainer" containerID="e0ae14fc5fe0f14686e8326e2bdbe59b63357154daf4a1580834a7e653a57a12" Jul 13 07:39:36.274396 ip-10-0-141-37 kubenswrapper[2586]: E0713 07:39:36.274377 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0ae14fc5fe0f14686e8326e2bdbe59b63357154daf4a1580834a7e653a57a12\": container with ID starting with e0ae14fc5fe0f14686e8326e2bdbe59b63357154daf4a1580834a7e653a57a12 not found: ID does not exist" containerID="e0ae14fc5fe0f14686e8326e2bdbe59b63357154daf4a1580834a7e653a57a12" Jul 13 07:39:36.274473 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:39:36.274406 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0ae14fc5fe0f14686e8326e2bdbe59b63357154daf4a1580834a7e653a57a12"} err="failed to get container status \"e0ae14fc5fe0f14686e8326e2bdbe59b63357154daf4a1580834a7e653a57a12\": rpc error: code = NotFound desc = could not find container \"e0ae14fc5fe0f14686e8326e2bdbe59b63357154daf4a1580834a7e653a57a12\": container with ID starting with e0ae14fc5fe0f14686e8326e2bdbe59b63357154daf4a1580834a7e653a57a12 not found: ID does not exist" Jul 13 07:39:36.287712 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:39:36.287656 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-cc53e7-cd447d5d7-c4xw9"] Jul 13 07:39:36.289986 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:39:36.289965 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-cc53e7-cd447d5d7-c4xw9"] Jul 13 07:39:36.292046 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:39:36.292030 2586 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eeebeb52-dd8d-4cc6-8b2c-9bfa355b4458-maas-api-tls\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:39:36.292099 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:39:36.292050 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2qbw4\" (UniqueName: \"kubernetes.io/projected/eeebeb52-dd8d-4cc6-8b2c-9bfa355b4458-kube-api-access-2qbw4\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:39:36.292099 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:39:36.292059 2586 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eeebeb52-dd8d-4cc6-8b2c-9bfa355b4458-openshift-service-ca\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\"" Jul 13 07:39:37.805662 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:39:37.805626 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eeebeb52-dd8d-4cc6-8b2c-9bfa355b4458" path="/var/lib/kubelet/pods/eeebeb52-dd8d-4cc6-8b2c-9bfa355b4458/volumes" Jul 13 07:39:38.842639 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:39:38.842605 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:39:49.940636 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:39:49.940597 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-64cb9bbcff-xcvzz_e5df704d-8a0d-486f-b395-8ff096d648de/authorino/0.log" Jul 13 07:39:51.446293 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:39:51.446262 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2rksc"] Jul 13 07:39:54.627071 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:39:54.627023 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-4mlft_ce788bde-51f0-466a-93b9-248e4d3b5f5c/manager/0.log" Jul 13 07:39:55.008639 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:39:55.008599 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-v77jc_531be0a6-b340-4af9-8064-6f8066f1c69a/manager/0.log" Jul 13 07:39:55.125288 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:39:55.125260 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7487689bdd-959zg_59d14f61-a765-4c3b-a117-8bc2a9ee43a7/copy-manifests/0.log" Jul 13 07:39:55.132845 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:39:55.132820 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7487689bdd-959zg_59d14f61-a765-4c3b-a117-8bc2a9ee43a7/manager/0.log" Jul 13 07:39:56.167937 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:39:56.167908 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rbp4k_faa75440-9d18-4c09-b910-0f3d3c73c9d1/util/0.log" Jul 13 07:39:56.174891 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:39:56.174870 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rbp4k_faa75440-9d18-4c09-b910-0f3d3c73c9d1/pull/0.log" Jul 13 07:39:56.182705 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:39:56.182658 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rbp4k_faa75440-9d18-4c09-b910-0f3d3c73c9d1/extract/0.log" Jul 13 07:39:56.288795 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:39:56.288771 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p4t6d_adace2a3-d312-41d9-b47c-bb4a63aa0663/util/0.log" Jul 13 07:39:56.294635 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:39:56.294618 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p4t6d_adace2a3-d312-41d9-b47c-bb4a63aa0663/pull/0.log" Jul 13 07:39:56.300425 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:39:56.300404 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p4t6d_adace2a3-d312-41d9-b47c-bb4a63aa0663/extract/0.log" Jul 13 07:39:56.411467 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:39:56.411440 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738xbs6_e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e/util/0.log" Jul 13 07:39:56.417995 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:39:56.417927 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738xbs6_e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e/pull/0.log" Jul 13 07:39:56.424997 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:39:56.424973 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738xbs6_e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e/extract/0.log" Jul 13 07:39:56.537152 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:39:56.537126 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fvhbn_c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b/util/0.log" Jul 13 07:39:56.543815 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:39:56.543777 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fvhbn_c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b/pull/0.log" Jul 13 07:39:56.553220 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:39:56.553204 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fvhbn_c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b/extract/0.log" Jul 13 07:39:56.677949 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:39:56.677866 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-64cb9bbcff-xcvzz_e5df704d-8a0d-486f-b395-8ff096d648de/authorino/0.log" Jul 13 07:39:56.796389 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:39:56.796352 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-g4z7k_63d0ee4f-2e8e-49e0-9fa2-68e29d18b112/manager/0.log" Jul 13 07:39:57.134001 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:39:57.133970 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-bktks_5782c505-1c63-4707-a56e-85ceaf43af36/registry-server/0.log" Jul 13 07:39:57.249501 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:39:57.249466 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-nwwwh_b47e79da-5d27-4566-babe-1e2ce244a33b/manager/0.log" Jul 13 07:39:57.368076 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:39:57.368050 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-2rksc_a405d03d-88ae-493b-b1e0-d7ca25f52246/limitador/0.log" Jul 13 07:39:57.498697 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:39:57.498654 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-7xfkf_ff395397-599a-49c9-9ef2-0273bb1d8c87/manager/0.log" Jul 13 07:39:58.080991 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:39:58.080956 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-9fcd7cd9-5spff_45fe8f55-a887-467e-baf8-61f5b0cc4d78/kube-auth-proxy/0.log" Jul 13 07:39:58.685311 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:39:58.685279 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7f7c7b8964-kxkhw_c28d91ed-a056-47d9-ba7c-f789f42dd6cc/router/0.log" Jul 13 07:39:59.130029 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:39:59.129954 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-hc2sg_72534e7c-99e2-4679-a199-6eb856bcf30f/main/0.log" Jul 13 07:39:59.456723 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:39:59.456677 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-6zwph_81753bcc-9288-4af8-b919-86c11dfa654c/main/0.log" Jul 13 07:39:59.567643 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:39:59.567613 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-k5fkk_d3639c44-bb55-4d7f-a0b3-323e86441bb1/main/0.log" Jul 13 07:40:04.104952 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:04.104918 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-qmnmv/must-gather-z65dm"] Jul 13 07:40:04.105330 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:04.105310 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eeebeb52-dd8d-4cc6-8b2c-9bfa355b4458" containerName="maas-api" Jul 13 07:40:04.105330 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:04.105320 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="eeebeb52-dd8d-4cc6-8b2c-9bfa355b4458" containerName="maas-api" Jul 13 07:40:04.105412 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:04.105380 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="eeebeb52-dd8d-4cc6-8b2c-9bfa355b4458" containerName="maas-api" Jul 13 07:40:04.108831 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:04.108814 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qmnmv/must-gather-z65dm" Jul 13 07:40:04.111238 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:04.111208 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qmnmv\"/\"openshift-service-ca.crt\"" Jul 13 07:40:04.111886 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:04.111869 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-qmnmv\"/\"default-dockercfg-dd7q7\"" Jul 13 07:40:04.111994 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:04.111907 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qmnmv\"/\"kube-root-ca.crt\"" Jul 13 07:40:04.118060 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:04.118036 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qmnmv/must-gather-z65dm"] Jul 13 07:40:04.240959 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:04.240922 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f9bec698-71ba-45bb-917c-eb0e8a32f7fa-must-gather-output\") pod \"must-gather-z65dm\" (UID: \"f9bec698-71ba-45bb-917c-eb0e8a32f7fa\") " pod="openshift-must-gather-qmnmv/must-gather-z65dm" Jul 13 07:40:04.241143 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:04.241059 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slcmg\" (UniqueName: \"kubernetes.io/projected/f9bec698-71ba-45bb-917c-eb0e8a32f7fa-kube-api-access-slcmg\") pod \"must-gather-z65dm\" (UID: \"f9bec698-71ba-45bb-917c-eb0e8a32f7fa\") " pod="openshift-must-gather-qmnmv/must-gather-z65dm" Jul 13 07:40:04.341977 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:04.341940 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f9bec698-71ba-45bb-917c-eb0e8a32f7fa-must-gather-output\") pod \"must-gather-z65dm\" (UID: \"f9bec698-71ba-45bb-917c-eb0e8a32f7fa\") " pod="openshift-must-gather-qmnmv/must-gather-z65dm" Jul 13 07:40:04.342160 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:04.342044 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slcmg\" (UniqueName: \"kubernetes.io/projected/f9bec698-71ba-45bb-917c-eb0e8a32f7fa-kube-api-access-slcmg\") pod \"must-gather-z65dm\" (UID: \"f9bec698-71ba-45bb-917c-eb0e8a32f7fa\") " pod="openshift-must-gather-qmnmv/must-gather-z65dm" Jul 13 07:40:04.342380 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:04.342359 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f9bec698-71ba-45bb-917c-eb0e8a32f7fa-must-gather-output\") pod \"must-gather-z65dm\" (UID: \"f9bec698-71ba-45bb-917c-eb0e8a32f7fa\") " pod="openshift-must-gather-qmnmv/must-gather-z65dm" Jul 13 07:40:04.350318 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:04.350294 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-slcmg\" (UniqueName: \"kubernetes.io/projected/f9bec698-71ba-45bb-917c-eb0e8a32f7fa-kube-api-access-slcmg\") pod \"must-gather-z65dm\" (UID: \"f9bec698-71ba-45bb-917c-eb0e8a32f7fa\") " pod="openshift-must-gather-qmnmv/must-gather-z65dm" Jul 13 07:40:04.419223 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:04.419138 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qmnmv/must-gather-z65dm" Jul 13 07:40:04.558070 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:04.558042 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qmnmv/must-gather-z65dm"] Jul 13 07:40:04.559904 ip-10-0-141-37 kubenswrapper[2586]: W0713 07:40:04.559871 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf9bec698_71ba_45bb_917c_eb0e8a32f7fa.slice/crio-fa95dc8e4973a0e981e9b9b1eed7781ce79e3b2f8db895b47a2bca5fb7868f9f WatchSource:0}: Error finding container fa95dc8e4973a0e981e9b9b1eed7781ce79e3b2f8db895b47a2bca5fb7868f9f: Status 404 returned error can't find the container with id fa95dc8e4973a0e981e9b9b1eed7781ce79e3b2f8db895b47a2bca5fb7868f9f Jul 13 07:40:04.561751 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:04.561732 2586 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 07:40:05.422911 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:05.422883 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qmnmv/must-gather-z65dm" event={"ID":"f9bec698-71ba-45bb-917c-eb0e8a32f7fa","Type":"ContainerStarted","Data":"fa95dc8e4973a0e981e9b9b1eed7781ce79e3b2f8db895b47a2bca5fb7868f9f"} Jul 13 07:40:06.432828 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:06.432759 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qmnmv/must-gather-z65dm" event={"ID":"f9bec698-71ba-45bb-917c-eb0e8a32f7fa","Type":"ContainerStarted","Data":"8b44ec8d3c1579cddec0db36b3ef97acd8f7b1510146bbe8e71c6e6bca7edd28"} Jul 13 07:40:06.432828 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:06.432808 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qmnmv/must-gather-z65dm" event={"ID":"f9bec698-71ba-45bb-917c-eb0e8a32f7fa","Type":"ContainerStarted","Data":"815d307479ecd82b6b83bc198b2b98e266ac6dbdf94338d15cbc08d9e739c018"} Jul 13 07:40:06.449840 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:06.449777 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-qmnmv/must-gather-z65dm" podStartSLOduration=1.663915078 podStartE2EDuration="2.44975496s" totalImagesPullingTime="785.839882ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 07:40:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:40:06.44777923 +0000 UTC m=+2281.283343062" watchObservedRunningTime="2026-07-13 07:40:06.44975496 +0000 UTC m=+2281.285318791" Jul 13 07:40:07.000583 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:07.000555 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-lb8th_c3724c5e-1a2b-410b-a745-b0f90bf27358/global-pull-secret-syncer/0.log" Jul 13 07:40:07.133159 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:07.133126 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-xs7q5_2abf1e96-1ba2-4cc1-a632-48e28b374830/konnectivity-agent/0.log" Jul 13 07:40:07.214703 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:07.214649 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-37.ec2.internal_6df327e3e80f10552c93b38d2fd449c4/haproxy/0.log" Jul 13 07:40:12.250740 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:12.250708 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rbp4k_faa75440-9d18-4c09-b910-0f3d3c73c9d1/extract/0.log" Jul 13 07:40:12.278604 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:12.278558 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rbp4k_faa75440-9d18-4c09-b910-0f3d3c73c9d1/util/0.log" Jul 13 07:40:12.299645 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:12.299586 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rbp4k_faa75440-9d18-4c09-b910-0f3d3c73c9d1/pull/0.log" Jul 13 07:40:12.333625 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:12.333579 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p4t6d_adace2a3-d312-41d9-b47c-bb4a63aa0663/extract/0.log" Jul 13 07:40:12.358960 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:12.358935 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p4t6d_adace2a3-d312-41d9-b47c-bb4a63aa0663/util/0.log" Jul 13 07:40:12.383900 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:12.383874 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p4t6d_adace2a3-d312-41d9-b47c-bb4a63aa0663/pull/0.log" Jul 13 07:40:12.412307 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:12.412280 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738xbs6_e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e/extract/0.log" Jul 13 07:40:12.436118 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:12.436047 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738xbs6_e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e/util/0.log" Jul 13 07:40:12.460623 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:12.460565 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738xbs6_e62a8b69-94ba-4bb5-a9da-6fa0c7f2173e/pull/0.log" Jul 13 07:40:12.496034 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:12.496007 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fvhbn_c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b/extract/0.log" Jul 13 07:40:12.516289 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:12.516214 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fvhbn_c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b/util/0.log" Jul 13 07:40:12.541586 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:12.541559 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fvhbn_c2b47d1b-8bc4-4d22-ab1c-bfcf6effdb4b/pull/0.log" Jul 13 07:40:12.919749 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:12.919080 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-64cb9bbcff-xcvzz_e5df704d-8a0d-486f-b395-8ff096d648de/authorino/0.log" Jul 13 07:40:12.947707 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:12.947065 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-g4z7k_63d0ee4f-2e8e-49e0-9fa2-68e29d18b112/manager/0.log" Jul 13 07:40:13.047693 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:13.047276 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-bktks_5782c505-1c63-4707-a56e-85ceaf43af36/registry-server/0.log" Jul 13 07:40:13.201896 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:13.201859 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-nwwwh_b47e79da-5d27-4566-babe-1e2ce244a33b/manager/0.log" Jul 13 07:40:13.228958 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:13.228929 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-2rksc_a405d03d-88ae-493b-b1e0-d7ca25f52246/limitador/0.log" Jul 13 07:40:13.311703 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:13.310735 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-7xfkf_ff395397-599a-49c9-9ef2-0273bb1d8c87/manager/0.log" Jul 13 07:40:15.227952 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:15.227919 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-74xdm_5b0bbb6f-a4f2-428a-8c0b-8561d031061c/cluster-monitoring-operator/0.log" Jul 13 07:40:15.488366 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:15.488283 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5qhl8_c1c21537-9e03-4149-b45f-00aa313d26b0/node-exporter/0.log" Jul 13 07:40:15.522814 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:15.522769 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5qhl8_c1c21537-9e03-4149-b45f-00aa313d26b0/kube-rbac-proxy/0.log" Jul 13 07:40:15.562760 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:15.562730 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5qhl8_c1c21537-9e03-4149-b45f-00aa313d26b0/init-textfile/0.log" Jul 13 07:40:16.164379 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:16.164347 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-ph8pt_a558e17f-d128-45d0-ac7d-835bf2094fdf/prometheus-operator-admission-webhook/0.log" Jul 13 07:40:16.832003 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:16.831967 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-qmnmv/perf-node-gather-daemonset-vqx62"] Jul 13 07:40:16.836470 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:16.836440 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qmnmv/perf-node-gather-daemonset-vqx62" Jul 13 07:40:16.847415 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:16.847375 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qmnmv/perf-node-gather-daemonset-vqx62"] Jul 13 07:40:16.885421 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:16.885371 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6ad6309d-4fc0-4db9-be96-1f2a9d790232-lib-modules\") pod \"perf-node-gather-daemonset-vqx62\" (UID: \"6ad6309d-4fc0-4db9-be96-1f2a9d790232\") " pod="openshift-must-gather-qmnmv/perf-node-gather-daemonset-vqx62" Jul 13 07:40:16.885711 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:16.885669 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6ad6309d-4fc0-4db9-be96-1f2a9d790232-podres\") pod \"perf-node-gather-daemonset-vqx62\" (UID: \"6ad6309d-4fc0-4db9-be96-1f2a9d790232\") " pod="openshift-must-gather-qmnmv/perf-node-gather-daemonset-vqx62" Jul 13 07:40:16.885875 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:16.885857 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6ad6309d-4fc0-4db9-be96-1f2a9d790232-sys\") pod \"perf-node-gather-daemonset-vqx62\" (UID: \"6ad6309d-4fc0-4db9-be96-1f2a9d790232\") " pod="openshift-must-gather-qmnmv/perf-node-gather-daemonset-vqx62" Jul 13 07:40:16.885966 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:16.885928 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6ad6309d-4fc0-4db9-be96-1f2a9d790232-proc\") pod \"perf-node-gather-daemonset-vqx62\" (UID: \"6ad6309d-4fc0-4db9-be96-1f2a9d790232\") " pod="openshift-must-gather-qmnmv/perf-node-gather-daemonset-vqx62" Jul 13 07:40:16.886022 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:16.885972 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6w5l\" (UniqueName: \"kubernetes.io/projected/6ad6309d-4fc0-4db9-be96-1f2a9d790232-kube-api-access-d6w5l\") pod \"perf-node-gather-daemonset-vqx62\" (UID: \"6ad6309d-4fc0-4db9-be96-1f2a9d790232\") " pod="openshift-must-gather-qmnmv/perf-node-gather-daemonset-vqx62" Jul 13 07:40:16.987518 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:16.987480 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6ad6309d-4fc0-4db9-be96-1f2a9d790232-lib-modules\") pod \"perf-node-gather-daemonset-vqx62\" (UID: \"6ad6309d-4fc0-4db9-be96-1f2a9d790232\") " pod="openshift-must-gather-qmnmv/perf-node-gather-daemonset-vqx62" Jul 13 07:40:16.987831 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:16.987795 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6ad6309d-4fc0-4db9-be96-1f2a9d790232-podres\") pod \"perf-node-gather-daemonset-vqx62\" (UID: \"6ad6309d-4fc0-4db9-be96-1f2a9d790232\") " pod="openshift-must-gather-qmnmv/perf-node-gather-daemonset-vqx62" Jul 13 07:40:16.987969 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:16.987942 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6ad6309d-4fc0-4db9-be96-1f2a9d790232-podres\") pod \"perf-node-gather-daemonset-vqx62\" (UID: \"6ad6309d-4fc0-4db9-be96-1f2a9d790232\") " pod="openshift-must-gather-qmnmv/perf-node-gather-daemonset-vqx62" Jul 13 07:40:16.988134 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:16.987988 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6ad6309d-4fc0-4db9-be96-1f2a9d790232-sys\") pod \"perf-node-gather-daemonset-vqx62\" (UID: \"6ad6309d-4fc0-4db9-be96-1f2a9d790232\") " pod="openshift-must-gather-qmnmv/perf-node-gather-daemonset-vqx62" Jul 13 07:40:16.988279 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:16.987692 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6ad6309d-4fc0-4db9-be96-1f2a9d790232-lib-modules\") pod \"perf-node-gather-daemonset-vqx62\" (UID: \"6ad6309d-4fc0-4db9-be96-1f2a9d790232\") " pod="openshift-must-gather-qmnmv/perf-node-gather-daemonset-vqx62" Jul 13 07:40:16.988425 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:16.988069 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6ad6309d-4fc0-4db9-be96-1f2a9d790232-sys\") pod \"perf-node-gather-daemonset-vqx62\" (UID: \"6ad6309d-4fc0-4db9-be96-1f2a9d790232\") " pod="openshift-must-gather-qmnmv/perf-node-gather-daemonset-vqx62" Jul 13 07:40:16.988558 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:16.988271 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6ad6309d-4fc0-4db9-be96-1f2a9d790232-proc\") pod \"perf-node-gather-daemonset-vqx62\" (UID: \"6ad6309d-4fc0-4db9-be96-1f2a9d790232\") " pod="openshift-must-gather-qmnmv/perf-node-gather-daemonset-vqx62" Jul 13 07:40:16.988741 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:16.988725 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6w5l\" (UniqueName: \"kubernetes.io/projected/6ad6309d-4fc0-4db9-be96-1f2a9d790232-kube-api-access-d6w5l\") pod \"perf-node-gather-daemonset-vqx62\" (UID: \"6ad6309d-4fc0-4db9-be96-1f2a9d790232\") " pod="openshift-must-gather-qmnmv/perf-node-gather-daemonset-vqx62" Jul 13 07:40:16.989090 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:16.988332 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6ad6309d-4fc0-4db9-be96-1f2a9d790232-proc\") pod \"perf-node-gather-daemonset-vqx62\" (UID: \"6ad6309d-4fc0-4db9-be96-1f2a9d790232\") " pod="openshift-must-gather-qmnmv/perf-node-gather-daemonset-vqx62" Jul 13 07:40:16.998245 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:16.998219 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6w5l\" (UniqueName: \"kubernetes.io/projected/6ad6309d-4fc0-4db9-be96-1f2a9d790232-kube-api-access-d6w5l\") pod \"perf-node-gather-daemonset-vqx62\" (UID: \"6ad6309d-4fc0-4db9-be96-1f2a9d790232\") " pod="openshift-must-gather-qmnmv/perf-node-gather-daemonset-vqx62" Jul 13 07:40:17.151149 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:17.151054 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qmnmv/perf-node-gather-daemonset-vqx62" Jul 13 07:40:17.321974 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:17.321947 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qmnmv/perf-node-gather-daemonset-vqx62"] Jul 13 07:40:17.526444 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:17.526405 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qmnmv/perf-node-gather-daemonset-vqx62" event={"ID":"6ad6309d-4fc0-4db9-be96-1f2a9d790232","Type":"ContainerStarted","Data":"0ba282bac0dfb674e36ba2f11ee723bb3645777bb23862649a1fe8a0479d9fde"} Jul 13 07:40:17.526656 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:17.526473 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qmnmv/perf-node-gather-daemonset-vqx62" event={"ID":"6ad6309d-4fc0-4db9-be96-1f2a9d790232","Type":"ContainerStarted","Data":"61ebd756e101f1f5b3cf5fe770ef73179f3d3aef6f66fbe2539de5463fab7ee0"} Jul 13 07:40:17.526656 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:17.526517 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-qmnmv/perf-node-gather-daemonset-vqx62" Jul 13 07:40:17.545057 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:17.544998 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-qmnmv/perf-node-gather-daemonset-vqx62" podStartSLOduration=1.544979259 podStartE2EDuration="1.544979259s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 07:40:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 07:40:17.543070677 +0000 UTC m=+2292.378634510" watchObservedRunningTime="2026-07-13 07:40:17.544979259 +0000 UTC m=+2292.380543091" Jul 13 07:40:17.589380 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:17.589350 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-6fw29_ec04b338-54ca-471f-b138-eabbf7009550/networking-console-plugin/0.log" Jul 13 07:40:18.115536 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:18.115502 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-r878q_293edc99-2691-4915-ba03-7407d88b89b4/console-operator/1.log" Jul 13 07:40:18.123082 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:18.123057 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-r878q_293edc99-2691-4915-ba03-7407d88b89b4/console-operator/2.log" Jul 13 07:40:18.587656 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:18.587630 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-8d967f6bf-47qmq_4f410a58-dbcb-4901-943c-eb158ae798a9/console/0.log" Jul 13 07:40:18.628655 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:18.628617 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-dc982_4c202845-1e5e-4d2d-9cd3-5ec97926bc84/download-server/0.log" Jul 13 07:40:19.337391 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:19.337364 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-drnth_c4a2167f-7cee-4066-914f-d9c9e4d3bff7/volume-data-source-validator/0.log" Jul 13 07:40:20.446241 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:20.446214 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wg6mf_a7c70271-ef81-4393-9206-d7957e5c86e1/dns/0.log" Jul 13 07:40:20.472615 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:20.472586 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wg6mf_a7c70271-ef81-4393-9206-d7957e5c86e1/kube-rbac-proxy/0.log" Jul 13 07:40:20.609961 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:20.609929 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-g6kgx_75604003-f810-475e-b0db-af8180c49604/dns-node-resolver/0.log" Jul 13 07:40:21.258676 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:21.258648 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vxcrv_be89589f-0a40-4e74-9ecf-aeac7e231126/node-ca/0.log" Jul 13 07:40:22.340701 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:22.340656 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-9fcd7cd9-5spff_45fe8f55-a887-467e-baf8-61f5b0cc4d78/kube-auth-proxy/0.log" Jul 13 07:40:22.609615 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:22.609528 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7f7c7b8964-kxkhw_c28d91ed-a056-47d9-ba7c-f789f42dd6cc/router/0.log" Jul 13 07:40:23.365800 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:23.365771 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-ps2kk_30689818-dbf9-4665-86dc-b81d96b6bed1/serve-healthcheck-canary/0.log" Jul 13 07:40:23.546273 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:23.546242 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-qmnmv/perf-node-gather-daemonset-vqx62" Jul 13 07:40:24.024186 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:24.024141 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-z7x8m_05bf1881-84d9-42e6-a7ed-1a348839015d/insights-operator/0.log" Jul 13 07:40:24.025135 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:24.025111 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-z7x8m_05bf1881-84d9-42e6-a7ed-1a348839015d/insights-operator/1.log" Jul 13 07:40:24.125080 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:24.125049 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-t5kqb_16425325-d8c8-40cb-a1c0-bfbc147f1416/kube-rbac-proxy/0.log" Jul 13 07:40:24.150002 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:24.149966 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-t5kqb_16425325-d8c8-40cb-a1c0-bfbc147f1416/exporter/0.log" Jul 13 07:40:24.174965 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:24.174931 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-t5kqb_16425325-d8c8-40cb-a1c0-bfbc147f1416/extractor/0.log" Jul 13 07:40:26.837987 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:26.837909 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-4mlft_ce788bde-51f0-466a-93b9-248e4d3b5f5c/manager/0.log" Jul 13 07:40:27.075508 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:27.075477 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-v77jc_531be0a6-b340-4af9-8064-6f8066f1c69a/manager/0.log" Jul 13 07:40:27.101621 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:27.101539 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7487689bdd-959zg_59d14f61-a765-4c3b-a117-8bc2a9ee43a7/manager/0.log" Jul 13 07:40:27.121595 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:27.121570 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7487689bdd-959zg_59d14f61-a765-4c3b-a117-8bc2a9ee43a7/copy-manifests/0.log" Jul 13 07:40:28.004272 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:28.004242 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-9fcd7cd9-5spff_45fe8f55-a887-467e-baf8-61f5b0cc4d78/kube-auth-proxy/0.log" Jul 13 07:40:28.234377 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:28.234339 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7f7c7b8964-kxkhw_c28d91ed-a056-47d9-ba7c-f789f42dd6cc/router/0.log" Jul 13 07:40:29.448201 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:29.448165 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-9fcd7cd9-5spff_45fe8f55-a887-467e-baf8-61f5b0cc4d78/kube-auth-proxy/0.log" Jul 13 07:40:29.663178 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:29.663143 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7f7c7b8964-kxkhw_c28d91ed-a056-47d9-ba7c-f789f42dd6cc/router/0.log" Jul 13 07:40:33.855137 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:33.855100 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-m7g8b_396e6c4f-4cbc-40ed-b6e8-babbbbc48d39/kube-storage-version-migrator-operator/1.log" Jul 13 07:40:33.856127 ip-10-0-141-37 kubenswrapper[2586]: I0713 07:40:33.856107 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-m7g8b_396e6c4f-4cbc-40ed-b6e8-babbbbc48d39/kube-storage-version-migrator-operator/0.log"