Jun 15 01:42:10.456247 ip-10-0-143-196 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jun 15 01:42:10.456263 ip-10-0-143-196 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jun 15 01:42:10.456273 ip-10-0-143-196 systemd[1]: kubelet.service: Failed with result 'resources'. Jun 15 01:42:10.456613 ip-10-0-143-196 systemd[1]: Failed to start Kubernetes Kubelet. Jun 15 01:42:20.568463 ip-10-0-143-196 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jun 15 01:42:20.568484 ip-10-0-143-196 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot c1889e0eb5ec4ea5b1cb5bf6b8d68af3 -- Jun 15 01:44:36.172485 ip-10-0-143-196 systemd[1]: Starting Kubernetes Kubelet... Jun 15 01:44:36.592141 ip-10-0-143-196 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 15 01:44:36.592141 ip-10-0-143-196 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 15 01:44:36.592141 ip-10-0-143-196 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 15 01:44:36.592141 ip-10-0-143-196 kubenswrapper[2578]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 15 01:44:36.592141 ip-10-0-143-196 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 15 01:44:36.594931 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.594837 2578 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 15 01:44:36.602636 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602616 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 15 01:44:36.602636 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602638 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 15 01:44:36.602710 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602642 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 15 01:44:36.602710 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602645 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 15 01:44:36.602710 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602649 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 15 01:44:36.602710 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602652 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 15 01:44:36.602710 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602656 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 15 01:44:36.602710 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602658 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 15 01:44:36.602710 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602661 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 15 01:44:36.602710 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602665 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 15 01:44:36.602710 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602667 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 15 01:44:36.602710 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602670 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 15 01:44:36.602710 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602673 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 15 01:44:36.602710 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602675 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 15 01:44:36.602710 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602679 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 15 01:44:36.602710 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602683 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 15 01:44:36.602710 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602686 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 15 01:44:36.602710 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602689 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 15 01:44:36.602710 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602691 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 15 01:44:36.602710 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602694 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 15 01:44:36.602710 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602697 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 15 01:44:36.602710 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602704 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 15 01:44:36.603200 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602707 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 15 01:44:36.603200 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602709 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jun 15 01:44:36.603200 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602712 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 15 01:44:36.603200 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602715 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 15 01:44:36.603200 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602717 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 15 01:44:36.603200 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602720 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 15 01:44:36.603200 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602723 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 15 01:44:36.603200 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602726 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 15 01:44:36.603200 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602730 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 15 01:44:36.603200 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602733 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 15 01:44:36.603200 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602735 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 15 01:44:36.603200 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602738 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 15 01:44:36.603200 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602740 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 15 01:44:36.603200 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602743 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 15 01:44:36.603200 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602746 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 15 01:44:36.603200 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602748 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 15 01:44:36.603200 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602751 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 15 01:44:36.603200 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602753 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 15 01:44:36.603200 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602756 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 15 01:44:36.603200 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602759 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 15 01:44:36.603682 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602762 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 15 01:44:36.603682 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602765 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 15 01:44:36.603682 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602767 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 15 01:44:36.603682 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602770 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 15 01:44:36.603682 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602772 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 15 01:44:36.603682 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602774 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 15 01:44:36.603682 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602777 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 15 01:44:36.603682 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602779 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 15 01:44:36.603682 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602781 2578 feature_gate.go:328] unrecognized feature gate: Example Jun 15 01:44:36.603682 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602784 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 15 01:44:36.603682 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602787 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 15 01:44:36.603682 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602789 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 15 01:44:36.603682 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602793 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 15 01:44:36.603682 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602797 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 15 01:44:36.603682 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602802 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 15 01:44:36.603682 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602805 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 15 01:44:36.603682 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602808 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 15 01:44:36.603682 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602810 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 15 01:44:36.603682 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602814 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 15 01:44:36.604168 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602818 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 15 01:44:36.604168 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602821 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 15 01:44:36.604168 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602824 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 15 01:44:36.604168 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602826 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 15 01:44:36.604168 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602829 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 15 01:44:36.604168 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602832 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 15 01:44:36.604168 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602835 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 15 01:44:36.604168 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602838 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 15 01:44:36.604168 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602841 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 15 01:44:36.604168 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602843 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 15 01:44:36.604168 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602846 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 15 01:44:36.604168 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602848 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 15 01:44:36.604168 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602851 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 15 01:44:36.604168 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602853 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 15 01:44:36.604168 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602884 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 15 01:44:36.604168 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602887 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 15 01:44:36.604168 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602891 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 15 01:44:36.604168 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602894 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 15 01:44:36.604168 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602897 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 15 01:44:36.604168 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602899 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 15 01:44:36.604641 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602902 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 15 01:44:36.604641 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602904 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 15 01:44:36.604641 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602907 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 15 01:44:36.604641 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602910 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 15 01:44:36.604641 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602912 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 15 01:44:36.604641 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602915 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 15 01:44:36.604641 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602918 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 15 01:44:36.604641 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602922 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 15 01:44:36.604641 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602924 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 15 01:44:36.604641 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602927 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 15 01:44:36.604641 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602929 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 15 01:44:36.604641 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602932 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 15 01:44:36.604641 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602934 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 15 01:44:36.604641 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602937 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 15 01:44:36.604641 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602939 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 15 01:44:36.604641 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602942 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 15 01:44:36.604641 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602945 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 15 01:44:36.604641 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.602947 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 15 01:44:36.604641 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603722 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 15 01:44:36.604641 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603729 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 15 01:44:36.605141 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603733 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 15 01:44:36.605141 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603736 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 15 01:44:36.605141 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603739 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 15 01:44:36.605141 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603741 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 15 01:44:36.605141 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603744 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 15 01:44:36.605141 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603747 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 15 01:44:36.605141 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603750 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 15 01:44:36.605141 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603752 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 15 01:44:36.605141 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603755 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 15 01:44:36.605141 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603757 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 15 01:44:36.605141 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603760 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 15 01:44:36.605141 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603762 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 15 01:44:36.605141 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603765 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 15 01:44:36.605141 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603767 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 15 01:44:36.605141 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603772 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 15 01:44:36.605141 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603776 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 15 01:44:36.605141 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603779 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 15 01:44:36.605141 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603785 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 15 01:44:36.605141 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603788 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 15 01:44:36.605638 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603792 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 15 01:44:36.605638 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603795 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 15 01:44:36.605638 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603798 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 15 01:44:36.605638 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603801 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 15 01:44:36.605638 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603803 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 15 01:44:36.605638 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603806 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 15 01:44:36.605638 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603809 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 15 01:44:36.605638 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603812 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 15 01:44:36.605638 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603814 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 15 01:44:36.605638 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603817 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 15 01:44:36.605638 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603819 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 15 01:44:36.605638 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603822 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 15 01:44:36.605638 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603824 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 15 01:44:36.605638 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603827 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jun 15 01:44:36.605638 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603829 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 15 01:44:36.605638 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603832 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 15 01:44:36.605638 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603834 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 15 01:44:36.605638 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603837 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 15 01:44:36.605638 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603840 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 15 01:44:36.605638 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603842 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 15 01:44:36.606142 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603845 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 15 01:44:36.606142 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603847 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 15 01:44:36.606142 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603849 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 15 01:44:36.606142 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603852 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 15 01:44:36.606142 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603854 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 15 01:44:36.606142 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603857 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 15 01:44:36.606142 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603859 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 15 01:44:36.606142 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603862 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 15 01:44:36.606142 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603865 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 15 01:44:36.606142 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603867 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 15 01:44:36.606142 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603870 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 15 01:44:36.606142 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603873 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 15 01:44:36.606142 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603875 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 15 01:44:36.606142 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603878 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 15 01:44:36.606142 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603881 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 15 01:44:36.606142 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603883 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 15 01:44:36.606142 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603886 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 15 01:44:36.606142 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603889 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 15 01:44:36.606142 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603891 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 15 01:44:36.606142 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603894 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 15 01:44:36.606638 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603897 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 15 01:44:36.606638 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603899 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 15 01:44:36.606638 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603901 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 15 01:44:36.606638 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603904 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 15 01:44:36.606638 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603906 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 15 01:44:36.606638 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603909 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 15 01:44:36.606638 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603911 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 15 01:44:36.606638 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603913 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 15 01:44:36.606638 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603916 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 15 01:44:36.606638 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603918 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 15 01:44:36.606638 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603921 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 15 01:44:36.606638 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603923 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 15 01:44:36.606638 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603926 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 15 01:44:36.606638 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603928 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 15 01:44:36.606638 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603931 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 15 01:44:36.606638 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603933 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 15 01:44:36.606638 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603936 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 15 01:44:36.606638 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603938 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 15 01:44:36.606638 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603940 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 15 01:44:36.606638 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603943 2578 feature_gate.go:328] unrecognized feature gate: Example Jun 15 01:44:36.607235 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603945 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 15 01:44:36.607235 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603948 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 15 01:44:36.607235 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603951 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 15 01:44:36.607235 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603953 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 15 01:44:36.607235 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603956 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 15 01:44:36.607235 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603960 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 15 01:44:36.607235 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603963 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 15 01:44:36.607235 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603966 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 15 01:44:36.607235 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603969 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 15 01:44:36.607235 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603972 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 15 01:44:36.607235 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603974 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 15 01:44:36.607235 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603977 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 15 01:44:36.607235 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603980 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 15 01:44:36.607235 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603982 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 15 01:44:36.607235 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603985 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 15 01:44:36.607235 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603987 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 15 01:44:36.607235 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603990 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 15 01:44:36.607235 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.603992 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 15 01:44:36.607235 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604397 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 15 01:44:36.607694 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604402 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 15 01:44:36.607694 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604405 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 15 01:44:36.607694 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604408 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 15 01:44:36.607694 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604410 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 15 01:44:36.607694 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604415 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 15 01:44:36.607694 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604418 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 15 01:44:36.607694 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604421 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 15 01:44:36.607694 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604424 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 15 01:44:36.607694 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604427 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 15 01:44:36.607694 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604430 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 15 01:44:36.607694 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604432 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 15 01:44:36.607694 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604435 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 15 01:44:36.607694 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604438 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 15 01:44:36.607694 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604441 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 15 01:44:36.607694 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604445 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 15 01:44:36.607694 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604449 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 15 01:44:36.607694 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604451 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 15 01:44:36.607694 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604454 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 15 01:44:36.607694 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604457 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 15 01:44:36.608189 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604461 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 15 01:44:36.608189 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604463 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 15 01:44:36.608189 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604465 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 15 01:44:36.608189 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604468 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 15 01:44:36.608189 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604470 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 15 01:44:36.608189 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604473 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 15 01:44:36.608189 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604476 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 15 01:44:36.608189 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604478 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 15 01:44:36.608189 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604481 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 15 01:44:36.608189 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604483 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 15 01:44:36.608189 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604485 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 15 01:44:36.608189 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604488 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 15 01:44:36.608189 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604490 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 15 01:44:36.608189 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604495 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 15 01:44:36.608189 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604501 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 15 01:44:36.608189 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604503 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 15 01:44:36.608189 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604506 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 15 01:44:36.608189 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604508 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 15 01:44:36.608189 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604511 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 15 01:44:36.608189 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604513 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 15 01:44:36.608666 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604516 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 15 01:44:36.608666 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604518 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 15 01:44:36.608666 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604521 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 15 01:44:36.608666 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604523 2578 feature_gate.go:328] unrecognized feature gate: Example Jun 15 01:44:36.608666 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604526 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 15 01:44:36.608666 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604528 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 15 01:44:36.608666 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604531 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 15 01:44:36.608666 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604533 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 15 01:44:36.608666 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604536 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 15 01:44:36.608666 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604538 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 15 01:44:36.608666 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604541 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 15 01:44:36.608666 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604543 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 15 01:44:36.608666 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604546 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 15 01:44:36.608666 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604548 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 15 01:44:36.608666 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604551 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 15 01:44:36.608666 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604553 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 15 01:44:36.608666 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604556 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 15 01:44:36.608666 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604558 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 15 01:44:36.608666 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604561 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 15 01:44:36.609139 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604563 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 15 01:44:36.609139 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604566 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 15 01:44:36.609139 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604569 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 15 01:44:36.609139 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604571 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 15 01:44:36.609139 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604573 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 15 01:44:36.609139 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604576 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 15 01:44:36.609139 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604580 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 15 01:44:36.609139 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604583 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 15 01:44:36.609139 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604586 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 15 01:44:36.609139 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604588 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 15 01:44:36.609139 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604592 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 15 01:44:36.609139 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604596 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 15 01:44:36.609139 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604599 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 15 01:44:36.609139 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604601 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 15 01:44:36.609139 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604604 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 15 01:44:36.609139 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604606 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 15 01:44:36.609139 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604608 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 15 01:44:36.609139 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604611 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 15 01:44:36.609139 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604614 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 15 01:44:36.609139 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604616 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 15 01:44:36.609660 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604619 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 15 01:44:36.609660 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604621 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 15 01:44:36.609660 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604624 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 15 01:44:36.609660 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604627 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 15 01:44:36.609660 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604629 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 15 01:44:36.609660 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604631 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 15 01:44:36.609660 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604634 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 15 01:44:36.609660 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604637 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 15 01:44:36.609660 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604639 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 15 01:44:36.609660 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604642 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 15 01:44:36.609660 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604644 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 15 01:44:36.609660 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604647 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 15 01:44:36.609660 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604649 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 15 01:44:36.609660 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604652 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jun 15 01:44:36.609660 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604655 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 15 01:44:36.609660 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604657 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 15 01:44:36.609660 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604659 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 15 01:44:36.609660 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604662 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 15 01:44:36.609660 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604665 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 15 01:44:36.609660 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.604669 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 15 01:44:36.609660 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604745 2578 flags.go:64] FLAG: --address="0.0.0.0" Jun 15 01:44:36.610221 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604752 2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 15 01:44:36.610221 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604757 2578 flags.go:64] FLAG: --anonymous-auth="true" Jun 15 01:44:36.610221 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604761 2578 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 15 01:44:36.610221 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604766 2578 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 15 01:44:36.610221 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604769 2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 15 01:44:36.610221 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604773 2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 15 01:44:36.610221 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604777 2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 15 01:44:36.610221 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604780 2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 15 01:44:36.610221 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604783 2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 15 01:44:36.610221 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604787 2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 15 01:44:36.610221 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604790 2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 15 01:44:36.610221 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604792 2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 15 01:44:36.610221 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604795 2578 flags.go:64] FLAG: --cgroup-root="" Jun 15 01:44:36.610221 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604798 2578 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 15 01:44:36.610221 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604801 2578 flags.go:64] FLAG: --client-ca-file="" Jun 15 01:44:36.610221 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604804 2578 flags.go:64] FLAG: --cloud-provider="external" Jun 15 01:44:36.610221 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604807 2578 flags.go:64] FLAG: --cluster-dns="[]" Jun 15 01:44:36.610221 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604811 2578 flags.go:64] FLAG: --cluster-domain="" Jun 15 01:44:36.610221 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604814 2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 15 01:44:36.610221 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604818 2578 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jun 15 01:44:36.610221 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604821 2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 15 01:44:36.610221 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604825 2578 flags.go:64] FLAG: --container-log-max-files="5" Jun 15 01:44:36.610221 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604828 2578 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 15 01:44:36.610221 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604831 2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 15 01:44:36.610815 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604834 2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 15 01:44:36.610815 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604837 2578 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 15 01:44:36.610815 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604840 2578 flags.go:64] FLAG: --contention-profiling="false" Jun 15 01:44:36.610815 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604843 2578 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 15 01:44:36.610815 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604846 2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 15 01:44:36.610815 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604851 2578 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 15 01:44:36.610815 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604854 2578 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 15 01:44:36.610815 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604858 2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 15 01:44:36.610815 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604861 2578 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 15 01:44:36.610815 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604864 2578 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 15 01:44:36.610815 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604867 2578 flags.go:64] FLAG: --enable-load-reader="false" Jun 15 01:44:36.610815 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604870 2578 flags.go:64] FLAG: --enable-server="true" Jun 15 01:44:36.610815 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604872 2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 15 01:44:36.610815 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604877 2578 flags.go:64] FLAG: --event-burst="100" Jun 15 01:44:36.610815 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604880 2578 flags.go:64] FLAG: --event-qps="50" Jun 15 01:44:36.610815 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604883 2578 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 15 01:44:36.610815 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604885 2578 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 15 01:44:36.610815 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604888 2578 flags.go:64] FLAG: --eviction-hard="" Jun 15 01:44:36.610815 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604892 2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 15 01:44:36.610815 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604895 2578 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 15 01:44:36.610815 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604897 2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 15 01:44:36.610815 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604900 2578 flags.go:64] FLAG: --eviction-soft="" Jun 15 01:44:36.610815 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604903 2578 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 15 01:44:36.610815 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604906 2578 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 15 01:44:36.610815 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604909 2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 15 01:44:36.611455 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604912 2578 flags.go:64] FLAG: --experimental-mounter-path="" Jun 15 01:44:36.611455 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604914 2578 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 15 01:44:36.611455 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604917 2578 flags.go:64] FLAG: --fail-swap-on="true" Jun 15 01:44:36.611455 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604921 2578 flags.go:64] FLAG: --feature-gates="" Jun 15 01:44:36.611455 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604925 2578 flags.go:64] FLAG: --file-check-frequency="20s" Jun 15 01:44:36.611455 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604928 2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 15 01:44:36.611455 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604930 2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 15 01:44:36.611455 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604933 2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 15 01:44:36.611455 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604936 2578 flags.go:64] FLAG: --healthz-port="10248" Jun 15 01:44:36.611455 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604939 2578 flags.go:64] FLAG: --help="false" Jun 15 01:44:36.611455 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604941 2578 flags.go:64] FLAG: --hostname-override="ip-10-0-143-196.ec2.internal" Jun 15 01:44:36.611455 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604945 2578 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 15 01:44:36.611455 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604953 2578 flags.go:64] FLAG: --http-check-frequency="20s" Jun 15 01:44:36.611455 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604957 2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 15 01:44:36.611455 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604960 2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 15 01:44:36.611455 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604963 2578 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 15 01:44:36.611455 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604966 2578 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 15 01:44:36.611455 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604969 2578 flags.go:64] FLAG: --image-service-endpoint="" Jun 15 01:44:36.611455 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604971 2578 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 15 01:44:36.611455 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604974 2578 flags.go:64] FLAG: --kube-api-burst="100" Jun 15 01:44:36.611455 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604977 2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 15 01:44:36.611455 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604980 2578 flags.go:64] FLAG: --kube-api-qps="50" Jun 15 01:44:36.611455 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604983 2578 flags.go:64] FLAG: --kube-reserved="" Jun 15 01:44:36.611455 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604986 2578 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 15 01:44:36.612102 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604988 2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 15 01:44:36.612102 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604991 2578 flags.go:64] FLAG: --kubelet-cgroups="" Jun 15 01:44:36.612102 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604994 2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 15 01:44:36.612102 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604997 2578 flags.go:64] FLAG: --lock-file="" Jun 15 01:44:36.612102 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.604999 2578 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 15 01:44:36.612102 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605002 2578 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 15 01:44:36.612102 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605005 2578 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 15 01:44:36.612102 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605010 2578 flags.go:64] FLAG: --log-json-split-stream="false" Jun 15 01:44:36.612102 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605013 2578 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 15 01:44:36.612102 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605016 2578 flags.go:64] FLAG: --log-text-split-stream="false" Jun 15 01:44:36.612102 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605019 2578 flags.go:64] FLAG: --logging-format="text" Jun 15 01:44:36.612102 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605021 2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 15 01:44:36.612102 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605025 2578 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 15 01:44:36.612102 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605028 2578 flags.go:64] FLAG: --manifest-url="" Jun 15 01:44:36.612102 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605031 2578 flags.go:64] FLAG: --manifest-url-header="" Jun 15 01:44:36.612102 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605034 2578 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 15 01:44:36.612102 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605037 2578 flags.go:64] FLAG: --max-open-files="1000000" Jun 15 01:44:36.612102 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605041 2578 flags.go:64] FLAG: --max-pods="110" Jun 15 01:44:36.612102 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605044 2578 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 15 01:44:36.612102 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605046 2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 15 01:44:36.612102 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605050 2578 flags.go:64] FLAG: --memory-manager-policy="None" Jun 15 01:44:36.612102 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605055 2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 15 01:44:36.612102 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605058 2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 15 01:44:36.612102 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605061 2578 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 15 01:44:36.612102 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605064 2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 15 01:44:36.612900 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605070 2578 flags.go:64] FLAG: --node-status-max-images="50" Jun 15 01:44:36.612900 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605073 2578 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 15 01:44:36.612900 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605076 2578 flags.go:64] FLAG: --oom-score-adj="-999" Jun 15 01:44:36.612900 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605079 2578 flags.go:64] FLAG: --pod-cidr="" Jun 15 01:44:36.612900 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605096 2578 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4a2e90c8ef7d783e4d635e24fb09435f5e1f6e954583512c0944f9e57bc77b52" Jun 15 01:44:36.612900 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605102 2578 flags.go:64] FLAG: --pod-manifest-path="" Jun 15 01:44:36.612900 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605105 2578 flags.go:64] FLAG: --pod-max-pids="-1" Jun 15 01:44:36.612900 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605108 2578 flags.go:64] FLAG: --pods-per-core="0" Jun 15 01:44:36.612900 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605111 2578 flags.go:64] FLAG: --port="10250" Jun 15 01:44:36.612900 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605114 2578 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 15 01:44:36.612900 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605117 2578 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0684e36ec70be1c2c" Jun 15 01:44:36.612900 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605120 2578 flags.go:64] FLAG: --qos-reserved="" Jun 15 01:44:36.612900 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605122 2578 flags.go:64] FLAG: --read-only-port="10255" Jun 15 01:44:36.612900 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605125 2578 flags.go:64] FLAG: --register-node="true" Jun 15 01:44:36.612900 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605128 2578 flags.go:64] FLAG: --register-with-taints="" Jun 15 01:44:36.612900 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605132 2578 flags.go:64] FLAG: --registry-burst="10" Jun 15 01:44:36.612900 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605135 2578 flags.go:64] FLAG: --registry-qps="5" Jun 15 01:44:36.612900 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605138 2578 flags.go:64] FLAG: --reserved-cpus="" Jun 15 01:44:36.612900 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605140 2578 flags.go:64] FLAG: --reserved-memory="" Jun 15 01:44:36.612900 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605144 2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 15 01:44:36.612900 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605148 2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 15 01:44:36.612900 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605151 2578 flags.go:64] FLAG: --rotate-certificates="false" Jun 15 01:44:36.612900 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605154 2578 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 15 01:44:36.612900 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605156 2578 flags.go:64] FLAG: --runonce="false" Jun 15 01:44:36.612900 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605159 2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 15 01:44:36.613773 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605162 2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 15 01:44:36.613773 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605165 2578 flags.go:64] FLAG: --seccomp-default="false" Jun 15 01:44:36.613773 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605168 2578 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 15 01:44:36.613773 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605171 2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 15 01:44:36.613773 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605174 2578 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 15 01:44:36.613773 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605177 2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 15 01:44:36.613773 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605180 2578 flags.go:64] FLAG: --storage-driver-password="root" Jun 15 01:44:36.613773 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605183 2578 flags.go:64] FLAG: --storage-driver-secure="false" Jun 15 01:44:36.613773 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605185 2578 flags.go:64] FLAG: --storage-driver-table="stats" Jun 15 01:44:36.613773 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605188 2578 flags.go:64] FLAG: --storage-driver-user="root" Jun 15 01:44:36.613773 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605191 2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 15 01:44:36.613773 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605194 2578 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 15 01:44:36.613773 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605197 2578 flags.go:64] FLAG: --system-cgroups="" Jun 15 01:44:36.613773 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605200 2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 15 01:44:36.613773 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605205 2578 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 15 01:44:36.613773 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605207 2578 flags.go:64] FLAG: --tls-cert-file="" Jun 15 01:44:36.613773 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605210 2578 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 15 01:44:36.613773 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605214 2578 flags.go:64] FLAG: --tls-min-version="" Jun 15 01:44:36.613773 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605217 2578 flags.go:64] FLAG: --tls-private-key-file="" Jun 15 01:44:36.613773 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605219 2578 flags.go:64] FLAG: --topology-manager-policy="none" Jun 15 01:44:36.613773 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605222 2578 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 15 01:44:36.613773 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605225 2578 flags.go:64] FLAG: --topology-manager-scope="container" Jun 15 01:44:36.613773 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605228 2578 flags.go:64] FLAG: --v="2" Jun 15 01:44:36.613773 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605232 2578 flags.go:64] FLAG: --version="false" Jun 15 01:44:36.613773 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605236 2578 flags.go:64] FLAG: --vmodule="" Jun 15 01:44:36.614440 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605240 2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 15 01:44:36.614440 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.605243 2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 15 01:44:36.614440 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605360 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 15 01:44:36.614440 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605364 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 15 01:44:36.614440 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605367 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 15 01:44:36.614440 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605370 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 15 01:44:36.614440 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605373 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 15 01:44:36.614440 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605376 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 15 01:44:36.614440 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605378 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 15 01:44:36.614440 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605381 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 15 01:44:36.614440 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605384 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 15 01:44:36.614440 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605387 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 15 01:44:36.614440 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605389 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 15 01:44:36.614440 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605392 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 15 01:44:36.614440 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605394 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 15 01:44:36.614440 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605397 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 15 01:44:36.614440 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605399 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 15 01:44:36.614440 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605402 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 15 01:44:36.614440 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605404 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 15 01:44:36.614440 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605407 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 15 01:44:36.614959 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605409 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 15 01:44:36.614959 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605411 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 15 01:44:36.614959 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605414 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 15 01:44:36.614959 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605416 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 15 01:44:36.614959 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605419 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 15 01:44:36.614959 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605421 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 15 01:44:36.614959 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605423 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 15 01:44:36.614959 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605426 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 15 01:44:36.614959 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605428 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 15 01:44:36.614959 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605430 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 15 01:44:36.614959 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605433 2578 feature_gate.go:328] unrecognized feature gate: Example Jun 15 01:44:36.614959 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605435 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 15 01:44:36.614959 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605438 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 15 01:44:36.614959 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605440 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 15 01:44:36.614959 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605443 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 15 01:44:36.614959 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605445 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 15 01:44:36.614959 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605449 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 15 01:44:36.614959 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605451 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 15 01:44:36.614959 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605453 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 15 01:44:36.614959 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605456 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 15 01:44:36.615469 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605458 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 15 01:44:36.615469 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605468 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 15 01:44:36.615469 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605472 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 15 01:44:36.615469 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605475 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 15 01:44:36.615469 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605477 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 15 01:44:36.615469 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605479 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 15 01:44:36.615469 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605482 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 15 01:44:36.615469 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605484 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 15 01:44:36.615469 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605487 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 15 01:44:36.615469 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605489 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 15 01:44:36.615469 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605492 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 15 01:44:36.615469 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605494 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 15 01:44:36.615469 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605496 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 15 01:44:36.615469 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605498 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 15 01:44:36.615469 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605501 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 15 01:44:36.615469 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605504 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 15 01:44:36.615469 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605506 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 15 01:44:36.615469 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605508 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 15 01:44:36.615469 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605511 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 15 01:44:36.615469 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605513 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 15 01:44:36.615958 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605515 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 15 01:44:36.615958 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605518 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 15 01:44:36.615958 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605520 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 15 01:44:36.615958 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605522 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 15 01:44:36.615958 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605525 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 15 01:44:36.615958 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605527 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 15 01:44:36.615958 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605530 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 15 01:44:36.615958 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605532 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 15 01:44:36.615958 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605534 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 15 01:44:36.615958 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605538 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 15 01:44:36.615958 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605540 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 15 01:44:36.615958 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605543 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 15 01:44:36.615958 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605546 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 15 01:44:36.615958 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605549 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 15 01:44:36.615958 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605553 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 15 01:44:36.615958 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605557 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 15 01:44:36.615958 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605560 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jun 15 01:44:36.615958 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605562 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 15 01:44:36.615958 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605565 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 15 01:44:36.616435 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605568 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 15 01:44:36.616435 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605570 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 15 01:44:36.616435 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605574 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 15 01:44:36.616435 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605577 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 15 01:44:36.616435 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605580 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 15 01:44:36.616435 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605583 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 15 01:44:36.616435 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605585 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 15 01:44:36.616435 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605588 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 15 01:44:36.616435 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605590 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 15 01:44:36.616435 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605593 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 15 01:44:36.616435 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605595 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 15 01:44:36.616435 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605597 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 15 01:44:36.616435 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605600 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 15 01:44:36.616435 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605602 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 15 01:44:36.616435 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605604 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 15 01:44:36.616435 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605607 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 15 01:44:36.616435 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605609 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 15 01:44:36.616435 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605611 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 15 01:44:36.616435 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605614 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 15 01:44:36.616896 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605616 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 15 01:44:36.616896 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605618 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 15 01:44:36.616896 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.605621 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 15 01:44:36.616896 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.606486 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 15 01:44:36.616896 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.615269 2578 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jun 15 01:44:36.616896 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.615413 2578 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 15 01:44:36.616896 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.615434 2578 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jun 15 01:44:36.616896 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.615444 2578 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 15 01:44:36.616896 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615505 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 15 01:44:36.616896 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615510 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 15 01:44:36.616896 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615514 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 15 01:44:36.616896 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615517 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 15 01:44:36.616896 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615520 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 15 01:44:36.616896 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615523 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 15 01:44:36.617264 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615526 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 15 01:44:36.617264 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615530 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 15 01:44:36.617264 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615533 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 15 01:44:36.617264 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615536 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 15 01:44:36.617264 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615539 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 15 01:44:36.617264 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615542 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jun 15 01:44:36.617264 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615544 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 15 01:44:36.617264 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615547 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 15 01:44:36.617264 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615550 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 15 01:44:36.617264 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615552 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 15 01:44:36.617264 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615555 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 15 01:44:36.617264 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615557 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 15 01:44:36.617264 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615560 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 15 01:44:36.617264 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615562 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 15 01:44:36.617264 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615565 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 15 01:44:36.617264 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615567 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 15 01:44:36.617264 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615570 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 15 01:44:36.617264 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615572 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 15 01:44:36.617264 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615575 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 15 01:44:36.617264 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615578 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 15 01:44:36.617751 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615581 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 15 01:44:36.617751 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615584 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 15 01:44:36.617751 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615586 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 15 01:44:36.617751 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615590 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 15 01:44:36.617751 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615593 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 15 01:44:36.617751 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615595 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 15 01:44:36.617751 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615598 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 15 01:44:36.617751 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615602 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 15 01:44:36.617751 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615606 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 15 01:44:36.617751 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615609 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 15 01:44:36.617751 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615612 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 15 01:44:36.617751 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615614 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 15 01:44:36.617751 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615617 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 15 01:44:36.617751 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615619 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 15 01:44:36.617751 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615622 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 15 01:44:36.617751 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615624 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 15 01:44:36.617751 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615627 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 15 01:44:36.617751 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615629 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 15 01:44:36.617751 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615632 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 15 01:44:36.618233 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615634 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 15 01:44:36.618233 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615643 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 15 01:44:36.618233 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615646 2578 feature_gate.go:328] unrecognized feature gate: Example Jun 15 01:44:36.618233 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615648 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 15 01:44:36.618233 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615650 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 15 01:44:36.618233 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615653 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 15 01:44:36.618233 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615657 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 15 01:44:36.618233 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615660 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 15 01:44:36.618233 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615663 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 15 01:44:36.618233 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615665 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 15 01:44:36.618233 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615668 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 15 01:44:36.618233 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615671 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 15 01:44:36.618233 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615674 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 15 01:44:36.618233 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615677 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 15 01:44:36.618233 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615681 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 15 01:44:36.618233 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615684 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 15 01:44:36.618233 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615687 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 15 01:44:36.618233 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615691 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 15 01:44:36.618233 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615694 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 15 01:44:36.618682 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615697 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 15 01:44:36.618682 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615700 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 15 01:44:36.618682 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615703 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 15 01:44:36.618682 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615706 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 15 01:44:36.618682 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615709 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 15 01:44:36.618682 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615712 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 15 01:44:36.618682 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615714 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 15 01:44:36.618682 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615717 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 15 01:44:36.618682 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615719 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 15 01:44:36.618682 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615722 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 15 01:44:36.618682 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615724 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 15 01:44:36.618682 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615727 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 15 01:44:36.618682 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615729 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 15 01:44:36.618682 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615732 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 15 01:44:36.618682 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615734 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 15 01:44:36.618682 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615738 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 15 01:44:36.618682 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615740 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 15 01:44:36.618682 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615743 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 15 01:44:36.618682 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615745 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 15 01:44:36.618682 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615748 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 15 01:44:36.619238 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615750 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 15 01:44:36.619238 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615753 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 15 01:44:36.619238 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615755 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 15 01:44:36.619238 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615758 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 15 01:44:36.619238 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615761 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 15 01:44:36.619238 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615763 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 15 01:44:36.619238 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615766 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 15 01:44:36.619238 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615768 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 15 01:44:36.619238 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615771 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 15 01:44:36.619238 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615774 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 15 01:44:36.619238 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615777 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 15 01:44:36.619238 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615779 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 15 01:44:36.619238 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615782 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 15 01:44:36.619238 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615784 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 15 01:44:36.619238 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615787 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 15 01:44:36.619605 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.615792 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 15 01:44:36.619605 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615919 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 15 01:44:36.619605 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615923 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 15 01:44:36.619605 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615926 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 15 01:44:36.619605 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615929 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 15 01:44:36.619605 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615932 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 15 01:44:36.619605 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615935 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 15 01:44:36.619605 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615938 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 15 01:44:36.619605 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615940 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 15 01:44:36.619605 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615943 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 15 01:44:36.619605 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615946 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 15 01:44:36.619605 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615948 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 15 01:44:36.619605 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615951 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 15 01:44:36.619605 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615954 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 15 01:44:36.619605 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615956 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 15 01:44:36.619965 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615959 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 15 01:44:36.619965 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615961 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 15 01:44:36.619965 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615964 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 15 01:44:36.619965 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615966 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 15 01:44:36.619965 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615969 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 15 01:44:36.619965 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615972 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 15 01:44:36.619965 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615974 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 15 01:44:36.619965 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615977 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 15 01:44:36.619965 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615979 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 15 01:44:36.619965 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615982 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 15 01:44:36.619965 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615985 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 15 01:44:36.619965 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615987 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 15 01:44:36.619965 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615990 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 15 01:44:36.619965 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615993 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 15 01:44:36.619965 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615996 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 15 01:44:36.619965 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.615999 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 15 01:44:36.619965 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616001 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 15 01:44:36.619965 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616004 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 15 01:44:36.619965 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616006 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 15 01:44:36.620441 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616009 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 15 01:44:36.620441 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616011 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 15 01:44:36.620441 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616013 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 15 01:44:36.620441 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616017 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 15 01:44:36.620441 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616019 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 15 01:44:36.620441 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616022 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 15 01:44:36.620441 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616024 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 15 01:44:36.620441 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616027 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 15 01:44:36.620441 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616029 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 15 01:44:36.620441 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616032 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 15 01:44:36.620441 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616034 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 15 01:44:36.620441 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616036 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 15 01:44:36.620441 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616040 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 15 01:44:36.620441 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616042 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 15 01:44:36.620441 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616045 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 15 01:44:36.620441 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616047 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 15 01:44:36.620441 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616050 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 15 01:44:36.620441 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616052 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 15 01:44:36.620441 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616054 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 15 01:44:36.620441 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616057 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 15 01:44:36.620942 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616059 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jun 15 01:44:36.620942 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616061 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 15 01:44:36.620942 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616064 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 15 01:44:36.620942 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616066 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 15 01:44:36.620942 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616071 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 15 01:44:36.620942 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616076 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 15 01:44:36.620942 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616079 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 15 01:44:36.620942 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616099 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 15 01:44:36.620942 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616104 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 15 01:44:36.620942 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616107 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 15 01:44:36.620942 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616110 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 15 01:44:36.620942 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616113 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 15 01:44:36.620942 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616116 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 15 01:44:36.620942 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616118 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 15 01:44:36.620942 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616121 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 15 01:44:36.620942 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616123 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 15 01:44:36.620942 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616126 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 15 01:44:36.620942 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616128 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 15 01:44:36.620942 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616131 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 15 01:44:36.620942 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616133 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 15 01:44:36.621441 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616136 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 15 01:44:36.621441 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616139 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 15 01:44:36.621441 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616141 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 15 01:44:36.621441 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616144 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 15 01:44:36.621441 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616146 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 15 01:44:36.621441 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616150 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 15 01:44:36.621441 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616153 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 15 01:44:36.621441 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616156 2578 feature_gate.go:328] unrecognized feature gate: Example Jun 15 01:44:36.621441 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616159 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 15 01:44:36.621441 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616161 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 15 01:44:36.621441 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616165 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 15 01:44:36.621441 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616168 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 15 01:44:36.621441 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616172 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 15 01:44:36.621441 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616174 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 15 01:44:36.621441 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616177 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 15 01:44:36.621441 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616179 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 15 01:44:36.621441 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616182 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 15 01:44:36.621441 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616184 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 15 01:44:36.621441 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616187 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 15 01:44:36.621441 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616190 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 15 01:44:36.621926 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616192 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 15 01:44:36.621926 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616194 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 15 01:44:36.621926 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616197 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 15 01:44:36.621926 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616201 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 15 01:44:36.621926 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616204 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 15 01:44:36.621926 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:36.616206 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 15 01:44:36.621926 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.616211 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 15 01:44:36.621926 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.616355 2578 server.go:962] "Client rotation is on, will bootstrap in background" Jun 15 01:44:36.621926 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.618866 2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 15 01:44:36.621926 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.620528 2578 server.go:1019] "Starting client certificate rotation" Jun 15 01:44:36.621926 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.620639 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 15 01:44:36.621926 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.620673 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 15 01:44:36.623419 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.623403 2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 15 01:44:36.626473 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.626452 2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 15 01:44:36.642118 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.642079 2578 log.go:25] "Validated CRI v1 runtime API" Jun 15 01:44:36.645915 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.645898 2578 log.go:25] "Validated CRI v1 image API" Jun 15 01:44:36.647590 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.647569 2578 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 15 01:44:36.650455 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.650372 2578 fs.go:135] Filesystem UUIDs: map[1dac81e8-c87b-4254-99a9-42db5acf2a48:/dev/nvme0n1p3 71c34479-831e-4b9b-a27f-0d0b33b5d118:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jun 15 01:44:36.650718 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.650666 2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 15 01:44:36.651778 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.651746 2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 15 01:44:36.657141 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.657008 2578 manager.go:216] Machine: {Timestamp:2026-06-15 01:44:36.656517478 +0000 UTC m=+0.370243445 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3096904 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec22b0c6ce8c665a5ac868d2d97bbf9c SystemUUID:ec22b0c6-ce8c-665a-5ac8-68d2d97bbf9c BootID:c1889e0e-b5ec-4ea5-b1cb-5bf6b8d68af3 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:f3:7b:ff:4c:cb Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:f3:7b:ff:4c:cb Speed:0 Mtu:9001} {Name:ovs-system MacAddress:16:5a:e5:a9:57:a5 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 15 01:44:36.657141 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.657133 2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 15 01:44:36.657281 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.657269 2578 manager.go:232] Version: {KernelVersion:5.14.0-570.120.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260608-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 15 01:44:36.658468 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.658442 2578 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 15 01:44:36.658616 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.658472 2578 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-196.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 15 01:44:36.659387 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.659378 2578 topology_manager.go:138] "Creating topology manager with none policy" Jun 15 01:44:36.659420 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.659390 2578 container_manager_linux.go:309] "Creating device plugin manager" Jun 15 01:44:36.659420 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.659398 2578 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 15 01:44:36.659420 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.659409 2578 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 15 01:44:36.661296 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.661286 2578 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jun 15 01:44:36.665854 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.665842 2578 state_mem.go:36] "Initialized new in-memory state store" Jun 15 01:44:36.665966 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.665957 2578 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jun 15 01:44:36.665996 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.665987 2578 kubelet.go:486] "Attempting to sync node with API server" Jun 15 01:44:36.666022 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.665996 2578 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 15 01:44:36.666022 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.666011 2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 15 01:44:36.666022 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.666021 2578 kubelet.go:398] "Adding apiserver pod source" Jun 15 01:44:36.666148 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.666030 2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 15 01:44:36.666517 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.666494 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 15 01:44:36.666611 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.666524 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 15 01:44:36.669132 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.669112 2578 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jun 15 01:44:36.671562 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.671550 2578 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 15 01:44:36.671618 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.671580 2578 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jun 15 01:44:36.672324 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.672305 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 15 01:44:36.672415 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.672330 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 15 01:44:36.672415 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.672340 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 15 01:44:36.672415 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.672348 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 15 01:44:36.672415 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.672364 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 15 01:44:36.672415 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.672386 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 15 01:44:36.672415 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.672397 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 15 01:44:36.672415 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.672404 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 15 01:44:36.672415 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.672412 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 15 01:44:36.672415 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.672417 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 15 01:44:36.672806 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.672426 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 15 01:44:36.672806 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.672434 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 15 01:44:36.673929 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.673919 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 15 01:44:36.673929 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.673929 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 15 01:44:36.676992 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.676977 2578 server.go:1268] "Started kubelet" Jun 15 01:44:36.678188 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.678103 2578 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 15 01:44:36.678470 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.678416 2578 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 15 01:44:36.678547 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.678492 2578 server_v1.go:49] "podresources" method="list" useActivePods=true Jun 15 01:44:36.679642 ip-10-0-143-196 systemd[1]: Started Kubernetes Kubelet. Jun 15 01:44:36.680642 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.680627 2578 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 15 01:44:36.681751 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.681738 2578 server.go:310] "Adding debug handlers to kubelet server" Jun 15 01:44:36.681965 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.681947 2578 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-196.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 15 01:44:36.682425 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:36.682392 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 15 01:44:36.684151 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:36.684129 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-196.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 15 01:44:36.684999 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:36.684980 2578 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 15 01:44:36.687325 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:36.686273 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-196.ec2.internal.18b91dc8fcfe9f02 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-196.ec2.internal,UID:ip-10-0-143-196.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-196.ec2.internal,},FirstTimestamp:2026-06-15 01:44:36.67695181 +0000 UTC m=+0.390677773,LastTimestamp:2026-06-15 01:44:36.67695181 +0000 UTC m=+0.390677773,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-196.ec2.internal,}" Jun 15 01:44:36.690295 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.690282 2578 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 15 01:44:36.690359 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.690295 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 15 01:44:36.690593 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:36.690427 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-196.ec2.internal\" not found" Jun 15 01:44:36.692643 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.692626 2578 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jun 15 01:44:36.692728 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.692661 2578 volume_manager.go:311] "The desired_state_of_world populator starts" Jun 15 01:44:36.692728 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.692677 2578 volume_manager.go:313] "Starting Kubelet Volume Manager" Jun 15 01:44:36.692884 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.692838 2578 reconstruct.go:97] "Volume reconstruction finished" Jun 15 01:44:36.692884 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.692882 2578 reconciler.go:29] "Reconciler: start to sync state" Jun 15 01:44:36.693150 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.693136 2578 factory.go:153] Registering CRI-O factory Jun 15 01:44:36.693215 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.693156 2578 factory.go:223] Registration of the crio container factory successfully Jun 15 01:44:36.693215 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.693208 2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 15 01:44:36.693302 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.693218 2578 factory.go:55] Registering systemd factory Jun 15 01:44:36.693302 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.693226 2578 factory.go:223] Registration of the systemd container factory successfully Jun 15 01:44:36.693359 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.693333 2578 factory.go:103] Registering Raw factory Jun 15 01:44:36.693463 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.693443 2578 manager.go:1196] Started watching for new ooms in manager Jun 15 01:44:36.693919 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.693903 2578 manager.go:349] Starting recovery of all containers Jun 15 01:44:36.694183 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.694162 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-pg8t4" Jun 15 01:44:36.695451 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:36.695429 2578 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-196.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jun 15 01:44:36.695526 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:36.695504 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jun 15 01:44:36.702517 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.702491 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-pg8t4" Jun 15 01:44:36.704344 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.704328 2578 manager.go:354] Recovery completed Jun 15 01:44:36.705498 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:36.705475 2578 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jun 15 01:44:36.708333 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.708321 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 15 01:44:36.710200 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.710184 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-196.ec2.internal" event="NodeHasSufficientMemory" Jun 15 01:44:36.710273 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.710213 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-196.ec2.internal" event="NodeHasNoDiskPressure" Jun 15 01:44:36.710273 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.710224 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-196.ec2.internal" event="NodeHasSufficientPID" Jun 15 01:44:36.710806 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.710788 2578 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 15 01:44:36.710806 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.710800 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 15 01:44:36.710806 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.710804 2578 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 15 01:44:36.710953 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.710822 2578 state_mem.go:36] "Initialized new in-memory state store" Jun 15 01:44:36.711798 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.711782 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-196.ec2.internal" event="NodeHasSufficientMemory" Jun 15 01:44:36.711857 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.711814 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-196.ec2.internal" event="NodeHasNoDiskPressure" Jun 15 01:44:36.711857 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.711826 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-196.ec2.internal" event="NodeHasSufficientPID" Jun 15 01:44:36.712721 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:36.712618 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-196.ec2.internal.18b91dc8fef9f154 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-196.ec2.internal,UID:ip-10-0-143-196.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-196.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-196.ec2.internal,},FirstTimestamp:2026-06-15 01:44:36.710199636 +0000 UTC m=+0.423925602,LastTimestamp:2026-06-15 01:44:36.710199636 +0000 UTC m=+0.423925602,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-196.ec2.internal,}" Jun 15 01:44:36.712912 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.712894 2578 policy_none.go:49] "None policy: Start" Jun 15 01:44:36.712996 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.712916 2578 memory_manager.go:187] "Starting memorymanager" policy="None" Jun 15 01:44:36.712996 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.712956 2578 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jun 15 01:44:36.715026 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.715015 2578 policy_none.go:47] "Start" Jun 15 01:44:36.746703 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.746686 2578 manager.go:337] "Starting Device Plugin manager" Jun 15 01:44:36.772862 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:36.746724 2578 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 15 01:44:36.772862 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.746737 2578 server.go:87] "Starting device plugin registration server" Jun 15 01:44:36.772862 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.746965 2578 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 15 01:44:36.772862 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.746975 2578 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 15 01:44:36.772862 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.747097 2578 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 15 01:44:36.772862 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.747172 2578 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 15 01:44:36.772862 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.747179 2578 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 15 01:44:36.772862 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:36.747938 2578 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 15 01:44:36.772862 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:36.747983 2578 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-196.ec2.internal\" not found" Jun 15 01:44:36.772862 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.758421 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jun 15 01:44:36.772862 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.759640 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jun 15 01:44:36.772862 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.759672 2578 status_manager.go:244] "Starting to sync pod status with apiserver" Jun 15 01:44:36.772862 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.759691 2578 kubelet.go:2448] "Starting kubelet main sync loop" Jun 15 01:44:36.772862 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:36.759749 2578 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 15 01:44:36.772862 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.762017 2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 15 01:44:36.847130 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.847036 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 15 01:44:36.847984 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.847967 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-196.ec2.internal" event="NodeHasSufficientMemory" Jun 15 01:44:36.848049 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.848008 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-196.ec2.internal" event="NodeHasNoDiskPressure" Jun 15 01:44:36.848049 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.848023 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-196.ec2.internal" event="NodeHasSufficientPID" Jun 15 01:44:36.848049 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.848047 2578 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-196.ec2.internal" Jun 15 01:44:36.857919 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.857898 2578 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-196.ec2.internal" Jun 15 01:44:36.857980 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:36.857922 2578 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-196.ec2.internal\": node \"ip-10-0-143-196.ec2.internal\" not found" Jun 15 01:44:36.859987 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.859965 2578 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-196.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-196.ec2.internal"] Jun 15 01:44:36.860064 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.860053 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 15 01:44:36.860858 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.860842 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-196.ec2.internal" event="NodeHasSufficientMemory" Jun 15 01:44:36.860942 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.860872 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-196.ec2.internal" event="NodeHasNoDiskPressure" Jun 15 01:44:36.860942 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.860886 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-196.ec2.internal" event="NodeHasSufficientPID" Jun 15 01:44:36.863185 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.863171 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 15 01:44:36.863329 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.863311 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-196.ec2.internal" Jun 15 01:44:36.863377 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.863343 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 15 01:44:36.863898 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.863883 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-196.ec2.internal" event="NodeHasSufficientMemory" Jun 15 01:44:36.863965 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.863900 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-196.ec2.internal" event="NodeHasSufficientMemory" Jun 15 01:44:36.863965 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.863911 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-196.ec2.internal" event="NodeHasNoDiskPressure" Jun 15 01:44:36.863965 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.863919 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-196.ec2.internal" event="NodeHasNoDiskPressure" Jun 15 01:44:36.863965 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.863923 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-196.ec2.internal" event="NodeHasSufficientPID" Jun 15 01:44:36.863965 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.863929 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-196.ec2.internal" event="NodeHasSufficientPID" Jun 15 01:44:36.866181 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.866165 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-196.ec2.internal" Jun 15 01:44:36.866269 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.866192 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 15 01:44:36.866911 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.866895 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-196.ec2.internal" event="NodeHasSufficientMemory" Jun 15 01:44:36.866992 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.866923 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-196.ec2.internal" event="NodeHasNoDiskPressure" Jun 15 01:44:36.866992 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.866936 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-196.ec2.internal" event="NodeHasSufficientPID" Jun 15 01:44:36.879642 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:36.879621 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-196.ec2.internal\" not found" Jun 15 01:44:36.883628 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:36.883613 2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-196.ec2.internal\" not found" node="ip-10-0-143-196.ec2.internal" Jun 15 01:44:36.887997 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:36.887983 2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-196.ec2.internal\" not found" node="ip-10-0-143-196.ec2.internal" Jun 15 01:44:36.894587 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.894572 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/36e239b4d7762116a74696279f22b0af-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-196.ec2.internal\" (UID: \"36e239b4d7762116a74696279f22b0af\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-196.ec2.internal" Jun 15 01:44:36.894648 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.894594 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/36e239b4d7762116a74696279f22b0af-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-196.ec2.internal\" (UID: \"36e239b4d7762116a74696279f22b0af\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-196.ec2.internal" Jun 15 01:44:36.894648 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.894611 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3461aba319c72ca2a9e406f3c9bf17da-config\") pod \"kube-apiserver-proxy-ip-10-0-143-196.ec2.internal\" (UID: \"3461aba319c72ca2a9e406f3c9bf17da\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-196.ec2.internal" Jun 15 01:44:36.979946 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:36.979901 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-196.ec2.internal\" not found" Jun 15 01:44:36.995449 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.995424 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/36e239b4d7762116a74696279f22b0af-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-196.ec2.internal\" (UID: \"36e239b4d7762116a74696279f22b0af\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-196.ec2.internal" Jun 15 01:44:36.995524 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.995374 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/36e239b4d7762116a74696279f22b0af-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-196.ec2.internal\" (UID: \"36e239b4d7762116a74696279f22b0af\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-196.ec2.internal" Jun 15 01:44:36.995524 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.995493 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3461aba319c72ca2a9e406f3c9bf17da-config\") pod \"kube-apiserver-proxy-ip-10-0-143-196.ec2.internal\" (UID: \"3461aba319c72ca2a9e406f3c9bf17da\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-196.ec2.internal" Jun 15 01:44:36.995524 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.995509 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/36e239b4d7762116a74696279f22b0af-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-196.ec2.internal\" (UID: \"36e239b4d7762116a74696279f22b0af\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-196.ec2.internal" Jun 15 01:44:36.995615 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.995534 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/36e239b4d7762116a74696279f22b0af-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-196.ec2.internal\" (UID: \"36e239b4d7762116a74696279f22b0af\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-196.ec2.internal" Jun 15 01:44:36.995615 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:36.995553 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3461aba319c72ca2a9e406f3c9bf17da-config\") pod \"kube-apiserver-proxy-ip-10-0-143-196.ec2.internal\" (UID: \"3461aba319c72ca2a9e406f3c9bf17da\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-196.ec2.internal" Jun 15 01:44:37.080557 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:37.080525 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-196.ec2.internal\" not found" Jun 15 01:44:37.181480 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:37.181422 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-196.ec2.internal\" not found" Jun 15 01:44:37.187593 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:37.187568 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-196.ec2.internal" Jun 15 01:44:37.191805 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:37.191791 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-196.ec2.internal" Jun 15 01:44:37.281532 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:37.281486 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-196.ec2.internal\" not found" Jun 15 01:44:37.382169 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:37.382122 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-196.ec2.internal\" not found" Jun 15 01:44:37.482843 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:37.482762 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-196.ec2.internal\" not found" Jun 15 01:44:37.583434 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:37.583380 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-196.ec2.internal\" not found" Jun 15 01:44:37.620003 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:37.619974 2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 15 01:44:37.620623 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:37.620166 2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 15 01:44:37.684221 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:37.684191 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-196.ec2.internal\" not found" Jun 15 01:44:37.709187 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:37.709151 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-14 01:39:36 +0000 UTC" deadline="2028-02-07 01:34:15.988044597 +0000 UTC" Jun 15 01:44:37.709187 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:37.709185 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14447h49m38.278862462s" Jun 15 01:44:37.741772 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:37.741712 2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 15 01:44:37.785100 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:37.785052 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-196.ec2.internal\" not found" Jun 15 01:44:37.834859 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:37.834682 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36e239b4d7762116a74696279f22b0af.slice/crio-f92f0c34dc4d9dcc162df479e2cc230bf860a821e2b209efc87d00fba947d1a8 WatchSource:0}: Error finding container f92f0c34dc4d9dcc162df479e2cc230bf860a821e2b209efc87d00fba947d1a8: Status 404 returned error can't find the container with id f92f0c34dc4d9dcc162df479e2cc230bf860a821e2b209efc87d00fba947d1a8 Jun 15 01:44:37.835110 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:37.835062 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3461aba319c72ca2a9e406f3c9bf17da.slice/crio-c298589dd41c065001f3051f7688eed9826f5ec7cee5f8354d91bfee690729cc WatchSource:0}: Error finding container c298589dd41c065001f3051f7688eed9826f5ec7cee5f8354d91bfee690729cc: Status 404 returned error can't find the container with id c298589dd41c065001f3051f7688eed9826f5ec7cee5f8354d91bfee690729cc Jun 15 01:44:37.840269 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:37.840252 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 15 01:44:37.886191 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:37.886141 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-196.ec2.internal\" not found" Jun 15 01:44:37.986790 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:37.986760 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-196.ec2.internal\" not found" Jun 15 01:44:38.013963 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.013859 2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 15 01:44:38.091209 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.091184 2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-196.ec2.internal" Jun 15 01:44:38.103819 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.103797 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 15 01:44:38.105916 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.105901 2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-196.ec2.internal" Jun 15 01:44:38.114357 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.114336 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 15 01:44:38.121657 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.121640 2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 15 01:44:38.561796 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.561766 2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 15 01:44:38.667388 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.667341 2578 apiserver.go:52] "Watching apiserver" Jun 15 01:44:38.679120 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.679070 2578 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 15 01:44:38.679593 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.679567 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-7l5ft","openshift-ovn-kubernetes/ovnkube-node-cf286","kube-system/global-pull-secret-syncer-6272r","kube-system/kube-apiserver-proxy-ip-10-0-143-196.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7nhx","openshift-dns/node-resolver-pmkl7","openshift-image-registry/node-ca-r8dqf","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-196.ec2.internal","openshift-multus/multus-4xrm5","openshift-multus/network-metrics-daemon-n4p82","kube-system/konnectivity-agent-tnmwb","openshift-cluster-node-tuning-operator/tuned-8h7vw","openshift-multus/multus-additional-cni-plugins-8r9kz","openshift-network-diagnostics/network-check-target-f2kkk"] Jun 15 01:44:38.682230 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.682204 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7nhx" Jun 15 01:44:38.684510 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.684488 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.685609 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.685538 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-28g4w\"" Jun 15 01:44:38.685609 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.685586 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 15 01:44:38.685776 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.685551 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 15 01:44:38.685834 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.685807 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 15 01:44:38.686752 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.686731 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-tnmwb" Jun 15 01:44:38.687135 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.687118 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 15 01:44:38.687232 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.687136 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-z4tcp\"" Jun 15 01:44:38.687527 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.687508 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 15 01:44:38.688625 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.688595 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 15 01:44:38.688824 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.688625 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 15 01:44:38.688824 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.688775 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 15 01:44:38.688824 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.688824 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 15 01:44:38.689169 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.689149 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 15 01:44:38.689268 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.689156 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-m22tp\"" Jun 15 01:44:38.689268 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.689207 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 15 01:44:38.690289 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.690272 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7l5ft" Jun 15 01:44:38.691470 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.691453 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 15 01:44:38.692555 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.692538 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4p82" Jun 15 01:44:38.692636 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:38.692607 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n4p82" podUID="e1719195-2a1b-4c96-abe3-c25bda154d3d" Jun 15 01:44:38.693334 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.693309 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-bn666\"" Jun 15 01:44:38.693427 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.693366 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 15 01:44:38.693812 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.693644 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 15 01:44:38.693812 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.693682 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 15 01:44:38.696057 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.696037 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pmkl7" Jun 15 01:44:38.698645 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.698626 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 15 01:44:38.698755 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.698673 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-pwggw\"" Jun 15 01:44:38.698755 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.698633 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 15 01:44:38.699386 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.699156 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-r8dqf" Jun 15 01:44:38.702114 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.701597 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4xrm5" Jun 15 01:44:38.702114 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.701798 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 15 01:44:38.702114 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.701908 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 15 01:44:38.702114 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.701992 2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 15 01:44:38.702114 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.702044 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-dnvzg\"" Jun 15 01:44:38.702114 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.701973 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 15 01:44:38.704695 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.704291 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6272r" Jun 15 01:44:38.704695 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:38.704391 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6272r" podUID="a6368c58-4355-4d1c-b621-75c5bf0e7394" Jun 15 01:44:38.704695 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.704542 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 15 01:44:38.704695 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.704613 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 15 01:44:38.706350 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.706324 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-rq744\"" Jun 15 01:44:38.707896 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.707250 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 15 01:44:38.707896 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.707521 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 15 01:44:38.708806 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.708789 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8h7vw" Jun 15 01:44:38.709039 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.708932 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3cf2d35f-dae7-42c6-8f35-5b0ccb5e4750-tmp-dir\") pod \"node-resolver-pmkl7\" (UID: \"3cf2d35f-dae7-42c6-8f35-5b0ccb5e4750\") " pod="openshift-dns/node-resolver-pmkl7" Jun 15 01:44:38.709155 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.709066 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjtjl\" (UniqueName: \"kubernetes.io/projected/029053ce-e237-4fa4-8a82-bc70b7be8faa-kube-api-access-sjtjl\") pod \"aws-ebs-csi-driver-node-v7nhx\" (UID: \"029053ce-e237-4fa4-8a82-bc70b7be8faa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7nhx" Jun 15 01:44:38.709217 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.709186 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/64f557b0-78d8-45a6-9f54-756417231d6c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.709268 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.709256 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/64f557b0-78d8-45a6-9f54-756417231d6c-env-overrides\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.709422 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.709402 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwhfs\" (UniqueName: \"kubernetes.io/projected/e1719195-2a1b-4c96-abe3-c25bda154d3d-kube-api-access-cwhfs\") pod \"network-metrics-daemon-n4p82\" (UID: \"e1719195-2a1b-4c96-abe3-c25bda154d3d\") " pod="openshift-multus/network-metrics-daemon-n4p82" Jun 15 01:44:38.709483 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.709439 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5ftj\" (UniqueName: \"kubernetes.io/projected/3cf2d35f-dae7-42c6-8f35-5b0ccb5e4750-kube-api-access-c5ftj\") pod \"node-resolver-pmkl7\" (UID: \"3cf2d35f-dae7-42c6-8f35-5b0ccb5e4750\") " pod="openshift-dns/node-resolver-pmkl7" Jun 15 01:44:38.709533 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.709465 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/029053ce-e237-4fa4-8a82-bc70b7be8faa-registration-dir\") pod \"aws-ebs-csi-driver-node-v7nhx\" (UID: \"029053ce-e237-4fa4-8a82-bc70b7be8faa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7nhx" Jun 15 01:44:38.709533 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.709511 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/64f557b0-78d8-45a6-9f54-756417231d6c-log-socket\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.709625 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.709532 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d9f8083b-6138-4b4d-b8b8-d11ca6811a9d-agent-certs\") pod \"konnectivity-agent-tnmwb\" (UID: \"d9f8083b-6138-4b4d-b8b8-d11ca6811a9d\") " pod="kube-system/konnectivity-agent-tnmwb" Jun 15 01:44:38.709680 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.709660 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/64f557b0-78d8-45a6-9f54-756417231d6c-host-kubelet\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.709736 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.709686 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/64f557b0-78d8-45a6-9f54-756417231d6c-etc-openvswitch\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.709736 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.709701 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/64f557b0-78d8-45a6-9f54-756417231d6c-ovn-node-metrics-cert\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.709736 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.709720 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cbbc74f1-9650-4453-955f-fa40d2adb7bc-host-slash\") pod \"iptables-alerter-7l5ft\" (UID: \"cbbc74f1-9650-4453-955f-fa40d2adb7bc\") " pod="openshift-network-operator/iptables-alerter-7l5ft" Jun 15 01:44:38.709859 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.709740 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxzg8\" (UniqueName: \"kubernetes.io/projected/cbbc74f1-9650-4453-955f-fa40d2adb7bc-kube-api-access-gxzg8\") pod \"iptables-alerter-7l5ft\" (UID: \"cbbc74f1-9650-4453-955f-fa40d2adb7bc\") " pod="openshift-network-operator/iptables-alerter-7l5ft" Jun 15 01:44:38.709859 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.709758 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/029053ce-e237-4fa4-8a82-bc70b7be8faa-sys-fs\") pod \"aws-ebs-csi-driver-node-v7nhx\" (UID: \"029053ce-e237-4fa4-8a82-bc70b7be8faa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7nhx" Jun 15 01:44:38.709859 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.709775 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/64f557b0-78d8-45a6-9f54-756417231d6c-run-openvswitch\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.709859 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.709848 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/64f557b0-78d8-45a6-9f54-756417231d6c-run-ovn\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.710035 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.709868 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/64f557b0-78d8-45a6-9f54-756417231d6c-host-run-ovn-kubernetes\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.710035 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.709941 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/64f557b0-78d8-45a6-9f54-756417231d6c-host-cni-bin\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.710035 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.709973 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92l2k\" (UniqueName: \"kubernetes.io/projected/64f557b0-78d8-45a6-9f54-756417231d6c-kube-api-access-92l2k\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.710035 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.710012 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/64f557b0-78d8-45a6-9f54-756417231d6c-systemd-units\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.710230 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.710058 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/64f557b0-78d8-45a6-9f54-756417231d6c-var-lib-openvswitch\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.710230 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.710215 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/64f557b0-78d8-45a6-9f54-756417231d6c-ovnkube-script-lib\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.710323 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.710253 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d9f8083b-6138-4b4d-b8b8-d11ca6811a9d-konnectivity-ca\") pod \"konnectivity-agent-tnmwb\" (UID: \"d9f8083b-6138-4b4d-b8b8-d11ca6811a9d\") " pod="kube-system/konnectivity-agent-tnmwb" Jun 15 01:44:38.710323 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.710284 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3cf2d35f-dae7-42c6-8f35-5b0ccb5e4750-hosts-file\") pod \"node-resolver-pmkl7\" (UID: \"3cf2d35f-dae7-42c6-8f35-5b0ccb5e4750\") " pod="openshift-dns/node-resolver-pmkl7" Jun 15 01:44:38.710409 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.710314 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/029053ce-e237-4fa4-8a82-bc70b7be8faa-kubelet-dir\") pod \"aws-ebs-csi-driver-node-v7nhx\" (UID: \"029053ce-e237-4fa4-8a82-bc70b7be8faa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7nhx" Jun 15 01:44:38.710857 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.710768 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/029053ce-e237-4fa4-8a82-bc70b7be8faa-device-dir\") pod \"aws-ebs-csi-driver-node-v7nhx\" (UID: \"029053ce-e237-4fa4-8a82-bc70b7be8faa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7nhx" Jun 15 01:44:38.710857 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.710803 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/029053ce-e237-4fa4-8a82-bc70b7be8faa-etc-selinux\") pod \"aws-ebs-csi-driver-node-v7nhx\" (UID: \"029053ce-e237-4fa4-8a82-bc70b7be8faa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7nhx" Jun 15 01:44:38.710857 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.710846 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/64f557b0-78d8-45a6-9f54-756417231d6c-host-run-netns\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.711060 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.710918 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/64f557b0-78d8-45a6-9f54-756417231d6c-node-log\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.711060 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.710951 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/64f557b0-78d8-45a6-9f54-756417231d6c-host-cni-netd\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.711060 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.710991 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/029053ce-e237-4fa4-8a82-bc70b7be8faa-socket-dir\") pod \"aws-ebs-csi-driver-node-v7nhx\" (UID: \"029053ce-e237-4fa4-8a82-bc70b7be8faa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7nhx" Jun 15 01:44:38.711060 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.711038 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/64f557b0-78d8-45a6-9f54-756417231d6c-ovnkube-config\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.711273 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.711069 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cbbc74f1-9650-4453-955f-fa40d2adb7bc-iptables-alerter-script\") pod \"iptables-alerter-7l5ft\" (UID: \"cbbc74f1-9650-4453-955f-fa40d2adb7bc\") " pod="openshift-network-operator/iptables-alerter-7l5ft" Jun 15 01:44:38.711273 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.711120 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/64f557b0-78d8-45a6-9f54-756417231d6c-host-slash\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.711373 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.711313 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/64f557b0-78d8-45a6-9f54-756417231d6c-run-systemd\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.711373 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.711345 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1719195-2a1b-4c96-abe3-c25bda154d3d-metrics-certs\") pod \"network-metrics-daemon-n4p82\" (UID: \"e1719195-2a1b-4c96-abe3-c25bda154d3d\") " pod="openshift-multus/network-metrics-daemon-n4p82" Jun 15 01:44:38.712186 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.711800 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-bwffd\"" Jun 15 01:44:38.712186 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.712129 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 15 01:44:38.712613 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.712330 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 15 01:44:38.713259 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.712784 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8r9kz" Jun 15 01:44:38.713259 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.713074 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2kkk" Jun 15 01:44:38.713259 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:38.713183 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2kkk" podUID="a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1" Jun 15 01:44:38.715248 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.715230 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-mdfjn\"" Jun 15 01:44:38.716243 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.716222 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 15 01:44:38.716318 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.716264 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 15 01:44:38.721598 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.721577 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-92gnq" Jun 15 01:44:38.729560 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.729539 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-92gnq" Jun 15 01:44:38.764288 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.764244 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-196.ec2.internal" event={"ID":"36e239b4d7762116a74696279f22b0af","Type":"ContainerStarted","Data":"f92f0c34dc4d9dcc162df479e2cc230bf860a821e2b209efc87d00fba947d1a8"} Jun 15 01:44:38.765519 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.765499 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-196.ec2.internal" event={"ID":"3461aba319c72ca2a9e406f3c9bf17da","Type":"ContainerStarted","Data":"c298589dd41c065001f3051f7688eed9826f5ec7cee5f8354d91bfee690729cc"} Jun 15 01:44:38.794333 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.794308 2578 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jun 15 01:44:38.812295 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.812215 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9a743adf-5ee6-4357-9206-1ccada80d911-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8r9kz\" (UID: \"9a743adf-5ee6-4357-9206-1ccada80d911\") " pod="openshift-multus/multus-additional-cni-plugins-8r9kz" Jun 15 01:44:38.812295 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.812263 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/029053ce-e237-4fa4-8a82-bc70b7be8faa-kubelet-dir\") pod \"aws-ebs-csi-driver-node-v7nhx\" (UID: \"029053ce-e237-4fa4-8a82-bc70b7be8faa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7nhx" Jun 15 01:44:38.812469 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.812296 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/029053ce-e237-4fa4-8a82-bc70b7be8faa-etc-selinux\") pod \"aws-ebs-csi-driver-node-v7nhx\" (UID: \"029053ce-e237-4fa4-8a82-bc70b7be8faa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7nhx" Jun 15 01:44:38.812469 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.812319 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cbbc74f1-9650-4453-955f-fa40d2adb7bc-iptables-alerter-script\") pod \"iptables-alerter-7l5ft\" (UID: \"cbbc74f1-9650-4453-955f-fa40d2adb7bc\") " pod="openshift-network-operator/iptables-alerter-7l5ft" Jun 15 01:44:38.812469 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.812344 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a6368c58-4355-4d1c-b621-75c5bf0e7394-dbus\") pod \"global-pull-secret-syncer-6272r\" (UID: \"a6368c58-4355-4d1c-b621-75c5bf0e7394\") " pod="kube-system/global-pull-secret-syncer-6272r" Jun 15 01:44:38.812469 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.812368 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9a743adf-5ee6-4357-9206-1ccada80d911-os-release\") pod \"multus-additional-cni-plugins-8r9kz\" (UID: \"9a743adf-5ee6-4357-9206-1ccada80d911\") " pod="openshift-multus/multus-additional-cni-plugins-8r9kz" Jun 15 01:44:38.812469 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.812395 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/64f557b0-78d8-45a6-9f54-756417231d6c-ovnkube-config\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.812469 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.812419 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9612108c-f217-4efc-9af1-9500c72da5e3-etc-sysctl-d\") pod \"tuned-8h7vw\" (UID: \"9612108c-f217-4efc-9af1-9500c72da5e3\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7vw" Jun 15 01:44:38.812469 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.812443 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9612108c-f217-4efc-9af1-9500c72da5e3-lib-modules\") pod \"tuned-8h7vw\" (UID: \"9612108c-f217-4efc-9af1-9500c72da5e3\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7vw" Jun 15 01:44:38.812469 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.812459 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9a743adf-5ee6-4357-9206-1ccada80d911-cnibin\") pod \"multus-additional-cni-plugins-8r9kz\" (UID: \"9a743adf-5ee6-4357-9206-1ccada80d911\") " pod="openshift-multus/multus-additional-cni-plugins-8r9kz" Jun 15 01:44:38.812700 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.812474 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3cf2d35f-dae7-42c6-8f35-5b0ccb5e4750-tmp-dir\") pod \"node-resolver-pmkl7\" (UID: \"3cf2d35f-dae7-42c6-8f35-5b0ccb5e4750\") " pod="openshift-dns/node-resolver-pmkl7" Jun 15 01:44:38.812700 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.812488 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/17743aa6-c9cb-458c-b3d3-16c6a774171a-host\") pod \"node-ca-r8dqf\" (UID: \"17743aa6-c9cb-458c-b3d3-16c6a774171a\") " pod="openshift-image-registry/node-ca-r8dqf" Jun 15 01:44:38.812700 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.812505 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/134d75e6-f2f5-4264-9f35-c231e06bdbb9-host-var-lib-kubelet\") pod \"multus-4xrm5\" (UID: \"134d75e6-f2f5-4264-9f35-c231e06bdbb9\") " pod="openshift-multus/multus-4xrm5" Jun 15 01:44:38.812700 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.812521 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d9f8083b-6138-4b4d-b8b8-d11ca6811a9d-agent-certs\") pod \"konnectivity-agent-tnmwb\" (UID: \"d9f8083b-6138-4b4d-b8b8-d11ca6811a9d\") " pod="kube-system/konnectivity-agent-tnmwb" Jun 15 01:44:38.812700 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.812542 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/134d75e6-f2f5-4264-9f35-c231e06bdbb9-multus-socket-dir-parent\") pod \"multus-4xrm5\" (UID: \"134d75e6-f2f5-4264-9f35-c231e06bdbb9\") " pod="openshift-multus/multus-4xrm5" Jun 15 01:44:38.812700 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.812558 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/64f557b0-78d8-45a6-9f54-756417231d6c-host-kubelet\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.812700 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.812573 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/64f557b0-78d8-45a6-9f54-756417231d6c-etc-openvswitch\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.812700 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.812587 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/134d75e6-f2f5-4264-9f35-c231e06bdbb9-cni-binary-copy\") pod \"multus-4xrm5\" (UID: \"134d75e6-f2f5-4264-9f35-c231e06bdbb9\") " pod="openshift-multus/multus-4xrm5" Jun 15 01:44:38.812700 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.812601 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9612108c-f217-4efc-9af1-9500c72da5e3-etc-sysctl-conf\") pod \"tuned-8h7vw\" (UID: \"9612108c-f217-4efc-9af1-9500c72da5e3\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7vw" Jun 15 01:44:38.812700 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.812616 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcmtt\" (UniqueName: \"kubernetes.io/projected/9612108c-f217-4efc-9af1-9500c72da5e3-kube-api-access-mcmtt\") pod \"tuned-8h7vw\" (UID: \"9612108c-f217-4efc-9af1-9500c72da5e3\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7vw" Jun 15 01:44:38.812700 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.812631 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9a743adf-5ee6-4357-9206-1ccada80d911-cni-binary-copy\") pod \"multus-additional-cni-plugins-8r9kz\" (UID: \"9a743adf-5ee6-4357-9206-1ccada80d911\") " pod="openshift-multus/multus-additional-cni-plugins-8r9kz" Jun 15 01:44:38.812700 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.812646 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ts659\" (UniqueName: \"kubernetes.io/projected/9a743adf-5ee6-4357-9206-1ccada80d911-kube-api-access-ts659\") pod \"multus-additional-cni-plugins-8r9kz\" (UID: \"9a743adf-5ee6-4357-9206-1ccada80d911\") " pod="openshift-multus/multus-additional-cni-plugins-8r9kz" Jun 15 01:44:38.812700 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.812660 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/029053ce-e237-4fa4-8a82-bc70b7be8faa-sys-fs\") pod \"aws-ebs-csi-driver-node-v7nhx\" (UID: \"029053ce-e237-4fa4-8a82-bc70b7be8faa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7nhx" Jun 15 01:44:38.812700 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.812681 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/64f557b0-78d8-45a6-9f54-756417231d6c-run-ovn\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.812700 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.812697 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/64f557b0-78d8-45a6-9f54-756417231d6c-host-run-ovn-kubernetes\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.813210 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.812711 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/64f557b0-78d8-45a6-9f54-756417231d6c-host-cni-bin\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.813210 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.812728 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/134d75e6-f2f5-4264-9f35-c231e06bdbb9-system-cni-dir\") pod \"multus-4xrm5\" (UID: \"134d75e6-f2f5-4264-9f35-c231e06bdbb9\") " pod="openshift-multus/multus-4xrm5" Jun 15 01:44:38.813210 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.812741 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/134d75e6-f2f5-4264-9f35-c231e06bdbb9-multus-conf-dir\") pod \"multus-4xrm5\" (UID: \"134d75e6-f2f5-4264-9f35-c231e06bdbb9\") " pod="openshift-multus/multus-4xrm5" Jun 15 01:44:38.813210 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.812765 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a6368c58-4355-4d1c-b621-75c5bf0e7394-kubelet-config\") pod \"global-pull-secret-syncer-6272r\" (UID: \"a6368c58-4355-4d1c-b621-75c5bf0e7394\") " pod="kube-system/global-pull-secret-syncer-6272r" Jun 15 01:44:38.813210 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.812782 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9612108c-f217-4efc-9af1-9500c72da5e3-etc-modprobe-d\") pod \"tuned-8h7vw\" (UID: \"9612108c-f217-4efc-9af1-9500c72da5e3\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7vw" Jun 15 01:44:38.813210 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.812807 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/64f557b0-78d8-45a6-9f54-756417231d6c-systemd-units\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.813210 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.812823 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/64f557b0-78d8-45a6-9f54-756417231d6c-var-lib-openvswitch\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.813210 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.812838 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/64f557b0-78d8-45a6-9f54-756417231d6c-ovnkube-script-lib\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.813210 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.812853 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d9f8083b-6138-4b4d-b8b8-d11ca6811a9d-konnectivity-ca\") pod \"konnectivity-agent-tnmwb\" (UID: \"d9f8083b-6138-4b4d-b8b8-d11ca6811a9d\") " pod="kube-system/konnectivity-agent-tnmwb" Jun 15 01:44:38.813210 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.812871 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/134d75e6-f2f5-4264-9f35-c231e06bdbb9-cnibin\") pod \"multus-4xrm5\" (UID: \"134d75e6-f2f5-4264-9f35-c231e06bdbb9\") " pod="openshift-multus/multus-4xrm5" Jun 15 01:44:38.813210 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.812884 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9612108c-f217-4efc-9af1-9500c72da5e3-host\") pod \"tuned-8h7vw\" (UID: \"9612108c-f217-4efc-9af1-9500c72da5e3\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7vw" Jun 15 01:44:38.813210 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.812899 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9612108c-f217-4efc-9af1-9500c72da5e3-etc-tuned\") pod \"tuned-8h7vw\" (UID: \"9612108c-f217-4efc-9af1-9500c72da5e3\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7vw" Jun 15 01:44:38.813210 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.812915 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3cf2d35f-dae7-42c6-8f35-5b0ccb5e4750-hosts-file\") pod \"node-resolver-pmkl7\" (UID: \"3cf2d35f-dae7-42c6-8f35-5b0ccb5e4750\") " pod="openshift-dns/node-resolver-pmkl7" Jun 15 01:44:38.813210 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.812929 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/029053ce-e237-4fa4-8a82-bc70b7be8faa-device-dir\") pod \"aws-ebs-csi-driver-node-v7nhx\" (UID: \"029053ce-e237-4fa4-8a82-bc70b7be8faa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7nhx" Jun 15 01:44:38.813210 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.812944 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/64f557b0-78d8-45a6-9f54-756417231d6c-host-run-netns\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.813210 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.812962 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/64f557b0-78d8-45a6-9f54-756417231d6c-node-log\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.813210 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.812985 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/64f557b0-78d8-45a6-9f54-756417231d6c-host-cni-netd\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.813683 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.813001 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxzqh\" (UniqueName: \"kubernetes.io/projected/134d75e6-f2f5-4264-9f35-c231e06bdbb9-kube-api-access-wxzqh\") pod \"multus-4xrm5\" (UID: \"134d75e6-f2f5-4264-9f35-c231e06bdbb9\") " pod="openshift-multus/multus-4xrm5" Jun 15 01:44:38.813683 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.813015 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnvlx\" (UniqueName: \"kubernetes.io/projected/a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1-kube-api-access-fnvlx\") pod \"network-check-target-f2kkk\" (UID: \"a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1\") " pod="openshift-network-diagnostics/network-check-target-f2kkk" Jun 15 01:44:38.813683 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.813034 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/029053ce-e237-4fa4-8a82-bc70b7be8faa-socket-dir\") pod \"aws-ebs-csi-driver-node-v7nhx\" (UID: \"029053ce-e237-4fa4-8a82-bc70b7be8faa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7nhx" Jun 15 01:44:38.813683 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.813048 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/134d75e6-f2f5-4264-9f35-c231e06bdbb9-multus-daemon-config\") pod \"multus-4xrm5\" (UID: \"134d75e6-f2f5-4264-9f35-c231e06bdbb9\") " pod="openshift-multus/multus-4xrm5" Jun 15 01:44:38.813683 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.813062 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9612108c-f217-4efc-9af1-9500c72da5e3-etc-sysconfig\") pod \"tuned-8h7vw\" (UID: \"9612108c-f217-4efc-9af1-9500c72da5e3\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7vw" Jun 15 01:44:38.813683 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.813077 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/64f557b0-78d8-45a6-9f54-756417231d6c-host-slash\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.813683 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.813109 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/64f557b0-78d8-45a6-9f54-756417231d6c-run-systemd\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.813683 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.813126 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1719195-2a1b-4c96-abe3-c25bda154d3d-metrics-certs\") pod \"network-metrics-daemon-n4p82\" (UID: \"e1719195-2a1b-4c96-abe3-c25bda154d3d\") " pod="openshift-multus/network-metrics-daemon-n4p82" Jun 15 01:44:38.813683 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.813140 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/134d75e6-f2f5-4264-9f35-c231e06bdbb9-hostroot\") pod \"multus-4xrm5\" (UID: \"134d75e6-f2f5-4264-9f35-c231e06bdbb9\") " pod="openshift-multus/multus-4xrm5" Jun 15 01:44:38.813683 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.813154 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjtjl\" (UniqueName: \"kubernetes.io/projected/029053ce-e237-4fa4-8a82-bc70b7be8faa-kube-api-access-sjtjl\") pod \"aws-ebs-csi-driver-node-v7nhx\" (UID: \"029053ce-e237-4fa4-8a82-bc70b7be8faa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7nhx" Jun 15 01:44:38.813683 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.813171 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/64f557b0-78d8-45a6-9f54-756417231d6c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.813683 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.813187 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/64f557b0-78d8-45a6-9f54-756417231d6c-env-overrides\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.813683 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.813205 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwhfs\" (UniqueName: \"kubernetes.io/projected/e1719195-2a1b-4c96-abe3-c25bda154d3d-kube-api-access-cwhfs\") pod \"network-metrics-daemon-n4p82\" (UID: \"e1719195-2a1b-4c96-abe3-c25bda154d3d\") " pod="openshift-multus/network-metrics-daemon-n4p82" Jun 15 01:44:38.813683 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.813221 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/134d75e6-f2f5-4264-9f35-c231e06bdbb9-multus-cni-dir\") pod \"multus-4xrm5\" (UID: \"134d75e6-f2f5-4264-9f35-c231e06bdbb9\") " pod="openshift-multus/multus-4xrm5" Jun 15 01:44:38.813683 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.813235 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/134d75e6-f2f5-4264-9f35-c231e06bdbb9-host-run-multus-certs\") pod \"multus-4xrm5\" (UID: \"134d75e6-f2f5-4264-9f35-c231e06bdbb9\") " pod="openshift-multus/multus-4xrm5" Jun 15 01:44:38.813683 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.813250 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9612108c-f217-4efc-9af1-9500c72da5e3-etc-systemd\") pod \"tuned-8h7vw\" (UID: \"9612108c-f217-4efc-9af1-9500c72da5e3\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7vw" Jun 15 01:44:38.813683 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.813264 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9612108c-f217-4efc-9af1-9500c72da5e3-run\") pod \"tuned-8h7vw\" (UID: \"9612108c-f217-4efc-9af1-9500c72da5e3\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7vw" Jun 15 01:44:38.814188 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.813280 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5ftj\" (UniqueName: \"kubernetes.io/projected/3cf2d35f-dae7-42c6-8f35-5b0ccb5e4750-kube-api-access-c5ftj\") pod \"node-resolver-pmkl7\" (UID: \"3cf2d35f-dae7-42c6-8f35-5b0ccb5e4750\") " pod="openshift-dns/node-resolver-pmkl7" Jun 15 01:44:38.814188 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.813297 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/029053ce-e237-4fa4-8a82-bc70b7be8faa-registration-dir\") pod \"aws-ebs-csi-driver-node-v7nhx\" (UID: \"029053ce-e237-4fa4-8a82-bc70b7be8faa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7nhx" Jun 15 01:44:38.814188 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.813312 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/64f557b0-78d8-45a6-9f54-756417231d6c-log-socket\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.814188 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.813336 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/134d75e6-f2f5-4264-9f35-c231e06bdbb9-host-run-netns\") pod \"multus-4xrm5\" (UID: \"134d75e6-f2f5-4264-9f35-c231e06bdbb9\") " pod="openshift-multus/multus-4xrm5" Jun 15 01:44:38.814188 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.813352 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/134d75e6-f2f5-4264-9f35-c231e06bdbb9-host-var-lib-cni-multus\") pod \"multus-4xrm5\" (UID: \"134d75e6-f2f5-4264-9f35-c231e06bdbb9\") " pod="openshift-multus/multus-4xrm5" Jun 15 01:44:38.814188 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.813366 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9612108c-f217-4efc-9af1-9500c72da5e3-etc-kubernetes\") pod \"tuned-8h7vw\" (UID: \"9612108c-f217-4efc-9af1-9500c72da5e3\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7vw" Jun 15 01:44:38.814188 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.813382 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9a743adf-5ee6-4357-9206-1ccada80d911-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8r9kz\" (UID: \"9a743adf-5ee6-4357-9206-1ccada80d911\") " pod="openshift-multus/multus-additional-cni-plugins-8r9kz" Jun 15 01:44:38.814188 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.813399 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/64f557b0-78d8-45a6-9f54-756417231d6c-ovn-node-metrics-cert\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.814188 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.813414 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cbbc74f1-9650-4453-955f-fa40d2adb7bc-host-slash\") pod \"iptables-alerter-7l5ft\" (UID: \"cbbc74f1-9650-4453-955f-fa40d2adb7bc\") " pod="openshift-network-operator/iptables-alerter-7l5ft" Jun 15 01:44:38.814188 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.813433 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxzg8\" (UniqueName: \"kubernetes.io/projected/cbbc74f1-9650-4453-955f-fa40d2adb7bc-kube-api-access-gxzg8\") pod \"iptables-alerter-7l5ft\" (UID: \"cbbc74f1-9650-4453-955f-fa40d2adb7bc\") " pod="openshift-network-operator/iptables-alerter-7l5ft" Jun 15 01:44:38.814188 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.813448 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/134d75e6-f2f5-4264-9f35-c231e06bdbb9-os-release\") pod \"multus-4xrm5\" (UID: \"134d75e6-f2f5-4264-9f35-c231e06bdbb9\") " pod="openshift-multus/multus-4xrm5" Jun 15 01:44:38.814188 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.813463 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/134d75e6-f2f5-4264-9f35-c231e06bdbb9-host-run-k8s-cni-cncf-io\") pod \"multus-4xrm5\" (UID: \"134d75e6-f2f5-4264-9f35-c231e06bdbb9\") " pod="openshift-multus/multus-4xrm5" Jun 15 01:44:38.814188 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.813479 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a6368c58-4355-4d1c-b621-75c5bf0e7394-original-pull-secret\") pod \"global-pull-secret-syncer-6272r\" (UID: \"a6368c58-4355-4d1c-b621-75c5bf0e7394\") " pod="kube-system/global-pull-secret-syncer-6272r" Jun 15 01:44:38.814188 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.813494 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cj4mm\" (UniqueName: \"kubernetes.io/projected/17743aa6-c9cb-458c-b3d3-16c6a774171a-kube-api-access-cj4mm\") pod \"node-ca-r8dqf\" (UID: \"17743aa6-c9cb-458c-b3d3-16c6a774171a\") " pod="openshift-image-registry/node-ca-r8dqf" Jun 15 01:44:38.814188 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.813509 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9612108c-f217-4efc-9af1-9500c72da5e3-tmp\") pod \"tuned-8h7vw\" (UID: \"9612108c-f217-4efc-9af1-9500c72da5e3\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7vw" Jun 15 01:44:38.814188 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.813524 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/64f557b0-78d8-45a6-9f54-756417231d6c-run-openvswitch\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.814188 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.813541 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92l2k\" (UniqueName: \"kubernetes.io/projected/64f557b0-78d8-45a6-9f54-756417231d6c-kube-api-access-92l2k\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.814930 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.813557 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/134d75e6-f2f5-4264-9f35-c231e06bdbb9-host-var-lib-cni-bin\") pod \"multus-4xrm5\" (UID: \"134d75e6-f2f5-4264-9f35-c231e06bdbb9\") " pod="openshift-multus/multus-4xrm5" Jun 15 01:44:38.814930 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.813574 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9a743adf-5ee6-4357-9206-1ccada80d911-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8r9kz\" (UID: \"9a743adf-5ee6-4357-9206-1ccada80d911\") " pod="openshift-multus/multus-additional-cni-plugins-8r9kz" Jun 15 01:44:38.814930 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.813588 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/134d75e6-f2f5-4264-9f35-c231e06bdbb9-etc-kubernetes\") pod \"multus-4xrm5\" (UID: \"134d75e6-f2f5-4264-9f35-c231e06bdbb9\") " pod="openshift-multus/multus-4xrm5" Jun 15 01:44:38.814930 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.813602 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/17743aa6-c9cb-458c-b3d3-16c6a774171a-serviceca\") pod \"node-ca-r8dqf\" (UID: \"17743aa6-c9cb-458c-b3d3-16c6a774171a\") " pod="openshift-image-registry/node-ca-r8dqf" Jun 15 01:44:38.814930 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.813617 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9612108c-f217-4efc-9af1-9500c72da5e3-sys\") pod \"tuned-8h7vw\" (UID: \"9612108c-f217-4efc-9af1-9500c72da5e3\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7vw" Jun 15 01:44:38.814930 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.813632 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9612108c-f217-4efc-9af1-9500c72da5e3-var-lib-kubelet\") pod \"tuned-8h7vw\" (UID: \"9612108c-f217-4efc-9af1-9500c72da5e3\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7vw" Jun 15 01:44:38.814930 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.813653 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9a743adf-5ee6-4357-9206-1ccada80d911-system-cni-dir\") pod \"multus-additional-cni-plugins-8r9kz\" (UID: \"9a743adf-5ee6-4357-9206-1ccada80d911\") " pod="openshift-multus/multus-additional-cni-plugins-8r9kz" Jun 15 01:44:38.814930 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.813738 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/029053ce-e237-4fa4-8a82-bc70b7be8faa-kubelet-dir\") pod \"aws-ebs-csi-driver-node-v7nhx\" (UID: \"029053ce-e237-4fa4-8a82-bc70b7be8faa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7nhx" Jun 15 01:44:38.814930 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.813773 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/029053ce-e237-4fa4-8a82-bc70b7be8faa-etc-selinux\") pod \"aws-ebs-csi-driver-node-v7nhx\" (UID: \"029053ce-e237-4fa4-8a82-bc70b7be8faa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7nhx" Jun 15 01:44:38.814930 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.814210 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cbbc74f1-9650-4453-955f-fa40d2adb7bc-iptables-alerter-script\") pod \"iptables-alerter-7l5ft\" (UID: \"cbbc74f1-9650-4453-955f-fa40d2adb7bc\") " pod="openshift-network-operator/iptables-alerter-7l5ft" Jun 15 01:44:38.814930 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.814231 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/029053ce-e237-4fa4-8a82-bc70b7be8faa-sys-fs\") pod \"aws-ebs-csi-driver-node-v7nhx\" (UID: \"029053ce-e237-4fa4-8a82-bc70b7be8faa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7nhx" Jun 15 01:44:38.814930 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.814259 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/64f557b0-78d8-45a6-9f54-756417231d6c-systemd-units\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.814930 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.814280 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/64f557b0-78d8-45a6-9f54-756417231d6c-host-cni-bin\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.814930 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.814315 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cbbc74f1-9650-4453-955f-fa40d2adb7bc-host-slash\") pod \"iptables-alerter-7l5ft\" (UID: \"cbbc74f1-9650-4453-955f-fa40d2adb7bc\") " pod="openshift-network-operator/iptables-alerter-7l5ft" Jun 15 01:44:38.814930 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.814326 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/029053ce-e237-4fa4-8a82-bc70b7be8faa-socket-dir\") pod \"aws-ebs-csi-driver-node-v7nhx\" (UID: \"029053ce-e237-4fa4-8a82-bc70b7be8faa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7nhx" Jun 15 01:44:38.814930 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.814370 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3cf2d35f-dae7-42c6-8f35-5b0ccb5e4750-hosts-file\") pod \"node-resolver-pmkl7\" (UID: \"3cf2d35f-dae7-42c6-8f35-5b0ccb5e4750\") " pod="openshift-dns/node-resolver-pmkl7" Jun 15 01:44:38.814930 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.814403 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/64f557b0-78d8-45a6-9f54-756417231d6c-run-openvswitch\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.815647 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.814425 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/029053ce-e237-4fa4-8a82-bc70b7be8faa-device-dir\") pod \"aws-ebs-csi-driver-node-v7nhx\" (UID: \"029053ce-e237-4fa4-8a82-bc70b7be8faa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7nhx" Jun 15 01:44:38.815647 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.814463 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/64f557b0-78d8-45a6-9f54-756417231d6c-host-run-netns\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.815647 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.814504 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/64f557b0-78d8-45a6-9f54-756417231d6c-node-log\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.815647 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.814507 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3cf2d35f-dae7-42c6-8f35-5b0ccb5e4750-tmp-dir\") pod \"node-resolver-pmkl7\" (UID: \"3cf2d35f-dae7-42c6-8f35-5b0ccb5e4750\") " pod="openshift-dns/node-resolver-pmkl7" Jun 15 01:44:38.815647 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.814576 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/64f557b0-78d8-45a6-9f54-756417231d6c-host-cni-netd\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.815647 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.814616 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/64f557b0-78d8-45a6-9f54-756417231d6c-var-lib-openvswitch\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.815647 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.814719 2578 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 15 01:44:38.815647 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.814816 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/64f557b0-78d8-45a6-9f54-756417231d6c-host-slash\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.815647 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.814860 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/64f557b0-78d8-45a6-9f54-756417231d6c-run-systemd\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.815647 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:38.814939 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 01:44:38.815647 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:38.815005 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1719195-2a1b-4c96-abe3-c25bda154d3d-metrics-certs podName:e1719195-2a1b-4c96-abe3-c25bda154d3d nodeName:}" failed. No retries permitted until 2026-06-15 01:44:39.314985726 +0000 UTC m=+3.028711696 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e1719195-2a1b-4c96-abe3-c25bda154d3d-metrics-certs") pod "network-metrics-daemon-n4p82" (UID: "e1719195-2a1b-4c96-abe3-c25bda154d3d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 01:44:38.815647 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.815150 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/64f557b0-78d8-45a6-9f54-756417231d6c-ovnkube-config\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.815647 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.815329 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/64f557b0-78d8-45a6-9f54-756417231d6c-host-run-ovn-kubernetes\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.815647 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.815377 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/64f557b0-78d8-45a6-9f54-756417231d6c-run-ovn\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.815647 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.815592 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d9f8083b-6138-4b4d-b8b8-d11ca6811a9d-konnectivity-ca\") pod \"konnectivity-agent-tnmwb\" (UID: \"d9f8083b-6138-4b4d-b8b8-d11ca6811a9d\") " pod="kube-system/konnectivity-agent-tnmwb" Jun 15 01:44:38.816341 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.815680 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/64f557b0-78d8-45a6-9f54-756417231d6c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.816341 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.815995 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/64f557b0-78d8-45a6-9f54-756417231d6c-ovnkube-script-lib\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.816341 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.816183 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/64f557b0-78d8-45a6-9f54-756417231d6c-env-overrides\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.816341 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.816236 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/64f557b0-78d8-45a6-9f54-756417231d6c-log-socket\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.816341 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.816287 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/029053ce-e237-4fa4-8a82-bc70b7be8faa-registration-dir\") pod \"aws-ebs-csi-driver-node-v7nhx\" (UID: \"029053ce-e237-4fa4-8a82-bc70b7be8faa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7nhx" Jun 15 01:44:38.816341 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.816307 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/64f557b0-78d8-45a6-9f54-756417231d6c-etc-openvswitch\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.816650 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.816158 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/64f557b0-78d8-45a6-9f54-756417231d6c-host-kubelet\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.818371 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.818353 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/64f557b0-78d8-45a6-9f54-756417231d6c-ovn-node-metrics-cert\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.818502 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.818393 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d9f8083b-6138-4b4d-b8b8-d11ca6811a9d-agent-certs\") pod \"konnectivity-agent-tnmwb\" (UID: \"d9f8083b-6138-4b4d-b8b8-d11ca6811a9d\") " pod="kube-system/konnectivity-agent-tnmwb" Jun 15 01:44:38.829648 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.829582 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwhfs\" (UniqueName: \"kubernetes.io/projected/e1719195-2a1b-4c96-abe3-c25bda154d3d-kube-api-access-cwhfs\") pod \"network-metrics-daemon-n4p82\" (UID: \"e1719195-2a1b-4c96-abe3-c25bda154d3d\") " pod="openshift-multus/network-metrics-daemon-n4p82" Jun 15 01:44:38.830663 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.830639 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjtjl\" (UniqueName: \"kubernetes.io/projected/029053ce-e237-4fa4-8a82-bc70b7be8faa-kube-api-access-sjtjl\") pod \"aws-ebs-csi-driver-node-v7nhx\" (UID: \"029053ce-e237-4fa4-8a82-bc70b7be8faa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7nhx" Jun 15 01:44:38.830860 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.830838 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5ftj\" (UniqueName: \"kubernetes.io/projected/3cf2d35f-dae7-42c6-8f35-5b0ccb5e4750-kube-api-access-c5ftj\") pod \"node-resolver-pmkl7\" (UID: \"3cf2d35f-dae7-42c6-8f35-5b0ccb5e4750\") " pod="openshift-dns/node-resolver-pmkl7" Jun 15 01:44:38.837692 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.837670 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxzg8\" (UniqueName: \"kubernetes.io/projected/cbbc74f1-9650-4453-955f-fa40d2adb7bc-kube-api-access-gxzg8\") pod \"iptables-alerter-7l5ft\" (UID: \"cbbc74f1-9650-4453-955f-fa40d2adb7bc\") " pod="openshift-network-operator/iptables-alerter-7l5ft" Jun 15 01:44:38.838063 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.838040 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-92l2k\" (UniqueName: \"kubernetes.io/projected/64f557b0-78d8-45a6-9f54-756417231d6c-kube-api-access-92l2k\") pod \"ovnkube-node-cf286\" (UID: \"64f557b0-78d8-45a6-9f54-756417231d6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:38.915118 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.914062 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/134d75e6-f2f5-4264-9f35-c231e06bdbb9-host-var-lib-kubelet\") pod \"multus-4xrm5\" (UID: \"134d75e6-f2f5-4264-9f35-c231e06bdbb9\") " pod="openshift-multus/multus-4xrm5" Jun 15 01:44:38.915118 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.914126 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/134d75e6-f2f5-4264-9f35-c231e06bdbb9-multus-socket-dir-parent\") pod \"multus-4xrm5\" (UID: \"134d75e6-f2f5-4264-9f35-c231e06bdbb9\") " pod="openshift-multus/multus-4xrm5" Jun 15 01:44:38.915118 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.914156 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/134d75e6-f2f5-4264-9f35-c231e06bdbb9-cni-binary-copy\") pod \"multus-4xrm5\" (UID: \"134d75e6-f2f5-4264-9f35-c231e06bdbb9\") " pod="openshift-multus/multus-4xrm5" Jun 15 01:44:38.915118 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.914161 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/134d75e6-f2f5-4264-9f35-c231e06bdbb9-host-var-lib-kubelet\") pod \"multus-4xrm5\" (UID: \"134d75e6-f2f5-4264-9f35-c231e06bdbb9\") " pod="openshift-multus/multus-4xrm5" Jun 15 01:44:38.915118 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.914178 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9612108c-f217-4efc-9af1-9500c72da5e3-etc-sysctl-conf\") pod \"tuned-8h7vw\" (UID: \"9612108c-f217-4efc-9af1-9500c72da5e3\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7vw" Jun 15 01:44:38.915118 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.914194 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcmtt\" (UniqueName: \"kubernetes.io/projected/9612108c-f217-4efc-9af1-9500c72da5e3-kube-api-access-mcmtt\") pod \"tuned-8h7vw\" (UID: \"9612108c-f217-4efc-9af1-9500c72da5e3\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7vw" Jun 15 01:44:38.915118 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.914209 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9a743adf-5ee6-4357-9206-1ccada80d911-cni-binary-copy\") pod \"multus-additional-cni-plugins-8r9kz\" (UID: \"9a743adf-5ee6-4357-9206-1ccada80d911\") " pod="openshift-multus/multus-additional-cni-plugins-8r9kz" Jun 15 01:44:38.915118 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.914233 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/134d75e6-f2f5-4264-9f35-c231e06bdbb9-multus-socket-dir-parent\") pod \"multus-4xrm5\" (UID: \"134d75e6-f2f5-4264-9f35-c231e06bdbb9\") " pod="openshift-multus/multus-4xrm5" Jun 15 01:44:38.915118 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.914233 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ts659\" (UniqueName: \"kubernetes.io/projected/9a743adf-5ee6-4357-9206-1ccada80d911-kube-api-access-ts659\") pod \"multus-additional-cni-plugins-8r9kz\" (UID: \"9a743adf-5ee6-4357-9206-1ccada80d911\") " pod="openshift-multus/multus-additional-cni-plugins-8r9kz" Jun 15 01:44:38.915118 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.914270 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/134d75e6-f2f5-4264-9f35-c231e06bdbb9-system-cni-dir\") pod \"multus-4xrm5\" (UID: \"134d75e6-f2f5-4264-9f35-c231e06bdbb9\") " pod="openshift-multus/multus-4xrm5" Jun 15 01:44:38.915118 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.914290 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/134d75e6-f2f5-4264-9f35-c231e06bdbb9-multus-conf-dir\") pod \"multus-4xrm5\" (UID: \"134d75e6-f2f5-4264-9f35-c231e06bdbb9\") " pod="openshift-multus/multus-4xrm5" Jun 15 01:44:38.915118 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.914316 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a6368c58-4355-4d1c-b621-75c5bf0e7394-kubelet-config\") pod \"global-pull-secret-syncer-6272r\" (UID: \"a6368c58-4355-4d1c-b621-75c5bf0e7394\") " pod="kube-system/global-pull-secret-syncer-6272r" Jun 15 01:44:38.915118 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.914340 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9612108c-f217-4efc-9af1-9500c72da5e3-etc-modprobe-d\") pod \"tuned-8h7vw\" (UID: \"9612108c-f217-4efc-9af1-9500c72da5e3\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7vw" Jun 15 01:44:38.915118 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.914362 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/134d75e6-f2f5-4264-9f35-c231e06bdbb9-cnibin\") pod \"multus-4xrm5\" (UID: \"134d75e6-f2f5-4264-9f35-c231e06bdbb9\") " pod="openshift-multus/multus-4xrm5" Jun 15 01:44:38.915118 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.914375 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9612108c-f217-4efc-9af1-9500c72da5e3-host\") pod \"tuned-8h7vw\" (UID: \"9612108c-f217-4efc-9af1-9500c72da5e3\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7vw" Jun 15 01:44:38.915118 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.914392 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9612108c-f217-4efc-9af1-9500c72da5e3-etc-tuned\") pod \"tuned-8h7vw\" (UID: \"9612108c-f217-4efc-9af1-9500c72da5e3\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7vw" Jun 15 01:44:38.915118 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.914416 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxzqh\" (UniqueName: \"kubernetes.io/projected/134d75e6-f2f5-4264-9f35-c231e06bdbb9-kube-api-access-wxzqh\") pod \"multus-4xrm5\" (UID: \"134d75e6-f2f5-4264-9f35-c231e06bdbb9\") " pod="openshift-multus/multus-4xrm5" Jun 15 01:44:38.915803 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.914432 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnvlx\" (UniqueName: \"kubernetes.io/projected/a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1-kube-api-access-fnvlx\") pod \"network-check-target-f2kkk\" (UID: \"a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1\") " pod="openshift-network-diagnostics/network-check-target-f2kkk" Jun 15 01:44:38.915803 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.914457 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/134d75e6-f2f5-4264-9f35-c231e06bdbb9-multus-daemon-config\") pod \"multus-4xrm5\" (UID: \"134d75e6-f2f5-4264-9f35-c231e06bdbb9\") " pod="openshift-multus/multus-4xrm5" Jun 15 01:44:38.915803 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.914481 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9612108c-f217-4efc-9af1-9500c72da5e3-etc-sysconfig\") pod \"tuned-8h7vw\" (UID: \"9612108c-f217-4efc-9af1-9500c72da5e3\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7vw" Jun 15 01:44:38.915803 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.914511 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/134d75e6-f2f5-4264-9f35-c231e06bdbb9-hostroot\") pod \"multus-4xrm5\" (UID: \"134d75e6-f2f5-4264-9f35-c231e06bdbb9\") " pod="openshift-multus/multus-4xrm5" Jun 15 01:44:38.915803 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.914529 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/134d75e6-f2f5-4264-9f35-c231e06bdbb9-multus-cni-dir\") pod \"multus-4xrm5\" (UID: \"134d75e6-f2f5-4264-9f35-c231e06bdbb9\") " pod="openshift-multus/multus-4xrm5" Jun 15 01:44:38.915803 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.914546 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/134d75e6-f2f5-4264-9f35-c231e06bdbb9-host-run-multus-certs\") pod \"multus-4xrm5\" (UID: \"134d75e6-f2f5-4264-9f35-c231e06bdbb9\") " pod="openshift-multus/multus-4xrm5" Jun 15 01:44:38.915803 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.914569 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9612108c-f217-4efc-9af1-9500c72da5e3-etc-systemd\") pod \"tuned-8h7vw\" (UID: \"9612108c-f217-4efc-9af1-9500c72da5e3\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7vw" Jun 15 01:44:38.915803 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.914583 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9612108c-f217-4efc-9af1-9500c72da5e3-run\") pod \"tuned-8h7vw\" (UID: \"9612108c-f217-4efc-9af1-9500c72da5e3\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7vw" Jun 15 01:44:38.915803 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.914598 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/134d75e6-f2f5-4264-9f35-c231e06bdbb9-host-run-netns\") pod \"multus-4xrm5\" (UID: \"134d75e6-f2f5-4264-9f35-c231e06bdbb9\") " pod="openshift-multus/multus-4xrm5" Jun 15 01:44:38.915803 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.914612 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/134d75e6-f2f5-4264-9f35-c231e06bdbb9-host-var-lib-cni-multus\") pod \"multus-4xrm5\" (UID: \"134d75e6-f2f5-4264-9f35-c231e06bdbb9\") " pod="openshift-multus/multus-4xrm5" Jun 15 01:44:38.915803 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.914629 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9612108c-f217-4efc-9af1-9500c72da5e3-etc-kubernetes\") pod \"tuned-8h7vw\" (UID: \"9612108c-f217-4efc-9af1-9500c72da5e3\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7vw" Jun 15 01:44:38.915803 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.914655 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9a743adf-5ee6-4357-9206-1ccada80d911-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8r9kz\" (UID: \"9a743adf-5ee6-4357-9206-1ccada80d911\") " pod="openshift-multus/multus-additional-cni-plugins-8r9kz" Jun 15 01:44:38.915803 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.914682 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/134d75e6-f2f5-4264-9f35-c231e06bdbb9-os-release\") pod \"multus-4xrm5\" (UID: \"134d75e6-f2f5-4264-9f35-c231e06bdbb9\") " pod="openshift-multus/multus-4xrm5" Jun 15 01:44:38.915803 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.914704 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/134d75e6-f2f5-4264-9f35-c231e06bdbb9-host-run-k8s-cni-cncf-io\") pod \"multus-4xrm5\" (UID: \"134d75e6-f2f5-4264-9f35-c231e06bdbb9\") " pod="openshift-multus/multus-4xrm5" Jun 15 01:44:38.915803 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.914719 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a6368c58-4355-4d1c-b621-75c5bf0e7394-original-pull-secret\") pod \"global-pull-secret-syncer-6272r\" (UID: \"a6368c58-4355-4d1c-b621-75c5bf0e7394\") " pod="kube-system/global-pull-secret-syncer-6272r" Jun 15 01:44:38.915803 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.914741 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cj4mm\" (UniqueName: \"kubernetes.io/projected/17743aa6-c9cb-458c-b3d3-16c6a774171a-kube-api-access-cj4mm\") pod \"node-ca-r8dqf\" (UID: \"17743aa6-c9cb-458c-b3d3-16c6a774171a\") " pod="openshift-image-registry/node-ca-r8dqf" Jun 15 01:44:38.915803 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.914756 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9612108c-f217-4efc-9af1-9500c72da5e3-tmp\") pod \"tuned-8h7vw\" (UID: \"9612108c-f217-4efc-9af1-9500c72da5e3\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7vw" Jun 15 01:44:38.915803 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.914771 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/134d75e6-f2f5-4264-9f35-c231e06bdbb9-host-var-lib-cni-bin\") pod \"multus-4xrm5\" (UID: \"134d75e6-f2f5-4264-9f35-c231e06bdbb9\") " pod="openshift-multus/multus-4xrm5" Jun 15 01:44:38.916921 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.914785 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9a743adf-5ee6-4357-9206-1ccada80d911-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8r9kz\" (UID: \"9a743adf-5ee6-4357-9206-1ccada80d911\") " pod="openshift-multus/multus-additional-cni-plugins-8r9kz" Jun 15 01:44:38.916921 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.914803 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/134d75e6-f2f5-4264-9f35-c231e06bdbb9-etc-kubernetes\") pod \"multus-4xrm5\" (UID: \"134d75e6-f2f5-4264-9f35-c231e06bdbb9\") " pod="openshift-multus/multus-4xrm5" Jun 15 01:44:38.916921 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.914826 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/17743aa6-c9cb-458c-b3d3-16c6a774171a-serviceca\") pod \"node-ca-r8dqf\" (UID: \"17743aa6-c9cb-458c-b3d3-16c6a774171a\") " pod="openshift-image-registry/node-ca-r8dqf" Jun 15 01:44:38.916921 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.914846 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9612108c-f217-4efc-9af1-9500c72da5e3-sys\") pod \"tuned-8h7vw\" (UID: \"9612108c-f217-4efc-9af1-9500c72da5e3\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7vw" Jun 15 01:44:38.916921 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.914860 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/134d75e6-f2f5-4264-9f35-c231e06bdbb9-cni-binary-copy\") pod \"multus-4xrm5\" (UID: \"134d75e6-f2f5-4264-9f35-c231e06bdbb9\") " pod="openshift-multus/multus-4xrm5" Jun 15 01:44:38.916921 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.914868 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9612108c-f217-4efc-9af1-9500c72da5e3-var-lib-kubelet\") pod \"tuned-8h7vw\" (UID: \"9612108c-f217-4efc-9af1-9500c72da5e3\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7vw" Jun 15 01:44:38.916921 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.914890 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9a743adf-5ee6-4357-9206-1ccada80d911-system-cni-dir\") pod \"multus-additional-cni-plugins-8r9kz\" (UID: \"9a743adf-5ee6-4357-9206-1ccada80d911\") " pod="openshift-multus/multus-additional-cni-plugins-8r9kz" Jun 15 01:44:38.916921 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.914912 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9a743adf-5ee6-4357-9206-1ccada80d911-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8r9kz\" (UID: \"9a743adf-5ee6-4357-9206-1ccada80d911\") " pod="openshift-multus/multus-additional-cni-plugins-8r9kz" Jun 15 01:44:38.916921 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.914937 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a6368c58-4355-4d1c-b621-75c5bf0e7394-dbus\") pod \"global-pull-secret-syncer-6272r\" (UID: \"a6368c58-4355-4d1c-b621-75c5bf0e7394\") " pod="kube-system/global-pull-secret-syncer-6272r" Jun 15 01:44:38.916921 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.914952 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9a743adf-5ee6-4357-9206-1ccada80d911-os-release\") pod \"multus-additional-cni-plugins-8r9kz\" (UID: \"9a743adf-5ee6-4357-9206-1ccada80d911\") " pod="openshift-multus/multus-additional-cni-plugins-8r9kz" Jun 15 01:44:38.916921 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.914965 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9612108c-f217-4efc-9af1-9500c72da5e3-etc-sysctl-conf\") pod \"tuned-8h7vw\" (UID: \"9612108c-f217-4efc-9af1-9500c72da5e3\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7vw" Jun 15 01:44:38.916921 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.914968 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9612108c-f217-4efc-9af1-9500c72da5e3-etc-sysctl-d\") pod \"tuned-8h7vw\" (UID: \"9612108c-f217-4efc-9af1-9500c72da5e3\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7vw" Jun 15 01:44:38.916921 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.914990 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9612108c-f217-4efc-9af1-9500c72da5e3-lib-modules\") pod \"tuned-8h7vw\" (UID: \"9612108c-f217-4efc-9af1-9500c72da5e3\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7vw" Jun 15 01:44:38.916921 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.915008 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9a743adf-5ee6-4357-9206-1ccada80d911-cnibin\") pod \"multus-additional-cni-plugins-8r9kz\" (UID: \"9a743adf-5ee6-4357-9206-1ccada80d911\") " pod="openshift-multus/multus-additional-cni-plugins-8r9kz" Jun 15 01:44:38.916921 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.915031 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/17743aa6-c9cb-458c-b3d3-16c6a774171a-host\") pod \"node-ca-r8dqf\" (UID: \"17743aa6-c9cb-458c-b3d3-16c6a774171a\") " pod="openshift-image-registry/node-ca-r8dqf" Jun 15 01:44:38.916921 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.915107 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/17743aa6-c9cb-458c-b3d3-16c6a774171a-host\") pod \"node-ca-r8dqf\" (UID: \"17743aa6-c9cb-458c-b3d3-16c6a774171a\") " pod="openshift-image-registry/node-ca-r8dqf" Jun 15 01:44:38.916921 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.915157 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/134d75e6-f2f5-4264-9f35-c231e06bdbb9-system-cni-dir\") pod \"multus-4xrm5\" (UID: \"134d75e6-f2f5-4264-9f35-c231e06bdbb9\") " pod="openshift-multus/multus-4xrm5" Jun 15 01:44:38.916921 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.915204 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/134d75e6-f2f5-4264-9f35-c231e06bdbb9-cnibin\") pod \"multus-4xrm5\" (UID: \"134d75e6-f2f5-4264-9f35-c231e06bdbb9\") " pod="openshift-multus/multus-4xrm5" Jun 15 01:44:38.917774 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.915224 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/134d75e6-f2f5-4264-9f35-c231e06bdbb9-os-release\") pod \"multus-4xrm5\" (UID: \"134d75e6-f2f5-4264-9f35-c231e06bdbb9\") " pod="openshift-multus/multus-4xrm5" Jun 15 01:44:38.917774 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.915253 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9612108c-f217-4efc-9af1-9500c72da5e3-host\") pod \"tuned-8h7vw\" (UID: \"9612108c-f217-4efc-9af1-9500c72da5e3\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7vw" Jun 15 01:44:38.917774 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.915263 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/134d75e6-f2f5-4264-9f35-c231e06bdbb9-host-run-k8s-cni-cncf-io\") pod \"multus-4xrm5\" (UID: \"134d75e6-f2f5-4264-9f35-c231e06bdbb9\") " pod="openshift-multus/multus-4xrm5" Jun 15 01:44:38.917774 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:38.915345 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 15 01:44:38.917774 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.915370 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9612108c-f217-4efc-9af1-9500c72da5e3-etc-sysconfig\") pod \"tuned-8h7vw\" (UID: \"9612108c-f217-4efc-9af1-9500c72da5e3\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7vw" Jun 15 01:44:38.917774 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:38.915396 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6368c58-4355-4d1c-b621-75c5bf0e7394-original-pull-secret podName:a6368c58-4355-4d1c-b621-75c5bf0e7394 nodeName:}" failed. No retries permitted until 2026-06-15 01:44:39.415380335 +0000 UTC m=+3.129106309 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a6368c58-4355-4d1c-b621-75c5bf0e7394-original-pull-secret") pod "global-pull-secret-syncer-6272r" (UID: "a6368c58-4355-4d1c-b621-75c5bf0e7394") : object "kube-system"/"original-pull-secret" not registered Jun 15 01:44:38.917774 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.915402 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/134d75e6-f2f5-4264-9f35-c231e06bdbb9-hostroot\") pod \"multus-4xrm5\" (UID: \"134d75e6-f2f5-4264-9f35-c231e06bdbb9\") " pod="openshift-multus/multus-4xrm5" Jun 15 01:44:38.917774 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.915443 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/134d75e6-f2f5-4264-9f35-c231e06bdbb9-host-run-multus-certs\") pod \"multus-4xrm5\" (UID: \"134d75e6-f2f5-4264-9f35-c231e06bdbb9\") " pod="openshift-multus/multus-4xrm5" Jun 15 01:44:38.917774 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.915493 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9612108c-f217-4efc-9af1-9500c72da5e3-etc-systemd\") pod \"tuned-8h7vw\" (UID: \"9612108c-f217-4efc-9af1-9500c72da5e3\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7vw" Jun 15 01:44:38.917774 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.915536 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9612108c-f217-4efc-9af1-9500c72da5e3-run\") pod \"tuned-8h7vw\" (UID: \"9612108c-f217-4efc-9af1-9500c72da5e3\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7vw" Jun 15 01:44:38.917774 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.915573 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/134d75e6-f2f5-4264-9f35-c231e06bdbb9-host-run-netns\") pod \"multus-4xrm5\" (UID: \"134d75e6-f2f5-4264-9f35-c231e06bdbb9\") " pod="openshift-multus/multus-4xrm5" Jun 15 01:44:38.917774 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.915610 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/134d75e6-f2f5-4264-9f35-c231e06bdbb9-host-var-lib-cni-multus\") pod \"multus-4xrm5\" (UID: \"134d75e6-f2f5-4264-9f35-c231e06bdbb9\") " pod="openshift-multus/multus-4xrm5" Jun 15 01:44:38.917774 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.915655 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9612108c-f217-4efc-9af1-9500c72da5e3-etc-kubernetes\") pod \"tuned-8h7vw\" (UID: \"9612108c-f217-4efc-9af1-9500c72da5e3\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7vw" Jun 15 01:44:38.917774 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.915692 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/134d75e6-f2f5-4264-9f35-c231e06bdbb9-multus-cni-dir\") pod \"multus-4xrm5\" (UID: \"134d75e6-f2f5-4264-9f35-c231e06bdbb9\") " pod="openshift-multus/multus-4xrm5" Jun 15 01:44:38.917774 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.915699 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a6368c58-4355-4d1c-b621-75c5bf0e7394-kubelet-config\") pod \"global-pull-secret-syncer-6272r\" (UID: \"a6368c58-4355-4d1c-b621-75c5bf0e7394\") " pod="kube-system/global-pull-secret-syncer-6272r" Jun 15 01:44:38.917774 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.915717 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9a743adf-5ee6-4357-9206-1ccada80d911-cni-binary-copy\") pod \"multus-additional-cni-plugins-8r9kz\" (UID: \"9a743adf-5ee6-4357-9206-1ccada80d911\") " pod="openshift-multus/multus-additional-cni-plugins-8r9kz" Jun 15 01:44:38.917774 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.915853 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9a743adf-5ee6-4357-9206-1ccada80d911-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8r9kz\" (UID: \"9a743adf-5ee6-4357-9206-1ccada80d911\") " pod="openshift-multus/multus-additional-cni-plugins-8r9kz" Jun 15 01:44:38.917774 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.915907 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/134d75e6-f2f5-4264-9f35-c231e06bdbb9-multus-conf-dir\") pod \"multus-4xrm5\" (UID: \"134d75e6-f2f5-4264-9f35-c231e06bdbb9\") " pod="openshift-multus/multus-4xrm5" Jun 15 01:44:38.918417 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.916003 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9612108c-f217-4efc-9af1-9500c72da5e3-lib-modules\") pod \"tuned-8h7vw\" (UID: \"9612108c-f217-4efc-9af1-9500c72da5e3\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7vw" Jun 15 01:44:38.918417 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.916040 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a6368c58-4355-4d1c-b621-75c5bf0e7394-dbus\") pod \"global-pull-secret-syncer-6272r\" (UID: \"a6368c58-4355-4d1c-b621-75c5bf0e7394\") " pod="kube-system/global-pull-secret-syncer-6272r" Jun 15 01:44:38.918417 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.916072 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9a743adf-5ee6-4357-9206-1ccada80d911-os-release\") pod \"multus-additional-cni-plugins-8r9kz\" (UID: \"9a743adf-5ee6-4357-9206-1ccada80d911\") " pod="openshift-multus/multus-additional-cni-plugins-8r9kz" Jun 15 01:44:38.918417 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.916174 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9612108c-f217-4efc-9af1-9500c72da5e3-etc-sysctl-d\") pod \"tuned-8h7vw\" (UID: \"9612108c-f217-4efc-9af1-9500c72da5e3\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7vw" Jun 15 01:44:38.918417 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.916172 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/134d75e6-f2f5-4264-9f35-c231e06bdbb9-multus-daemon-config\") pod \"multus-4xrm5\" (UID: \"134d75e6-f2f5-4264-9f35-c231e06bdbb9\") " pod="openshift-multus/multus-4xrm5" Jun 15 01:44:38.918417 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.916241 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/134d75e6-f2f5-4264-9f35-c231e06bdbb9-host-var-lib-cni-bin\") pod \"multus-4xrm5\" (UID: \"134d75e6-f2f5-4264-9f35-c231e06bdbb9\") " pod="openshift-multus/multus-4xrm5" Jun 15 01:44:38.918417 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.916252 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9612108c-f217-4efc-9af1-9500c72da5e3-var-lib-kubelet\") pod \"tuned-8h7vw\" (UID: \"9612108c-f217-4efc-9af1-9500c72da5e3\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7vw" Jun 15 01:44:38.918417 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.916316 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9a743adf-5ee6-4357-9206-1ccada80d911-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8r9kz\" (UID: \"9a743adf-5ee6-4357-9206-1ccada80d911\") " pod="openshift-multus/multus-additional-cni-plugins-8r9kz" Jun 15 01:44:38.918417 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.916368 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9a743adf-5ee6-4357-9206-1ccada80d911-system-cni-dir\") pod \"multus-additional-cni-plugins-8r9kz\" (UID: \"9a743adf-5ee6-4357-9206-1ccada80d911\") " pod="openshift-multus/multus-additional-cni-plugins-8r9kz" Jun 15 01:44:38.918417 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.916564 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9a743adf-5ee6-4357-9206-1ccada80d911-cnibin\") pod \"multus-additional-cni-plugins-8r9kz\" (UID: \"9a743adf-5ee6-4357-9206-1ccada80d911\") " pod="openshift-multus/multus-additional-cni-plugins-8r9kz" Jun 15 01:44:38.918417 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.915348 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9612108c-f217-4efc-9af1-9500c72da5e3-etc-modprobe-d\") pod \"tuned-8h7vw\" (UID: \"9612108c-f217-4efc-9af1-9500c72da5e3\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7vw" Jun 15 01:44:38.918417 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.916598 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9a743adf-5ee6-4357-9206-1ccada80d911-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8r9kz\" (UID: \"9a743adf-5ee6-4357-9206-1ccada80d911\") " pod="openshift-multus/multus-additional-cni-plugins-8r9kz" Jun 15 01:44:38.918417 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.916617 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/17743aa6-c9cb-458c-b3d3-16c6a774171a-serviceca\") pod \"node-ca-r8dqf\" (UID: \"17743aa6-c9cb-458c-b3d3-16c6a774171a\") " pod="openshift-image-registry/node-ca-r8dqf" Jun 15 01:44:38.918417 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.916634 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/134d75e6-f2f5-4264-9f35-c231e06bdbb9-etc-kubernetes\") pod \"multus-4xrm5\" (UID: \"134d75e6-f2f5-4264-9f35-c231e06bdbb9\") " pod="openshift-multus/multus-4xrm5" Jun 15 01:44:38.918417 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.916651 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9612108c-f217-4efc-9af1-9500c72da5e3-sys\") pod \"tuned-8h7vw\" (UID: \"9612108c-f217-4efc-9af1-9500c72da5e3\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7vw" Jun 15 01:44:38.918417 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.917762 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9612108c-f217-4efc-9af1-9500c72da5e3-etc-tuned\") pod \"tuned-8h7vw\" (UID: \"9612108c-f217-4efc-9af1-9500c72da5e3\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7vw" Jun 15 01:44:38.918417 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.917955 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9612108c-f217-4efc-9af1-9500c72da5e3-tmp\") pod \"tuned-8h7vw\" (UID: \"9612108c-f217-4efc-9af1-9500c72da5e3\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7vw" Jun 15 01:44:38.924994 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.924971 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ts659\" (UniqueName: \"kubernetes.io/projected/9a743adf-5ee6-4357-9206-1ccada80d911-kube-api-access-ts659\") pod \"multus-additional-cni-plugins-8r9kz\" (UID: \"9a743adf-5ee6-4357-9206-1ccada80d911\") " pod="openshift-multus/multus-additional-cni-plugins-8r9kz" Jun 15 01:44:38.925906 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:38.925886 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 15 01:44:38.926013 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:38.925913 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 15 01:44:38.926013 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:38.925928 2578 projected.go:196] Error preparing data for projected volume kube-api-access-fnvlx for pod openshift-network-diagnostics/network-check-target-f2kkk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 01:44:38.926013 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:38.925998 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1-kube-api-access-fnvlx podName:a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1 nodeName:}" failed. No retries permitted until 2026-06-15 01:44:39.425981798 +0000 UTC m=+3.139707764 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-fnvlx" (UniqueName: "kubernetes.io/projected/a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1-kube-api-access-fnvlx") pod "network-check-target-f2kkk" (UID: "a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 01:44:38.926689 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.926659 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cj4mm\" (UniqueName: \"kubernetes.io/projected/17743aa6-c9cb-458c-b3d3-16c6a774171a-kube-api-access-cj4mm\") pod \"node-ca-r8dqf\" (UID: \"17743aa6-c9cb-458c-b3d3-16c6a774171a\") " pod="openshift-image-registry/node-ca-r8dqf" Jun 15 01:44:38.928311 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.928287 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcmtt\" (UniqueName: \"kubernetes.io/projected/9612108c-f217-4efc-9af1-9500c72da5e3-kube-api-access-mcmtt\") pod \"tuned-8h7vw\" (UID: \"9612108c-f217-4efc-9af1-9500c72da5e3\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7vw" Jun 15 01:44:38.928400 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.928338 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxzqh\" (UniqueName: \"kubernetes.io/projected/134d75e6-f2f5-4264-9f35-c231e06bdbb9-kube-api-access-wxzqh\") pod \"multus-4xrm5\" (UID: \"134d75e6-f2f5-4264-9f35-c231e06bdbb9\") " pod="openshift-multus/multus-4xrm5" Jun 15 01:44:38.995239 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:38.995209 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7nhx" Jun 15 01:44:39.005272 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:39.005249 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:44:39.018221 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:39.018197 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-tnmwb" Jun 15 01:44:39.025998 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:39.025972 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7l5ft" Jun 15 01:44:39.031796 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:39.031776 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pmkl7" Jun 15 01:44:39.039583 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:39.039550 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-r8dqf" Jun 15 01:44:39.044288 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:39.044269 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4xrm5" Jun 15 01:44:39.049981 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:39.049957 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8h7vw" Jun 15 01:44:39.053619 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:39.053601 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8r9kz" Jun 15 01:44:39.318159 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:39.318128 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1719195-2a1b-4c96-abe3-c25bda154d3d-metrics-certs\") pod \"network-metrics-daemon-n4p82\" (UID: \"e1719195-2a1b-4c96-abe3-c25bda154d3d\") " pod="openshift-multus/network-metrics-daemon-n4p82" Jun 15 01:44:39.318329 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:39.318273 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 01:44:39.318392 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:39.318362 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1719195-2a1b-4c96-abe3-c25bda154d3d-metrics-certs podName:e1719195-2a1b-4c96-abe3-c25bda154d3d nodeName:}" failed. No retries permitted until 2026-06-15 01:44:40.318341718 +0000 UTC m=+4.032067672 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e1719195-2a1b-4c96-abe3-c25bda154d3d-metrics-certs") pod "network-metrics-daemon-n4p82" (UID: "e1719195-2a1b-4c96-abe3-c25bda154d3d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 01:44:39.419151 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:39.419121 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a6368c58-4355-4d1c-b621-75c5bf0e7394-original-pull-secret\") pod \"global-pull-secret-syncer-6272r\" (UID: \"a6368c58-4355-4d1c-b621-75c5bf0e7394\") " pod="kube-system/global-pull-secret-syncer-6272r" Jun 15 01:44:39.419339 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:39.419282 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 15 01:44:39.419401 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:39.419355 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6368c58-4355-4d1c-b621-75c5bf0e7394-original-pull-secret podName:a6368c58-4355-4d1c-b621-75c5bf0e7394 nodeName:}" failed. No retries permitted until 2026-06-15 01:44:40.41933636 +0000 UTC m=+4.133062314 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a6368c58-4355-4d1c-b621-75c5bf0e7394-original-pull-secret") pod "global-pull-secret-syncer-6272r" (UID: "a6368c58-4355-4d1c-b621-75c5bf0e7394") : object "kube-system"/"original-pull-secret" not registered Jun 15 01:44:39.519555 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:39.519526 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnvlx\" (UniqueName: \"kubernetes.io/projected/a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1-kube-api-access-fnvlx\") pod \"network-check-target-f2kkk\" (UID: \"a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1\") " pod="openshift-network-diagnostics/network-check-target-f2kkk" Jun 15 01:44:39.519698 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:39.519673 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 15 01:44:39.519698 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:39.519691 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 15 01:44:39.519776 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:39.519701 2578 projected.go:196] Error preparing data for projected volume kube-api-access-fnvlx for pod openshift-network-diagnostics/network-check-target-f2kkk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 01:44:39.519776 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:39.519750 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1-kube-api-access-fnvlx podName:a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1 nodeName:}" failed. No retries permitted until 2026-06-15 01:44:40.519735314 +0000 UTC m=+4.233461265 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-fnvlx" (UniqueName: "kubernetes.io/projected/a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1-kube-api-access-fnvlx") pod "network-check-target-f2kkk" (UID: "a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 01:44:39.732821 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:39.732736 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-14 01:39:38 +0000 UTC" deadline="2027-11-14 17:03:22.329212556 +0000 UTC" Jun 15 01:44:39.732821 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:39.732770 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12423h18m42.59644595s" Jun 15 01:44:39.913008 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:39.912973 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9612108c_f217_4efc_9af1_9500c72da5e3.slice/crio-77c400a7be28253783a6f4d64578df6fefd3e03eee9e5a08692684a5a4e3748c WatchSource:0}: Error finding container 77c400a7be28253783a6f4d64578df6fefd3e03eee9e5a08692684a5a4e3748c: Status 404 returned error can't find the container with id 77c400a7be28253783a6f4d64578df6fefd3e03eee9e5a08692684a5a4e3748c Jun 15 01:44:39.921664 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:39.921641 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64f557b0_78d8_45a6_9f54_756417231d6c.slice/crio-322d0ed0f2d9f6685f03a49cd6086c6e9bb977e4bab17aeb35a5dff35d576813 WatchSource:0}: Error finding container 322d0ed0f2d9f6685f03a49cd6086c6e9bb977e4bab17aeb35a5dff35d576813: Status 404 returned error can't find the container with id 322d0ed0f2d9f6685f03a49cd6086c6e9bb977e4bab17aeb35a5dff35d576813 Jun 15 01:44:39.922863 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:39.922841 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbbc74f1_9650_4453_955f_fa40d2adb7bc.slice/crio-1c81a6077c728e8f75223e9794ea640955dbda13719b181a37aa6d115cb2210b WatchSource:0}: Error finding container 1c81a6077c728e8f75223e9794ea640955dbda13719b181a37aa6d115cb2210b: Status 404 returned error can't find the container with id 1c81a6077c728e8f75223e9794ea640955dbda13719b181a37aa6d115cb2210b Jun 15 01:44:39.924462 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:39.924443 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a743adf_5ee6_4357_9206_1ccada80d911.slice/crio-7daf21dbe2d054dee05bb38a6a730f6a87b30a0fb8e2feb768c1a20aa3700b0f WatchSource:0}: Error finding container 7daf21dbe2d054dee05bb38a6a730f6a87b30a0fb8e2feb768c1a20aa3700b0f: Status 404 returned error can't find the container with id 7daf21dbe2d054dee05bb38a6a730f6a87b30a0fb8e2feb768c1a20aa3700b0f Jun 15 01:44:39.925116 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:39.924995 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod029053ce_e237_4fa4_8a82_bc70b7be8faa.slice/crio-afcda7e9a3194bf514b27026e6dc1c0ee34e344b3a96b09c442991502c912289 WatchSource:0}: Error finding container afcda7e9a3194bf514b27026e6dc1c0ee34e344b3a96b09c442991502c912289: Status 404 returned error can't find the container with id afcda7e9a3194bf514b27026e6dc1c0ee34e344b3a96b09c442991502c912289 Jun 15 01:44:39.926288 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:39.926266 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17743aa6_c9cb_458c_b3d3_16c6a774171a.slice/crio-7c3ef29e307587b4a01cd8382a4ecb50e47323e8d9f46c035c1e66ebf9615e46 WatchSource:0}: Error finding container 7c3ef29e307587b4a01cd8382a4ecb50e47323e8d9f46c035c1e66ebf9615e46: Status 404 returned error can't find the container with id 7c3ef29e307587b4a01cd8382a4ecb50e47323e8d9f46c035c1e66ebf9615e46 Jun 15 01:44:39.927225 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:39.927203 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cf2d35f_dae7_42c6_8f35_5b0ccb5e4750.slice/crio-3a2e38a70ab8dfcfebc09873b40456841a71aea268d2f4dda54d0fcb28a29abb WatchSource:0}: Error finding container 3a2e38a70ab8dfcfebc09873b40456841a71aea268d2f4dda54d0fcb28a29abb: Status 404 returned error can't find the container with id 3a2e38a70ab8dfcfebc09873b40456841a71aea268d2f4dda54d0fcb28a29abb Jun 15 01:44:39.928459 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:39.928348 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod134d75e6_f2f5_4264_9f35_c231e06bdbb9.slice/crio-ebc34265b6ebf8373056ae9451fa67da1fd240c0fa7536fb9324961bb2126059 WatchSource:0}: Error finding container ebc34265b6ebf8373056ae9451fa67da1fd240c0fa7536fb9324961bb2126059: Status 404 returned error can't find the container with id ebc34265b6ebf8373056ae9451fa67da1fd240c0fa7536fb9324961bb2126059 Jun 15 01:44:39.930531 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:44:39.929680 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9f8083b_6138_4b4d_b8b8_d11ca6811a9d.slice/crio-7db545f5818b19b32ee3adaed62e16523ef1d34ae494ff19f1a3c893dab72693 WatchSource:0}: Error finding container 7db545f5818b19b32ee3adaed62e16523ef1d34ae494ff19f1a3c893dab72693: Status 404 returned error can't find the container with id 7db545f5818b19b32ee3adaed62e16523ef1d34ae494ff19f1a3c893dab72693 Jun 15 01:44:40.324870 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:40.324760 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1719195-2a1b-4c96-abe3-c25bda154d3d-metrics-certs\") pod \"network-metrics-daemon-n4p82\" (UID: \"e1719195-2a1b-4c96-abe3-c25bda154d3d\") " pod="openshift-multus/network-metrics-daemon-n4p82" Jun 15 01:44:40.325214 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:40.324913 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 01:44:40.325214 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:40.324975 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1719195-2a1b-4c96-abe3-c25bda154d3d-metrics-certs podName:e1719195-2a1b-4c96-abe3-c25bda154d3d nodeName:}" failed. No retries permitted until 2026-06-15 01:44:42.324955818 +0000 UTC m=+6.038681771 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e1719195-2a1b-4c96-abe3-c25bda154d3d-metrics-certs") pod "network-metrics-daemon-n4p82" (UID: "e1719195-2a1b-4c96-abe3-c25bda154d3d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 01:44:40.425767 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:40.425286 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a6368c58-4355-4d1c-b621-75c5bf0e7394-original-pull-secret\") pod \"global-pull-secret-syncer-6272r\" (UID: \"a6368c58-4355-4d1c-b621-75c5bf0e7394\") " pod="kube-system/global-pull-secret-syncer-6272r" Jun 15 01:44:40.425767 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:40.425433 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 15 01:44:40.425767 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:40.425490 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6368c58-4355-4d1c-b621-75c5bf0e7394-original-pull-secret podName:a6368c58-4355-4d1c-b621-75c5bf0e7394 nodeName:}" failed. No retries permitted until 2026-06-15 01:44:42.425473157 +0000 UTC m=+6.139199109 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a6368c58-4355-4d1c-b621-75c5bf0e7394-original-pull-secret") pod "global-pull-secret-syncer-6272r" (UID: "a6368c58-4355-4d1c-b621-75c5bf0e7394") : object "kube-system"/"original-pull-secret" not registered Jun 15 01:44:40.525834 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:40.525794 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnvlx\" (UniqueName: \"kubernetes.io/projected/a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1-kube-api-access-fnvlx\") pod \"network-check-target-f2kkk\" (UID: \"a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1\") " pod="openshift-network-diagnostics/network-check-target-f2kkk" Jun 15 01:44:40.526051 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:40.526002 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 15 01:44:40.526051 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:40.526026 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 15 01:44:40.526051 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:40.526039 2578 projected.go:196] Error preparing data for projected volume kube-api-access-fnvlx for pod openshift-network-diagnostics/network-check-target-f2kkk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 01:44:40.526233 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:40.526111 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1-kube-api-access-fnvlx podName:a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1 nodeName:}" failed. No retries permitted until 2026-06-15 01:44:42.526076643 +0000 UTC m=+6.239802595 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-fnvlx" (UniqueName: "kubernetes.io/projected/a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1-kube-api-access-fnvlx") pod "network-check-target-f2kkk" (UID: "a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 01:44:40.733736 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:40.733622 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-14 01:39:38 +0000 UTC" deadline="2028-03-12 15:36:27.8396363 +0000 UTC" Jun 15 01:44:40.733736 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:40.733685 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15277h51m47.105955706s" Jun 15 01:44:40.762924 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:40.762890 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2kkk" Jun 15 01:44:40.763107 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:40.763016 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2kkk" podUID="a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1" Jun 15 01:44:40.763471 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:40.763449 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4p82" Jun 15 01:44:40.763582 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:40.763563 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n4p82" podUID="e1719195-2a1b-4c96-abe3-c25bda154d3d" Jun 15 01:44:40.763648 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:40.763626 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6272r" Jun 15 01:44:40.763706 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:40.763696 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6272r" podUID="a6368c58-4355-4d1c-b621-75c5bf0e7394" Jun 15 01:44:40.781600 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:40.781569 2578 generic.go:356] "Generic (PLEG): container finished" podID="36e239b4d7762116a74696279f22b0af" containerID="dc1df907e7c5c783fc1b25017116ca3cab718ccfcb7114245b2415f8979ec53c" exitCode=0 Jun 15 01:44:40.781737 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:40.781652 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-196.ec2.internal" event={"ID":"36e239b4d7762116a74696279f22b0af","Type":"ContainerDied","Data":"dc1df907e7c5c783fc1b25017116ca3cab718ccfcb7114245b2415f8979ec53c"} Jun 15 01:44:40.796478 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:40.796446 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-196.ec2.internal" event={"ID":"3461aba319c72ca2a9e406f3c9bf17da","Type":"ContainerStarted","Data":"4a4f8a022eb8386292e70b71590c76bf42fd816335c6420c14280eaf3451b6ce"} Jun 15 01:44:40.811069 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:40.811034 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4xrm5" event={"ID":"134d75e6-f2f5-4264-9f35-c231e06bdbb9","Type":"ContainerStarted","Data":"ebc34265b6ebf8373056ae9451fa67da1fd240c0fa7536fb9324961bb2126059"} Jun 15 01:44:40.825200 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:40.825130 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pmkl7" event={"ID":"3cf2d35f-dae7-42c6-8f35-5b0ccb5e4750","Type":"ContainerStarted","Data":"3a2e38a70ab8dfcfebc09873b40456841a71aea268d2f4dda54d0fcb28a29abb"} Jun 15 01:44:40.841578 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:40.838960 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8r9kz" event={"ID":"9a743adf-5ee6-4357-9206-1ccada80d911","Type":"ContainerStarted","Data":"7daf21dbe2d054dee05bb38a6a730f6a87b30a0fb8e2feb768c1a20aa3700b0f"} Jun 15 01:44:40.843499 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:40.843258 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-tnmwb" event={"ID":"d9f8083b-6138-4b4d-b8b8-d11ca6811a9d","Type":"ContainerStarted","Data":"7db545f5818b19b32ee3adaed62e16523ef1d34ae494ff19f1a3c893dab72693"} Jun 15 01:44:40.846355 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:40.846328 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-r8dqf" event={"ID":"17743aa6-c9cb-458c-b3d3-16c6a774171a","Type":"ContainerStarted","Data":"7c3ef29e307587b4a01cd8382a4ecb50e47323e8d9f46c035c1e66ebf9615e46"} Jun 15 01:44:40.853960 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:40.853932 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7l5ft" event={"ID":"cbbc74f1-9650-4453-955f-fa40d2adb7bc","Type":"ContainerStarted","Data":"1c81a6077c728e8f75223e9794ea640955dbda13719b181a37aa6d115cb2210b"} Jun 15 01:44:40.869939 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:40.869905 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7nhx" event={"ID":"029053ce-e237-4fa4-8a82-bc70b7be8faa","Type":"ContainerStarted","Data":"afcda7e9a3194bf514b27026e6dc1c0ee34e344b3a96b09c442991502c912289"} Jun 15 01:44:40.885985 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:40.885691 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cf286" event={"ID":"64f557b0-78d8-45a6-9f54-756417231d6c","Type":"ContainerStarted","Data":"322d0ed0f2d9f6685f03a49cd6086c6e9bb977e4bab17aeb35a5dff35d576813"} Jun 15 01:44:40.906347 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:40.906309 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8h7vw" event={"ID":"9612108c-f217-4efc-9af1-9500c72da5e3","Type":"ContainerStarted","Data":"77c400a7be28253783a6f4d64578df6fefd3e03eee9e5a08692684a5a4e3748c"} Jun 15 01:44:41.920163 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:41.920070 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-196.ec2.internal" event={"ID":"36e239b4d7762116a74696279f22b0af","Type":"ContainerStarted","Data":"58a361e65a6ed8cb984031fe85ecacc7e8b6ab8f36b51b73cc62b5ddfc751b5a"} Jun 15 01:44:41.936133 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:41.936056 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-196.ec2.internal" podStartSLOduration=3.936035704 podStartE2EDuration="3.936035704s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 01:44:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 01:44:40.811007527 +0000 UTC m=+4.524733500" watchObservedRunningTime="2026-06-15 01:44:41.936035704 +0000 UTC m=+5.649761671" Jun 15 01:44:41.936476 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:41.936436 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-196.ec2.internal" podStartSLOduration=3.936422146 podStartE2EDuration="3.936422146s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 01:44:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 01:44:41.935884169 +0000 UTC m=+5.649610141" watchObservedRunningTime="2026-06-15 01:44:41.936422146 +0000 UTC m=+5.650148121" Jun 15 01:44:42.340105 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:42.339459 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1719195-2a1b-4c96-abe3-c25bda154d3d-metrics-certs\") pod \"network-metrics-daemon-n4p82\" (UID: \"e1719195-2a1b-4c96-abe3-c25bda154d3d\") " pod="openshift-multus/network-metrics-daemon-n4p82" Jun 15 01:44:42.340105 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:42.339643 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 01:44:42.340105 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:42.339702 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1719195-2a1b-4c96-abe3-c25bda154d3d-metrics-certs podName:e1719195-2a1b-4c96-abe3-c25bda154d3d nodeName:}" failed. No retries permitted until 2026-06-15 01:44:46.339681237 +0000 UTC m=+10.053407203 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e1719195-2a1b-4c96-abe3-c25bda154d3d-metrics-certs") pod "network-metrics-daemon-n4p82" (UID: "e1719195-2a1b-4c96-abe3-c25bda154d3d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 01:44:42.440154 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:42.439795 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a6368c58-4355-4d1c-b621-75c5bf0e7394-original-pull-secret\") pod \"global-pull-secret-syncer-6272r\" (UID: \"a6368c58-4355-4d1c-b621-75c5bf0e7394\") " pod="kube-system/global-pull-secret-syncer-6272r" Jun 15 01:44:42.440154 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:42.439932 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 15 01:44:42.440154 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:42.439988 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6368c58-4355-4d1c-b621-75c5bf0e7394-original-pull-secret podName:a6368c58-4355-4d1c-b621-75c5bf0e7394 nodeName:}" failed. No retries permitted until 2026-06-15 01:44:46.439971586 +0000 UTC m=+10.153697538 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a6368c58-4355-4d1c-b621-75c5bf0e7394-original-pull-secret") pod "global-pull-secret-syncer-6272r" (UID: "a6368c58-4355-4d1c-b621-75c5bf0e7394") : object "kube-system"/"original-pull-secret" not registered Jun 15 01:44:42.540273 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:42.540233 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnvlx\" (UniqueName: \"kubernetes.io/projected/a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1-kube-api-access-fnvlx\") pod \"network-check-target-f2kkk\" (UID: \"a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1\") " pod="openshift-network-diagnostics/network-check-target-f2kkk" Jun 15 01:44:42.540473 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:42.540453 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 15 01:44:42.540538 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:42.540481 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 15 01:44:42.540538 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:42.540493 2578 projected.go:196] Error preparing data for projected volume kube-api-access-fnvlx for pod openshift-network-diagnostics/network-check-target-f2kkk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 01:44:42.540603 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:42.540551 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1-kube-api-access-fnvlx podName:a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1 nodeName:}" failed. No retries permitted until 2026-06-15 01:44:46.540533859 +0000 UTC m=+10.254259816 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-fnvlx" (UniqueName: "kubernetes.io/projected/a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1-kube-api-access-fnvlx") pod "network-check-target-f2kkk" (UID: "a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 01:44:42.760701 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:42.760628 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2kkk" Jun 15 01:44:42.760856 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:42.760726 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2kkk" podUID="a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1" Jun 15 01:44:42.761130 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:42.761112 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4p82" Jun 15 01:44:42.761225 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:42.761190 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n4p82" podUID="e1719195-2a1b-4c96-abe3-c25bda154d3d" Jun 15 01:44:42.761331 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:42.761312 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6272r" Jun 15 01:44:42.761391 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:42.761376 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6272r" podUID="a6368c58-4355-4d1c-b621-75c5bf0e7394" Jun 15 01:44:44.760235 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:44.760200 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4p82" Jun 15 01:44:44.760684 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:44.760265 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2kkk" Jun 15 01:44:44.760684 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:44.760285 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6272r" Jun 15 01:44:44.760684 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:44.760371 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2kkk" podUID="a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1" Jun 15 01:44:44.760684 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:44.760462 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6272r" podUID="a6368c58-4355-4d1c-b621-75c5bf0e7394" Jun 15 01:44:44.760684 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:44.760541 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n4p82" podUID="e1719195-2a1b-4c96-abe3-c25bda154d3d" Jun 15 01:44:46.371690 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:46.371648 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1719195-2a1b-4c96-abe3-c25bda154d3d-metrics-certs\") pod \"network-metrics-daemon-n4p82\" (UID: \"e1719195-2a1b-4c96-abe3-c25bda154d3d\") " pod="openshift-multus/network-metrics-daemon-n4p82" Jun 15 01:44:46.372180 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:46.371795 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 01:44:46.372180 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:46.371873 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1719195-2a1b-4c96-abe3-c25bda154d3d-metrics-certs podName:e1719195-2a1b-4c96-abe3-c25bda154d3d nodeName:}" failed. No retries permitted until 2026-06-15 01:44:54.371854047 +0000 UTC m=+18.085580005 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e1719195-2a1b-4c96-abe3-c25bda154d3d-metrics-certs") pod "network-metrics-daemon-n4p82" (UID: "e1719195-2a1b-4c96-abe3-c25bda154d3d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 01:44:46.472259 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:46.472228 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a6368c58-4355-4d1c-b621-75c5bf0e7394-original-pull-secret\") pod \"global-pull-secret-syncer-6272r\" (UID: \"a6368c58-4355-4d1c-b621-75c5bf0e7394\") " pod="kube-system/global-pull-secret-syncer-6272r" Jun 15 01:44:46.472430 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:46.472360 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 15 01:44:46.472430 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:46.472416 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6368c58-4355-4d1c-b621-75c5bf0e7394-original-pull-secret podName:a6368c58-4355-4d1c-b621-75c5bf0e7394 nodeName:}" failed. No retries permitted until 2026-06-15 01:44:54.472402168 +0000 UTC m=+18.186128121 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a6368c58-4355-4d1c-b621-75c5bf0e7394-original-pull-secret") pod "global-pull-secret-syncer-6272r" (UID: "a6368c58-4355-4d1c-b621-75c5bf0e7394") : object "kube-system"/"original-pull-secret" not registered Jun 15 01:44:46.573102 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:46.573045 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnvlx\" (UniqueName: \"kubernetes.io/projected/a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1-kube-api-access-fnvlx\") pod \"network-check-target-f2kkk\" (UID: \"a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1\") " pod="openshift-network-diagnostics/network-check-target-f2kkk" Jun 15 01:44:46.578238 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:46.578201 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 15 01:44:46.578391 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:46.578239 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 15 01:44:46.578391 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:46.578261 2578 projected.go:196] Error preparing data for projected volume kube-api-access-fnvlx for pod openshift-network-diagnostics/network-check-target-f2kkk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 01:44:46.578391 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:46.578329 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1-kube-api-access-fnvlx podName:a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1 nodeName:}" failed. No retries permitted until 2026-06-15 01:44:54.578303981 +0000 UTC m=+18.292029941 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-fnvlx" (UniqueName: "kubernetes.io/projected/a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1-kube-api-access-fnvlx") pod "network-check-target-f2kkk" (UID: "a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 01:44:46.763793 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:46.763515 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4p82" Jun 15 01:44:46.763957 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:46.763853 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n4p82" podUID="e1719195-2a1b-4c96-abe3-c25bda154d3d" Jun 15 01:44:46.764296 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:46.764273 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6272r" Jun 15 01:44:46.764393 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:46.764372 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6272r" podUID="a6368c58-4355-4d1c-b621-75c5bf0e7394" Jun 15 01:44:46.764463 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:46.764437 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2kkk" Jun 15 01:44:46.764526 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:46.764500 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2kkk" podUID="a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1" Jun 15 01:44:48.760355 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:48.760321 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4p82" Jun 15 01:44:48.760819 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:48.760461 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n4p82" podUID="e1719195-2a1b-4c96-abe3-c25bda154d3d" Jun 15 01:44:48.760819 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:48.760790 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6272r" Jun 15 01:44:48.760931 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:48.760863 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6272r" podUID="a6368c58-4355-4d1c-b621-75c5bf0e7394" Jun 15 01:44:48.760931 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:48.760901 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2kkk" Jun 15 01:44:48.761026 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:48.760940 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2kkk" podUID="a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1" Jun 15 01:44:50.761120 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:50.760543 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4p82" Jun 15 01:44:50.761120 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:50.760555 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6272r" Jun 15 01:44:50.761120 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:50.760691 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2kkk" Jun 15 01:44:50.761120 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:50.760688 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n4p82" podUID="e1719195-2a1b-4c96-abe3-c25bda154d3d" Jun 15 01:44:50.761120 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:50.760814 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6272r" podUID="a6368c58-4355-4d1c-b621-75c5bf0e7394" Jun 15 01:44:50.761120 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:50.760910 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2kkk" podUID="a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1" Jun 15 01:44:52.760305 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:52.760273 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2kkk" Jun 15 01:44:52.760305 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:52.760290 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4p82" Jun 15 01:44:52.760883 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:52.760400 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2kkk" podUID="a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1" Jun 15 01:44:52.760883 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:52.760509 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n4p82" podUID="e1719195-2a1b-4c96-abe3-c25bda154d3d" Jun 15 01:44:52.760883 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:52.760562 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6272r" Jun 15 01:44:52.760883 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:52.760647 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6272r" podUID="a6368c58-4355-4d1c-b621-75c5bf0e7394" Jun 15 01:44:54.433774 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:54.433605 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1719195-2a1b-4c96-abe3-c25bda154d3d-metrics-certs\") pod \"network-metrics-daemon-n4p82\" (UID: \"e1719195-2a1b-4c96-abe3-c25bda154d3d\") " pod="openshift-multus/network-metrics-daemon-n4p82" Jun 15 01:44:54.434261 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:54.433788 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 01:44:54.434261 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:54.433871 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1719195-2a1b-4c96-abe3-c25bda154d3d-metrics-certs podName:e1719195-2a1b-4c96-abe3-c25bda154d3d nodeName:}" failed. No retries permitted until 2026-06-15 01:45:10.43385099 +0000 UTC m=+34.147576945 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e1719195-2a1b-4c96-abe3-c25bda154d3d-metrics-certs") pod "network-metrics-daemon-n4p82" (UID: "e1719195-2a1b-4c96-abe3-c25bda154d3d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 01:44:54.534309 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:54.534264 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a6368c58-4355-4d1c-b621-75c5bf0e7394-original-pull-secret\") pod \"global-pull-secret-syncer-6272r\" (UID: \"a6368c58-4355-4d1c-b621-75c5bf0e7394\") " pod="kube-system/global-pull-secret-syncer-6272r" Jun 15 01:44:54.534483 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:54.534415 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 15 01:44:54.534546 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:54.534499 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6368c58-4355-4d1c-b621-75c5bf0e7394-original-pull-secret podName:a6368c58-4355-4d1c-b621-75c5bf0e7394 nodeName:}" failed. No retries permitted until 2026-06-15 01:45:10.534476094 +0000 UTC m=+34.248202049 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a6368c58-4355-4d1c-b621-75c5bf0e7394-original-pull-secret") pod "global-pull-secret-syncer-6272r" (UID: "a6368c58-4355-4d1c-b621-75c5bf0e7394") : object "kube-system"/"original-pull-secret" not registered Jun 15 01:44:54.635440 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:54.635396 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnvlx\" (UniqueName: \"kubernetes.io/projected/a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1-kube-api-access-fnvlx\") pod \"network-check-target-f2kkk\" (UID: \"a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1\") " pod="openshift-network-diagnostics/network-check-target-f2kkk" Jun 15 01:44:54.635598 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:54.635573 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 15 01:44:54.635644 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:54.635600 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 15 01:44:54.635644 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:54.635612 2578 projected.go:196] Error preparing data for projected volume kube-api-access-fnvlx for pod openshift-network-diagnostics/network-check-target-f2kkk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 01:44:54.635762 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:54.635673 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1-kube-api-access-fnvlx podName:a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1 nodeName:}" failed. No retries permitted until 2026-06-15 01:45:10.63565404 +0000 UTC m=+34.349380005 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-fnvlx" (UniqueName: "kubernetes.io/projected/a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1-kube-api-access-fnvlx") pod "network-check-target-f2kkk" (UID: "a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 01:44:54.760371 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:54.760299 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6272r" Jun 15 01:44:54.760371 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:54.760329 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2kkk" Jun 15 01:44:54.760558 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:54.760299 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4p82" Jun 15 01:44:54.760558 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:54.760423 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6272r" podUID="a6368c58-4355-4d1c-b621-75c5bf0e7394" Jun 15 01:44:54.760558 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:54.760481 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n4p82" podUID="e1719195-2a1b-4c96-abe3-c25bda154d3d" Jun 15 01:44:54.760711 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:54.760595 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2kkk" podUID="a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1" Jun 15 01:44:56.763157 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:56.763117 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2kkk" Jun 15 01:44:56.763589 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:56.763230 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2kkk" podUID="a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1" Jun 15 01:44:56.763646 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:56.763618 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4p82" Jun 15 01:44:56.763745 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:56.763723 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n4p82" podUID="e1719195-2a1b-4c96-abe3-c25bda154d3d" Jun 15 01:44:56.763891 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:56.763862 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6272r" Jun 15 01:44:56.763999 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:56.763963 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6272r" podUID="a6368c58-4355-4d1c-b621-75c5bf0e7394" Jun 15 01:44:58.760519 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:58.760487 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4p82" Jun 15 01:44:58.761019 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:58.760487 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6272r" Jun 15 01:44:58.761019 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:58.760606 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n4p82" podUID="e1719195-2a1b-4c96-abe3-c25bda154d3d" Jun 15 01:44:58.761019 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:58.760626 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2kkk" Jun 15 01:44:58.761019 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:58.760692 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6272r" podUID="a6368c58-4355-4d1c-b621-75c5bf0e7394" Jun 15 01:44:58.761019 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:44:58.760810 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2kkk" podUID="a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1" Jun 15 01:44:59.956634 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:59.956259 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4xrm5" event={"ID":"134d75e6-f2f5-4264-9f35-c231e06bdbb9","Type":"ContainerStarted","Data":"1cfcf1d95194c079d5a1f9900647a132769a272108aa4a27955bd0840469e61c"} Jun 15 01:44:59.957812 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:59.957787 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pmkl7" event={"ID":"3cf2d35f-dae7-42c6-8f35-5b0ccb5e4750","Type":"ContainerStarted","Data":"ab81a6437de30bd8faa896a2d5eb5fbe4ba92ef97b7c1fa43da33d3a105df5da"} Jun 15 01:44:59.959867 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:59.959840 2578 generic.go:356] "Generic (PLEG): container finished" podID="9a743adf-5ee6-4357-9206-1ccada80d911" containerID="5b0ff7fa0800b80af6d11c28ecd4063f73d5d1f2ed095175faaac410ce63da72" exitCode=0 Jun 15 01:44:59.959986 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:59.959920 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8r9kz" event={"ID":"9a743adf-5ee6-4357-9206-1ccada80d911","Type":"ContainerDied","Data":"5b0ff7fa0800b80af6d11c28ecd4063f73d5d1f2ed095175faaac410ce63da72"} Jun 15 01:44:59.961533 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:59.961506 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-tnmwb" event={"ID":"d9f8083b-6138-4b4d-b8b8-d11ca6811a9d","Type":"ContainerStarted","Data":"373cfc62408154d05dbb57467c55da11fdea7c71e5119f86752bfa5f1c1ab75c"} Jun 15 01:44:59.963045 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:59.963022 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-r8dqf" event={"ID":"17743aa6-c9cb-458c-b3d3-16c6a774171a","Type":"ContainerStarted","Data":"29046ad3296c0b2570b1578f19bf0a847c1e1bf919c9e477972edf7f28383051"} Jun 15 01:44:59.964498 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:59.964475 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7nhx" event={"ID":"029053ce-e237-4fa4-8a82-bc70b7be8faa","Type":"ContainerStarted","Data":"1cb03c9bae4352d1b2f72f69fa95942d1b5393ded375f0fd7f61f13dbf07e395"} Jun 15 01:44:59.967522 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:59.967505 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cf286_64f557b0-78d8-45a6-9f54-756417231d6c/ovn-acl-logging/0.log" Jun 15 01:44:59.967835 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:59.967813 2578 generic.go:356] "Generic (PLEG): container finished" podID="64f557b0-78d8-45a6-9f54-756417231d6c" containerID="4b76a87dc093ae8ff29fa72988b00930bd45df260130845e956c131a28b94471" exitCode=1 Jun 15 01:44:59.967893 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:59.967865 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cf286" event={"ID":"64f557b0-78d8-45a6-9f54-756417231d6c","Type":"ContainerStarted","Data":"277b1e34536032d10fe86611e406697a3220a7cec795bf6d6a699460f0b6f6ae"} Jun 15 01:44:59.967960 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:59.967929 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cf286" event={"ID":"64f557b0-78d8-45a6-9f54-756417231d6c","Type":"ContainerStarted","Data":"b9ce63938d7cc7b189df5615bbe2aa1998ecac12242b34f70ffe96b6d3bfaede"} Jun 15 01:44:59.968018 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:59.967979 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cf286" event={"ID":"64f557b0-78d8-45a6-9f54-756417231d6c","Type":"ContainerStarted","Data":"9c877c05bcbf153a0d8a18574b7971490f6b032fb0f9831c34008151c2b37d3c"} Jun 15 01:44:59.968075 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:59.968028 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cf286" event={"ID":"64f557b0-78d8-45a6-9f54-756417231d6c","Type":"ContainerStarted","Data":"959b9999d94b83abd7d130e2043ec5f7bfee7f9c310cdf7a1b8f94a35956732f"} Jun 15 01:44:59.968147 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:59.968075 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cf286" event={"ID":"64f557b0-78d8-45a6-9f54-756417231d6c","Type":"ContainerDied","Data":"4b76a87dc093ae8ff29fa72988b00930bd45df260130845e956c131a28b94471"} Jun 15 01:44:59.968147 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:59.968137 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cf286" event={"ID":"64f557b0-78d8-45a6-9f54-756417231d6c","Type":"ContainerStarted","Data":"62622873414cb4d0860b02dc23cced123791a63cc6d687ed879e51ec1614fdfb"} Jun 15 01:44:59.969122 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:59.969104 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8h7vw" event={"ID":"9612108c-f217-4efc-9af1-9500c72da5e3","Type":"ContainerStarted","Data":"714cf26aa389afd6b9a811e5d7cba84178dfceee6475395ffef3107bf709cfdc"} Jun 15 01:44:59.973240 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:59.973203 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-4xrm5" podStartSLOduration=3.9653611890000002 podStartE2EDuration="22.973192269s" totalImagesPullingTime="19.00783108s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 01:44:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 01:44:59.972599927 +0000 UTC m=+23.686325900" watchObservedRunningTime="2026-06-15 01:44:59.973192269 +0000 UTC m=+23.686918242" Jun 15 01:44:59.986482 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:59.986459 2578 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 15 01:44:59.986759 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:44:59.986721 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-pmkl7" podStartSLOduration=4.004567896 podStartE2EDuration="22.986708558s" totalImagesPullingTime="18.982140662s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 01:44:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 01:44:59.986245137 +0000 UTC m=+23.699971108" watchObservedRunningTime="2026-06-15 01:44:59.986708558 +0000 UTC m=+23.700434530" Jun 15 01:45:00.018821 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:00.018772 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-r8dqf" podStartSLOduration=4.036473483 podStartE2EDuration="23.018756091s" totalImagesPullingTime="18.982282608s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 01:44:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 01:45:00.018717553 +0000 UTC m=+23.732443527" watchObservedRunningTime="2026-06-15 01:45:00.018756091 +0000 UTC m=+23.732482062" Jun 15 01:45:00.034470 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:00.034424 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-8h7vw" podStartSLOduration=4.039223532 podStartE2EDuration="23.034407686s" totalImagesPullingTime="18.995184154s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 01:44:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 01:45:00.034114006 +0000 UTC m=+23.747839968" watchObservedRunningTime="2026-06-15 01:45:00.034407686 +0000 UTC m=+23.748133657" Jun 15 01:45:00.760758 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:00.760726 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4p82" Jun 15 01:45:00.760936 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:00.760726 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2kkk" Jun 15 01:45:00.760936 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:00.760861 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n4p82" podUID="e1719195-2a1b-4c96-abe3-c25bda154d3d" Jun 15 01:45:00.761053 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:00.760974 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2kkk" podUID="a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1" Jun 15 01:45:00.761053 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:00.760737 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6272r" Jun 15 01:45:00.761230 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:00.761070 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6272r" podUID="a6368c58-4355-4d1c-b621-75c5bf0e7394" Jun 15 01:45:00.761593 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:00.761466 2578 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-15T01:44:59.986477733Z","UUID":"8318c9e4-6890-4a00-9dfa-24672fa9fe11","Handler":null,"Name":"","Endpoint":""} Jun 15 01:45:00.765325 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:00.763445 2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 15 01:45:00.765325 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:00.763486 2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 15 01:45:00.973185 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:00.973139 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7l5ft" event={"ID":"cbbc74f1-9650-4453-955f-fa40d2adb7bc","Type":"ContainerStarted","Data":"1432de539420281b4713b2a4b256a84f11bb780efe0ad0679ac519579a941fbf"} Jun 15 01:45:00.975488 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:00.975452 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7nhx" event={"ID":"029053ce-e237-4fa4-8a82-bc70b7be8faa","Type":"ContainerStarted","Data":"0049daf2a2630db8b535c6d1ad7864b051f2603c3606f27820679c5732a32559"} Jun 15 01:45:00.975611 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:00.975514 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7nhx" event={"ID":"029053ce-e237-4fa4-8a82-bc70b7be8faa","Type":"ContainerStarted","Data":"aa8e7f6f828e009387b560acc64ecd82a43a7186dad7fdc1b78fd68f1f9380d2"} Jun 15 01:45:00.989553 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:00.989485 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-7l5ft" podStartSLOduration=6.003506606 podStartE2EDuration="24.989464747s" totalImagesPullingTime="18.985958141s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 01:44:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 01:45:00.989441925 +0000 UTC m=+24.703167897" watchObservedRunningTime="2026-06-15 01:45:00.989464747 +0000 UTC m=+24.703190719" Jun 15 01:45:00.990073 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:00.990041 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-tnmwb" podStartSLOduration=14.429791745 podStartE2EDuration="24.990031768s" totalImagesPullingTime="10.560240023s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 01:44:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 01:45:00.05193545 +0000 UTC m=+23.765661422" watchObservedRunningTime="2026-06-15 01:45:00.990031768 +0000 UTC m=+24.703757741" Jun 15 01:45:01.981719 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:01.981686 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cf286_64f557b0-78d8-45a6-9f54-756417231d6c/ovn-acl-logging/0.log" Jun 15 01:45:01.982315 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:01.982222 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cf286" event={"ID":"64f557b0-78d8-45a6-9f54-756417231d6c","Type":"ContainerStarted","Data":"f7bbffe2eb06b241b417bc5a93aa410440a12e98b6f5b2fe3b5e03de863d4b32"} Jun 15 01:45:02.760599 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:02.760564 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2kkk" Jun 15 01:45:02.760791 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:02.760568 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6272r" Jun 15 01:45:02.760791 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:02.760692 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2kkk" podUID="a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1" Jun 15 01:45:02.760791 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:02.760568 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4p82" Jun 15 01:45:02.760948 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:02.760739 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6272r" podUID="a6368c58-4355-4d1c-b621-75c5bf0e7394" Jun 15 01:45:02.760948 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:02.760875 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n4p82" podUID="e1719195-2a1b-4c96-abe3-c25bda154d3d" Jun 15 01:45:03.901979 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:03.901937 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-tnmwb" Jun 15 01:45:03.902837 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:03.902814 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-tnmwb" Jun 15 01:45:03.918599 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:03.918504 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7nhx" podStartSLOduration=7.408158633 podStartE2EDuration="27.918484237s" totalImagesPullingTime="20.510325604s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 01:44:36 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 01:45:01.012780275 +0000 UTC m=+24.726506247" watchObservedRunningTime="2026-06-15 01:45:03.918484237 +0000 UTC m=+27.632210549" Jun 15 01:45:03.987180 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:03.987131 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-tnmwb" Jun 15 01:45:03.987708 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:03.987682 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-tnmwb" Jun 15 01:45:04.760387 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:04.760190 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2kkk" Jun 15 01:45:04.760531 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:04.760190 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4p82" Jun 15 01:45:04.760531 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:04.760460 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2kkk" podUID="a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1" Jun 15 01:45:04.760591 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:04.760259 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6272r" Jun 15 01:45:04.760591 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:04.760567 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n4p82" podUID="e1719195-2a1b-4c96-abe3-c25bda154d3d" Jun 15 01:45:04.760671 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:04.760624 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6272r" podUID="a6368c58-4355-4d1c-b621-75c5bf0e7394" Jun 15 01:45:04.992668 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:04.992591 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cf286_64f557b0-78d8-45a6-9f54-756417231d6c/ovn-acl-logging/0.log" Jun 15 01:45:04.993109 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:04.992951 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cf286" event={"ID":"64f557b0-78d8-45a6-9f54-756417231d6c","Type":"ContainerStarted","Data":"8f347c185453f7c6592ccdd5e4d2d5aded49eda8e9682bb516f856dddd22b2f8"} Jun 15 01:45:04.993233 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:04.993213 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:45:04.993306 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:04.993282 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:45:04.993476 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:04.993457 2578 scope.go:117] "RemoveContainer" containerID="4b76a87dc093ae8ff29fa72988b00930bd45df260130845e956c131a28b94471" Jun 15 01:45:04.995072 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:04.995047 2578 generic.go:356] "Generic (PLEG): container finished" podID="9a743adf-5ee6-4357-9206-1ccada80d911" containerID="b421c5c9201238addb749dac005d090de0d560e97219161faf6e8c36f7a11977" exitCode=0 Jun 15 01:45:04.995197 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:04.995126 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8r9kz" event={"ID":"9a743adf-5ee6-4357-9206-1ccada80d911","Type":"ContainerDied","Data":"b421c5c9201238addb749dac005d090de0d560e97219161faf6e8c36f7a11977"} Jun 15 01:45:05.010661 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:05.010640 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:45:06.001559 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:06.001534 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cf286_64f557b0-78d8-45a6-9f54-756417231d6c/ovn-acl-logging/0.log" Jun 15 01:45:06.001957 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:06.001931 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cf286" event={"ID":"64f557b0-78d8-45a6-9f54-756417231d6c","Type":"ContainerStarted","Data":"3f3c5f7d8c6130ed968b254698a1391c42cbb5cf72f09ffe9e4c7f66bfd27650"} Jun 15 01:45:06.002267 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:06.002243 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:45:06.017657 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:06.017632 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:45:06.028584 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:06.028544 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-cf286" podStartSLOduration=11.002747533 podStartE2EDuration="30.028525866s" totalImagesPullingTime="19.025778333s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 01:44:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 01:45:06.028493106 +0000 UTC m=+29.742219078" watchObservedRunningTime="2026-06-15 01:45:06.028525866 +0000 UTC m=+29.742251819" Jun 15 01:45:06.622963 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:06.622776 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-f2kkk"] Jun 15 01:45:06.623119 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:06.623071 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2kkk" Jun 15 01:45:06.623181 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:06.623163 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2kkk" podUID="a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1" Jun 15 01:45:06.625759 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:06.625735 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6272r"] Jun 15 01:45:06.625909 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:06.625842 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6272r" Jun 15 01:45:06.625972 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:06.625939 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6272r" podUID="a6368c58-4355-4d1c-b621-75c5bf0e7394" Jun 15 01:45:06.626357 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:06.626334 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-n4p82"] Jun 15 01:45:06.626462 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:06.626418 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4p82" Jun 15 01:45:06.626527 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:06.626485 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n4p82" podUID="e1719195-2a1b-4c96-abe3-c25bda154d3d" Jun 15 01:45:07.006237 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:07.006151 2578 generic.go:356] "Generic (PLEG): container finished" podID="9a743adf-5ee6-4357-9206-1ccada80d911" containerID="4bb9a1f45429030004a7ad17e53792c005038eac6e9423c245f796097a612746" exitCode=0 Jun 15 01:45:07.006590 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:07.006240 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8r9kz" event={"ID":"9a743adf-5ee6-4357-9206-1ccada80d911","Type":"ContainerDied","Data":"4bb9a1f45429030004a7ad17e53792c005038eac6e9423c245f796097a612746"} Jun 15 01:45:07.760870 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:07.760843 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2kkk" Jun 15 01:45:07.760993 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:07.760885 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6272r" Jun 15 01:45:07.761042 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:07.760999 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2kkk" podUID="a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1" Jun 15 01:45:07.764156 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:07.761425 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6272r" podUID="a6368c58-4355-4d1c-b621-75c5bf0e7394" Jun 15 01:45:08.760471 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:08.760393 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4p82" Jun 15 01:45:08.761111 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:08.760499 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n4p82" podUID="e1719195-2a1b-4c96-abe3-c25bda154d3d" Jun 15 01:45:09.012437 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:09.012352 2578 generic.go:356] "Generic (PLEG): container finished" podID="9a743adf-5ee6-4357-9206-1ccada80d911" containerID="5b170962bc8453adf76b3656b531eb3ebd4736d38105bd1a9a5ac8ee245ec33c" exitCode=0 Jun 15 01:45:09.012437 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:09.012411 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8r9kz" event={"ID":"9a743adf-5ee6-4357-9206-1ccada80d911","Type":"ContainerDied","Data":"5b170962bc8453adf76b3656b531eb3ebd4736d38105bd1a9a5ac8ee245ec33c"} Jun 15 01:45:09.760828 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:09.760769 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6272r" Jun 15 01:45:09.761408 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:09.760903 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6272r" podUID="a6368c58-4355-4d1c-b621-75c5bf0e7394" Jun 15 01:45:09.761408 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:09.760954 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2kkk" Jun 15 01:45:09.761408 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:09.761056 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2kkk" podUID="a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1" Jun 15 01:45:10.451862 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:10.451763 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1719195-2a1b-4c96-abe3-c25bda154d3d-metrics-certs\") pod \"network-metrics-daemon-n4p82\" (UID: \"e1719195-2a1b-4c96-abe3-c25bda154d3d\") " pod="openshift-multus/network-metrics-daemon-n4p82" Jun 15 01:45:10.452066 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:10.451957 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 01:45:10.452066 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:10.452019 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1719195-2a1b-4c96-abe3-c25bda154d3d-metrics-certs podName:e1719195-2a1b-4c96-abe3-c25bda154d3d nodeName:}" failed. No retries permitted until 2026-06-15 01:45:42.452004697 +0000 UTC m=+66.165730647 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e1719195-2a1b-4c96-abe3-c25bda154d3d-metrics-certs") pod "network-metrics-daemon-n4p82" (UID: "e1719195-2a1b-4c96-abe3-c25bda154d3d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 01:45:10.552765 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:10.552734 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a6368c58-4355-4d1c-b621-75c5bf0e7394-original-pull-secret\") pod \"global-pull-secret-syncer-6272r\" (UID: \"a6368c58-4355-4d1c-b621-75c5bf0e7394\") " pod="kube-system/global-pull-secret-syncer-6272r" Jun 15 01:45:10.552961 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:10.552872 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 15 01:45:10.552961 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:10.552945 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6368c58-4355-4d1c-b621-75c5bf0e7394-original-pull-secret podName:a6368c58-4355-4d1c-b621-75c5bf0e7394 nodeName:}" failed. No retries permitted until 2026-06-15 01:45:42.552926807 +0000 UTC m=+66.266652757 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a6368c58-4355-4d1c-b621-75c5bf0e7394-original-pull-secret") pod "global-pull-secret-syncer-6272r" (UID: "a6368c58-4355-4d1c-b621-75c5bf0e7394") : object "kube-system"/"original-pull-secret" not registered Jun 15 01:45:10.653835 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:10.653801 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnvlx\" (UniqueName: \"kubernetes.io/projected/a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1-kube-api-access-fnvlx\") pod \"network-check-target-f2kkk\" (UID: \"a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1\") " pod="openshift-network-diagnostics/network-check-target-f2kkk" Jun 15 01:45:10.654025 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:10.653996 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 15 01:45:10.654025 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:10.654022 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 15 01:45:10.654171 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:10.654035 2578 projected.go:196] Error preparing data for projected volume kube-api-access-fnvlx for pod openshift-network-diagnostics/network-check-target-f2kkk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 01:45:10.654171 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:10.654126 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1-kube-api-access-fnvlx podName:a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1 nodeName:}" failed. No retries permitted until 2026-06-15 01:45:42.654106983 +0000 UTC m=+66.367832948 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-fnvlx" (UniqueName: "kubernetes.io/projected/a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1-kube-api-access-fnvlx") pod "network-check-target-f2kkk" (UID: "a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 01:45:10.760525 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:10.760018 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4p82" Jun 15 01:45:10.760525 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:10.760172 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n4p82" podUID="e1719195-2a1b-4c96-abe3-c25bda154d3d" Jun 15 01:45:11.543932 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.543734 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-196.ec2.internal" event="NodeReady" Jun 15 01:45:11.544384 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.544065 2578 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jun 15 01:45:11.583458 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.583429 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5b5c665f7c-7mtmh"] Jun 15 01:45:11.598454 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.598421 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-bmrhc"] Jun 15 01:45:11.598618 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.598559 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b5c665f7c-7mtmh" Jun 15 01:45:11.602242 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.601731 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 15 01:45:11.602242 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.602044 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 15 01:45:11.602408 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.602297 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-h8zgg\"" Jun 15 01:45:11.602537 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.602516 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 15 01:45:11.608013 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.607821 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 15 01:45:11.608363 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.608343 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5b5c665f7c-7mtmh"] Jun 15 01:45:11.608452 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.608382 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-48xtp"] Jun 15 01:45:11.608545 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.608527 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bmrhc" Jun 15 01:45:11.611742 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.611719 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 15 01:45:11.611863 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.611720 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 15 01:45:11.612027 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.612010 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2blxm\"" Jun 15 01:45:11.622589 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.622563 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bmrhc"] Jun 15 01:45:11.622701 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.622609 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-48xtp"] Jun 15 01:45:11.622701 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.622685 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-48xtp" Jun 15 01:45:11.625733 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.625708 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-m47tl\"" Jun 15 01:45:11.625837 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.625800 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 15 01:45:11.626073 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.626059 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 15 01:45:11.626186 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.626169 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 15 01:45:11.760195 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.760103 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6272r" Jun 15 01:45:11.760332 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.760263 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2kkk" Jun 15 01:45:11.761334 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.761300 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f6afa77-1931-4efb-aeae-7a7fdef97959-tmp-dir\") pod \"dns-default-bmrhc\" (UID: \"7f6afa77-1931-4efb-aeae-7a7fdef97959\") " pod="openshift-dns/dns-default-bmrhc" Jun 15 01:45:11.761435 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.761346 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/26023146-86ca-4a77-b73c-429dd24c4e5d-bound-sa-token\") pod \"image-registry-5b5c665f7c-7mtmh\" (UID: \"26023146-86ca-4a77-b73c-429dd24c4e5d\") " pod="openshift-image-registry/image-registry-5b5c665f7c-7mtmh" Jun 15 01:45:11.761435 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.761397 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l87tg\" (UniqueName: \"kubernetes.io/projected/26023146-86ca-4a77-b73c-429dd24c4e5d-kube-api-access-l87tg\") pod \"image-registry-5b5c665f7c-7mtmh\" (UID: \"26023146-86ca-4a77-b73c-429dd24c4e5d\") " pod="openshift-image-registry/image-registry-5b5c665f7c-7mtmh" Jun 15 01:45:11.761435 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.761434 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjbkh\" (UniqueName: \"kubernetes.io/projected/878da7f4-63c4-45b4-a4c6-fc6a72556516-kube-api-access-mjbkh\") pod \"ingress-canary-48xtp\" (UID: \"878da7f4-63c4-45b4-a4c6-fc6a72556516\") " pod="openshift-ingress-canary/ingress-canary-48xtp" Jun 15 01:45:11.761606 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.761508 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/26023146-86ca-4a77-b73c-429dd24c4e5d-registry-tls\") pod \"image-registry-5b5c665f7c-7mtmh\" (UID: \"26023146-86ca-4a77-b73c-429dd24c4e5d\") " pod="openshift-image-registry/image-registry-5b5c665f7c-7mtmh" Jun 15 01:45:11.761606 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.761536 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/26023146-86ca-4a77-b73c-429dd24c4e5d-ca-trust-extracted\") pod \"image-registry-5b5c665f7c-7mtmh\" (UID: \"26023146-86ca-4a77-b73c-429dd24c4e5d\") " pod="openshift-image-registry/image-registry-5b5c665f7c-7mtmh" Jun 15 01:45:11.761606 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.761559 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/26023146-86ca-4a77-b73c-429dd24c4e5d-registry-certificates\") pod \"image-registry-5b5c665f7c-7mtmh\" (UID: \"26023146-86ca-4a77-b73c-429dd24c4e5d\") " pod="openshift-image-registry/image-registry-5b5c665f7c-7mtmh" Jun 15 01:45:11.761606 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.761583 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/26023146-86ca-4a77-b73c-429dd24c4e5d-trusted-ca\") pod \"image-registry-5b5c665f7c-7mtmh\" (UID: \"26023146-86ca-4a77-b73c-429dd24c4e5d\") " pod="openshift-image-registry/image-registry-5b5c665f7c-7mtmh" Jun 15 01:45:11.761770 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.761619 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/878da7f4-63c4-45b4-a4c6-fc6a72556516-cert\") pod \"ingress-canary-48xtp\" (UID: \"878da7f4-63c4-45b4-a4c6-fc6a72556516\") " pod="openshift-ingress-canary/ingress-canary-48xtp" Jun 15 01:45:11.761770 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.761643 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/26023146-86ca-4a77-b73c-429dd24c4e5d-image-registry-private-configuration\") pod \"image-registry-5b5c665f7c-7mtmh\" (UID: \"26023146-86ca-4a77-b73c-429dd24c4e5d\") " pod="openshift-image-registry/image-registry-5b5c665f7c-7mtmh" Jun 15 01:45:11.761770 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.761677 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/26023146-86ca-4a77-b73c-429dd24c4e5d-installation-pull-secrets\") pod \"image-registry-5b5c665f7c-7mtmh\" (UID: \"26023146-86ca-4a77-b73c-429dd24c4e5d\") " pod="openshift-image-registry/image-registry-5b5c665f7c-7mtmh" Jun 15 01:45:11.761770 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.761722 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7f6afa77-1931-4efb-aeae-7a7fdef97959-config-volume\") pod \"dns-default-bmrhc\" (UID: \"7f6afa77-1931-4efb-aeae-7a7fdef97959\") " pod="openshift-dns/dns-default-bmrhc" Jun 15 01:45:11.761770 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.761742 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7f6afa77-1931-4efb-aeae-7a7fdef97959-metrics-tls\") pod \"dns-default-bmrhc\" (UID: \"7f6afa77-1931-4efb-aeae-7a7fdef97959\") " pod="openshift-dns/dns-default-bmrhc" Jun 15 01:45:11.761770 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.761756 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kh97b\" (UniqueName: \"kubernetes.io/projected/7f6afa77-1931-4efb-aeae-7a7fdef97959-kube-api-access-kh97b\") pod \"dns-default-bmrhc\" (UID: \"7f6afa77-1931-4efb-aeae-7a7fdef97959\") " pod="openshift-dns/dns-default-bmrhc" Jun 15 01:45:11.763437 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.763422 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7pbsq\"" Jun 15 01:45:11.763532 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.763508 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 15 01:45:11.763634 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.763562 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 15 01:45:11.763634 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.763607 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 15 01:45:11.862278 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.862241 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjbkh\" (UniqueName: \"kubernetes.io/projected/878da7f4-63c4-45b4-a4c6-fc6a72556516-kube-api-access-mjbkh\") pod \"ingress-canary-48xtp\" (UID: \"878da7f4-63c4-45b4-a4c6-fc6a72556516\") " pod="openshift-ingress-canary/ingress-canary-48xtp" Jun 15 01:45:11.862470 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.862306 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/26023146-86ca-4a77-b73c-429dd24c4e5d-registry-tls\") pod \"image-registry-5b5c665f7c-7mtmh\" (UID: \"26023146-86ca-4a77-b73c-429dd24c4e5d\") " pod="openshift-image-registry/image-registry-5b5c665f7c-7mtmh" Jun 15 01:45:11.862470 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:11.862403 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 15 01:45:11.862470 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:11.862417 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b5c665f7c-7mtmh: secret "image-registry-tls" not found Jun 15 01:45:11.862679 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:11.862480 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/26023146-86ca-4a77-b73c-429dd24c4e5d-registry-tls podName:26023146-86ca-4a77-b73c-429dd24c4e5d nodeName:}" failed. No retries permitted until 2026-06-15 01:45:12.362460563 +0000 UTC m=+36.076186516 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/26023146-86ca-4a77-b73c-429dd24c4e5d-registry-tls") pod "image-registry-5b5c665f7c-7mtmh" (UID: "26023146-86ca-4a77-b73c-429dd24c4e5d") : secret "image-registry-tls" not found Jun 15 01:45:11.862679 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.862479 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/26023146-86ca-4a77-b73c-429dd24c4e5d-ca-trust-extracted\") pod \"image-registry-5b5c665f7c-7mtmh\" (UID: \"26023146-86ca-4a77-b73c-429dd24c4e5d\") " pod="openshift-image-registry/image-registry-5b5c665f7c-7mtmh" Jun 15 01:45:11.862679 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.862533 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/26023146-86ca-4a77-b73c-429dd24c4e5d-registry-certificates\") pod \"image-registry-5b5c665f7c-7mtmh\" (UID: \"26023146-86ca-4a77-b73c-429dd24c4e5d\") " pod="openshift-image-registry/image-registry-5b5c665f7c-7mtmh" Jun 15 01:45:11.862679 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.862607 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/26023146-86ca-4a77-b73c-429dd24c4e5d-trusted-ca\") pod \"image-registry-5b5c665f7c-7mtmh\" (UID: \"26023146-86ca-4a77-b73c-429dd24c4e5d\") " pod="openshift-image-registry/image-registry-5b5c665f7c-7mtmh" Jun 15 01:45:11.862679 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.862650 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/878da7f4-63c4-45b4-a4c6-fc6a72556516-cert\") pod \"ingress-canary-48xtp\" (UID: \"878da7f4-63c4-45b4-a4c6-fc6a72556516\") " pod="openshift-ingress-canary/ingress-canary-48xtp" Jun 15 01:45:11.862921 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.862679 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/26023146-86ca-4a77-b73c-429dd24c4e5d-image-registry-private-configuration\") pod \"image-registry-5b5c665f7c-7mtmh\" (UID: \"26023146-86ca-4a77-b73c-429dd24c4e5d\") " pod="openshift-image-registry/image-registry-5b5c665f7c-7mtmh" Jun 15 01:45:11.862976 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.862953 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/26023146-86ca-4a77-b73c-429dd24c4e5d-installation-pull-secrets\") pod \"image-registry-5b5c665f7c-7mtmh\" (UID: \"26023146-86ca-4a77-b73c-429dd24c4e5d\") " pod="openshift-image-registry/image-registry-5b5c665f7c-7mtmh" Jun 15 01:45:11.863028 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.862965 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/26023146-86ca-4a77-b73c-429dd24c4e5d-ca-trust-extracted\") pod \"image-registry-5b5c665f7c-7mtmh\" (UID: \"26023146-86ca-4a77-b73c-429dd24c4e5d\") " pod="openshift-image-registry/image-registry-5b5c665f7c-7mtmh" Jun 15 01:45:11.863078 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.863027 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7f6afa77-1931-4efb-aeae-7a7fdef97959-config-volume\") pod \"dns-default-bmrhc\" (UID: \"7f6afa77-1931-4efb-aeae-7a7fdef97959\") " pod="openshift-dns/dns-default-bmrhc" Jun 15 01:45:11.863078 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.863059 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7f6afa77-1931-4efb-aeae-7a7fdef97959-metrics-tls\") pod \"dns-default-bmrhc\" (UID: \"7f6afa77-1931-4efb-aeae-7a7fdef97959\") " pod="openshift-dns/dns-default-bmrhc" Jun 15 01:45:11.863215 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.863115 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kh97b\" (UniqueName: \"kubernetes.io/projected/7f6afa77-1931-4efb-aeae-7a7fdef97959-kube-api-access-kh97b\") pod \"dns-default-bmrhc\" (UID: \"7f6afa77-1931-4efb-aeae-7a7fdef97959\") " pod="openshift-dns/dns-default-bmrhc" Jun 15 01:45:11.863215 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.863149 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f6afa77-1931-4efb-aeae-7a7fdef97959-tmp-dir\") pod \"dns-default-bmrhc\" (UID: \"7f6afa77-1931-4efb-aeae-7a7fdef97959\") " pod="openshift-dns/dns-default-bmrhc" Jun 15 01:45:11.863215 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.863175 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/26023146-86ca-4a77-b73c-429dd24c4e5d-bound-sa-token\") pod \"image-registry-5b5c665f7c-7mtmh\" (UID: \"26023146-86ca-4a77-b73c-429dd24c4e5d\") " pod="openshift-image-registry/image-registry-5b5c665f7c-7mtmh" Jun 15 01:45:11.863356 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.863197 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l87tg\" (UniqueName: \"kubernetes.io/projected/26023146-86ca-4a77-b73c-429dd24c4e5d-kube-api-access-l87tg\") pod \"image-registry-5b5c665f7c-7mtmh\" (UID: \"26023146-86ca-4a77-b73c-429dd24c4e5d\") " pod="openshift-image-registry/image-registry-5b5c665f7c-7mtmh" Jun 15 01:45:11.863356 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.863267 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/26023146-86ca-4a77-b73c-429dd24c4e5d-registry-certificates\") pod \"image-registry-5b5c665f7c-7mtmh\" (UID: \"26023146-86ca-4a77-b73c-429dd24c4e5d\") " pod="openshift-image-registry/image-registry-5b5c665f7c-7mtmh" Jun 15 01:45:11.863489 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:11.863471 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 15 01:45:11.863547 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:11.863537 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f6afa77-1931-4efb-aeae-7a7fdef97959-metrics-tls podName:7f6afa77-1931-4efb-aeae-7a7fdef97959 nodeName:}" failed. No retries permitted until 2026-06-15 01:45:12.363519447 +0000 UTC m=+36.077245399 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7f6afa77-1931-4efb-aeae-7a7fdef97959-metrics-tls") pod "dns-default-bmrhc" (UID: "7f6afa77-1931-4efb-aeae-7a7fdef97959") : secret "dns-default-metrics-tls" not found Jun 15 01:45:11.864048 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.863792 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f6afa77-1931-4efb-aeae-7a7fdef97959-tmp-dir\") pod \"dns-default-bmrhc\" (UID: \"7f6afa77-1931-4efb-aeae-7a7fdef97959\") " pod="openshift-dns/dns-default-bmrhc" Jun 15 01:45:11.864048 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.863967 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7f6afa77-1931-4efb-aeae-7a7fdef97959-config-volume\") pod \"dns-default-bmrhc\" (UID: \"7f6afa77-1931-4efb-aeae-7a7fdef97959\") " pod="openshift-dns/dns-default-bmrhc" Jun 15 01:45:11.864246 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.864107 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/26023146-86ca-4a77-b73c-429dd24c4e5d-trusted-ca\") pod \"image-registry-5b5c665f7c-7mtmh\" (UID: \"26023146-86ca-4a77-b73c-429dd24c4e5d\") " pod="openshift-image-registry/image-registry-5b5c665f7c-7mtmh" Jun 15 01:45:11.867942 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.867920 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/26023146-86ca-4a77-b73c-429dd24c4e5d-installation-pull-secrets\") pod \"image-registry-5b5c665f7c-7mtmh\" (UID: \"26023146-86ca-4a77-b73c-429dd24c4e5d\") " pod="openshift-image-registry/image-registry-5b5c665f7c-7mtmh" Jun 15 01:45:11.868050 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.867923 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/878da7f4-63c4-45b4-a4c6-fc6a72556516-cert\") pod \"ingress-canary-48xtp\" (UID: \"878da7f4-63c4-45b4-a4c6-fc6a72556516\") " pod="openshift-ingress-canary/ingress-canary-48xtp" Jun 15 01:45:11.868050 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.867925 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/26023146-86ca-4a77-b73c-429dd24c4e5d-image-registry-private-configuration\") pod \"image-registry-5b5c665f7c-7mtmh\" (UID: \"26023146-86ca-4a77-b73c-429dd24c4e5d\") " pod="openshift-image-registry/image-registry-5b5c665f7c-7mtmh" Jun 15 01:45:11.872224 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.872199 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjbkh\" (UniqueName: \"kubernetes.io/projected/878da7f4-63c4-45b4-a4c6-fc6a72556516-kube-api-access-mjbkh\") pod \"ingress-canary-48xtp\" (UID: \"878da7f4-63c4-45b4-a4c6-fc6a72556516\") " pod="openshift-ingress-canary/ingress-canary-48xtp" Jun 15 01:45:11.872815 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.872798 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kh97b\" (UniqueName: \"kubernetes.io/projected/7f6afa77-1931-4efb-aeae-7a7fdef97959-kube-api-access-kh97b\") pod \"dns-default-bmrhc\" (UID: \"7f6afa77-1931-4efb-aeae-7a7fdef97959\") " pod="openshift-dns/dns-default-bmrhc" Jun 15 01:45:11.873956 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.873931 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l87tg\" (UniqueName: \"kubernetes.io/projected/26023146-86ca-4a77-b73c-429dd24c4e5d-kube-api-access-l87tg\") pod \"image-registry-5b5c665f7c-7mtmh\" (UID: \"26023146-86ca-4a77-b73c-429dd24c4e5d\") " pod="openshift-image-registry/image-registry-5b5c665f7c-7mtmh" Jun 15 01:45:11.877128 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.877105 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/26023146-86ca-4a77-b73c-429dd24c4e5d-bound-sa-token\") pod \"image-registry-5b5c665f7c-7mtmh\" (UID: \"26023146-86ca-4a77-b73c-429dd24c4e5d\") " pod="openshift-image-registry/image-registry-5b5c665f7c-7mtmh" Jun 15 01:45:11.934097 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:11.934057 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-48xtp" Jun 15 01:45:12.369051 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:12.369013 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/26023146-86ca-4a77-b73c-429dd24c4e5d-registry-tls\") pod \"image-registry-5b5c665f7c-7mtmh\" (UID: \"26023146-86ca-4a77-b73c-429dd24c4e5d\") " pod="openshift-image-registry/image-registry-5b5c665f7c-7mtmh" Jun 15 01:45:12.369258 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:12.369123 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7f6afa77-1931-4efb-aeae-7a7fdef97959-metrics-tls\") pod \"dns-default-bmrhc\" (UID: \"7f6afa77-1931-4efb-aeae-7a7fdef97959\") " pod="openshift-dns/dns-default-bmrhc" Jun 15 01:45:12.369258 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:12.369197 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 15 01:45:12.369258 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:12.369217 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b5c665f7c-7mtmh: secret "image-registry-tls" not found Jun 15 01:45:12.369258 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:12.369252 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 15 01:45:12.369477 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:12.369293 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/26023146-86ca-4a77-b73c-429dd24c4e5d-registry-tls podName:26023146-86ca-4a77-b73c-429dd24c4e5d nodeName:}" failed. No retries permitted until 2026-06-15 01:45:13.369276026 +0000 UTC m=+37.083001978 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/26023146-86ca-4a77-b73c-429dd24c4e5d-registry-tls") pod "image-registry-5b5c665f7c-7mtmh" (UID: "26023146-86ca-4a77-b73c-429dd24c4e5d") : secret "image-registry-tls" not found Jun 15 01:45:12.369477 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:12.369315 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f6afa77-1931-4efb-aeae-7a7fdef97959-metrics-tls podName:7f6afa77-1931-4efb-aeae-7a7fdef97959 nodeName:}" failed. No retries permitted until 2026-06-15 01:45:13.369301788 +0000 UTC m=+37.083027738 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7f6afa77-1931-4efb-aeae-7a7fdef97959-metrics-tls") pod "dns-default-bmrhc" (UID: "7f6afa77-1931-4efb-aeae-7a7fdef97959") : secret "dns-default-metrics-tls" not found Jun 15 01:45:12.760775 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:12.760734 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4p82" Jun 15 01:45:12.763795 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:12.763769 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 15 01:45:12.763951 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:12.763775 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-d5972\"" Jun 15 01:45:13.376716 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:13.376675 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/26023146-86ca-4a77-b73c-429dd24c4e5d-registry-tls\") pod \"image-registry-5b5c665f7c-7mtmh\" (UID: \"26023146-86ca-4a77-b73c-429dd24c4e5d\") " pod="openshift-image-registry/image-registry-5b5c665f7c-7mtmh" Jun 15 01:45:13.376879 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:13.376799 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7f6afa77-1931-4efb-aeae-7a7fdef97959-metrics-tls\") pod \"dns-default-bmrhc\" (UID: \"7f6afa77-1931-4efb-aeae-7a7fdef97959\") " pod="openshift-dns/dns-default-bmrhc" Jun 15 01:45:13.376879 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:13.376826 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 15 01:45:13.376879 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:13.376850 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b5c665f7c-7mtmh: secret "image-registry-tls" not found Jun 15 01:45:13.377028 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:13.376908 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/26023146-86ca-4a77-b73c-429dd24c4e5d-registry-tls podName:26023146-86ca-4a77-b73c-429dd24c4e5d nodeName:}" failed. No retries permitted until 2026-06-15 01:45:15.376892816 +0000 UTC m=+39.090618769 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/26023146-86ca-4a77-b73c-429dd24c4e5d-registry-tls") pod "image-registry-5b5c665f7c-7mtmh" (UID: "26023146-86ca-4a77-b73c-429dd24c4e5d") : secret "image-registry-tls" not found Jun 15 01:45:13.377028 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:13.376961 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 15 01:45:13.377028 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:13.377025 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f6afa77-1931-4efb-aeae-7a7fdef97959-metrics-tls podName:7f6afa77-1931-4efb-aeae-7a7fdef97959 nodeName:}" failed. No retries permitted until 2026-06-15 01:45:15.377008564 +0000 UTC m=+39.090734528 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7f6afa77-1931-4efb-aeae-7a7fdef97959-metrics-tls") pod "dns-default-bmrhc" (UID: "7f6afa77-1931-4efb-aeae-7a7fdef97959") : secret "dns-default-metrics-tls" not found Jun 15 01:45:13.516643 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:13.516611 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-48xtp"] Jun 15 01:45:13.568666 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:45:13.568639 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod878da7f4_63c4_45b4_a4c6_fc6a72556516.slice/crio-1c8f474a535f3153d0d2754657e991d61917e08cf37c8b17ce3fee9c7c61be84 WatchSource:0}: Error finding container 1c8f474a535f3153d0d2754657e991d61917e08cf37c8b17ce3fee9c7c61be84: Status 404 returned error can't find the container with id 1c8f474a535f3153d0d2754657e991d61917e08cf37c8b17ce3fee9c7c61be84 Jun 15 01:45:14.028773 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:14.028524 2578 generic.go:356] "Generic (PLEG): container finished" podID="9a743adf-5ee6-4357-9206-1ccada80d911" containerID="9a6eb60f3f87a9f0c837824300b3cccf18fef706e7b82fd57c05c459325d46fd" exitCode=0 Jun 15 01:45:14.029233 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:14.028604 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8r9kz" event={"ID":"9a743adf-5ee6-4357-9206-1ccada80d911","Type":"ContainerDied","Data":"9a6eb60f3f87a9f0c837824300b3cccf18fef706e7b82fd57c05c459325d46fd"} Jun 15 01:45:14.030178 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:14.030146 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-48xtp" event={"ID":"878da7f4-63c4-45b4-a4c6-fc6a72556516","Type":"ContainerStarted","Data":"1c8f474a535f3153d0d2754657e991d61917e08cf37c8b17ce3fee9c7c61be84"} Jun 15 01:45:15.036633 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:15.036507 2578 generic.go:356] "Generic (PLEG): container finished" podID="9a743adf-5ee6-4357-9206-1ccada80d911" containerID="bbd452a084da89ed4d090ae3dbcf6ddb471e2415f455b04dd279803bc4d93093" exitCode=0 Jun 15 01:45:15.036633 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:15.036580 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8r9kz" event={"ID":"9a743adf-5ee6-4357-9206-1ccada80d911","Type":"ContainerDied","Data":"bbd452a084da89ed4d090ae3dbcf6ddb471e2415f455b04dd279803bc4d93093"} Jun 15 01:45:15.393454 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:15.393417 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/26023146-86ca-4a77-b73c-429dd24c4e5d-registry-tls\") pod \"image-registry-5b5c665f7c-7mtmh\" (UID: \"26023146-86ca-4a77-b73c-429dd24c4e5d\") " pod="openshift-image-registry/image-registry-5b5c665f7c-7mtmh" Jun 15 01:45:15.393623 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:15.393486 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7f6afa77-1931-4efb-aeae-7a7fdef97959-metrics-tls\") pod \"dns-default-bmrhc\" (UID: \"7f6afa77-1931-4efb-aeae-7a7fdef97959\") " pod="openshift-dns/dns-default-bmrhc" Jun 15 01:45:15.393623 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:15.393585 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 15 01:45:15.393623 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:15.393603 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 15 01:45:15.393762 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:15.393631 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b5c665f7c-7mtmh: secret "image-registry-tls" not found Jun 15 01:45:15.393762 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:15.393638 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f6afa77-1931-4efb-aeae-7a7fdef97959-metrics-tls podName:7f6afa77-1931-4efb-aeae-7a7fdef97959 nodeName:}" failed. No retries permitted until 2026-06-15 01:45:19.39362534 +0000 UTC m=+43.107351290 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7f6afa77-1931-4efb-aeae-7a7fdef97959-metrics-tls") pod "dns-default-bmrhc" (UID: "7f6afa77-1931-4efb-aeae-7a7fdef97959") : secret "dns-default-metrics-tls" not found Jun 15 01:45:15.393762 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:15.393691 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/26023146-86ca-4a77-b73c-429dd24c4e5d-registry-tls podName:26023146-86ca-4a77-b73c-429dd24c4e5d nodeName:}" failed. No retries permitted until 2026-06-15 01:45:19.393673759 +0000 UTC m=+43.107399725 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/26023146-86ca-4a77-b73c-429dd24c4e5d-registry-tls") pod "image-registry-5b5c665f7c-7mtmh" (UID: "26023146-86ca-4a77-b73c-429dd24c4e5d") : secret "image-registry-tls" not found Jun 15 01:45:16.042254 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:16.042226 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8r9kz" event={"ID":"9a743adf-5ee6-4357-9206-1ccada80d911","Type":"ContainerStarted","Data":"283f53d69ded76098b8630de9ed0f70cc65e519e4e515e3a1ff4b5fa22e05935"} Jun 15 01:45:16.043674 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:16.043653 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-48xtp" event={"ID":"878da7f4-63c4-45b4-a4c6-fc6a72556516","Type":"ContainerStarted","Data":"72be2b46c028263cd87c388106e30466b24d8cf383ad176ea5a231b5af5140fb"} Jun 15 01:45:16.066261 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:16.066208 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-8r9kz" podStartSLOduration=8.818066573 podStartE2EDuration="39.066192737s" totalImagesPullingTime="30.248126164s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 01:44:37 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 01:45:16.06440004 +0000 UTC m=+39.778126012" watchObservedRunningTime="2026-06-15 01:45:16.066192737 +0000 UTC m=+39.779918709" Jun 15 01:45:16.079432 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:16.079385 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-48xtp" podStartSLOduration=3.413020918 podStartE2EDuration="5.079370653s" totalImagesPullingTime="1.666349735s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 01:45:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 01:45:16.079178191 +0000 UTC m=+39.792904163" watchObservedRunningTime="2026-06-15 01:45:16.079370653 +0000 UTC m=+39.793096667" Jun 15 01:45:19.424334 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:19.424291 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/26023146-86ca-4a77-b73c-429dd24c4e5d-registry-tls\") pod \"image-registry-5b5c665f7c-7mtmh\" (UID: \"26023146-86ca-4a77-b73c-429dd24c4e5d\") " pod="openshift-image-registry/image-registry-5b5c665f7c-7mtmh" Jun 15 01:45:19.424716 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:19.424376 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7f6afa77-1931-4efb-aeae-7a7fdef97959-metrics-tls\") pod \"dns-default-bmrhc\" (UID: \"7f6afa77-1931-4efb-aeae-7a7fdef97959\") " pod="openshift-dns/dns-default-bmrhc" Jun 15 01:45:19.424716 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:19.424453 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 15 01:45:19.424716 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:19.424472 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b5c665f7c-7mtmh: secret "image-registry-tls" not found Jun 15 01:45:19.424716 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:19.424483 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 15 01:45:19.424716 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:19.424533 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f6afa77-1931-4efb-aeae-7a7fdef97959-metrics-tls podName:7f6afa77-1931-4efb-aeae-7a7fdef97959 nodeName:}" failed. No retries permitted until 2026-06-15 01:45:27.42451891 +0000 UTC m=+51.138244862 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7f6afa77-1931-4efb-aeae-7a7fdef97959-metrics-tls") pod "dns-default-bmrhc" (UID: "7f6afa77-1931-4efb-aeae-7a7fdef97959") : secret "dns-default-metrics-tls" not found Jun 15 01:45:19.424716 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:19.424547 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/26023146-86ca-4a77-b73c-429dd24c4e5d-registry-tls podName:26023146-86ca-4a77-b73c-429dd24c4e5d nodeName:}" failed. No retries permitted until 2026-06-15 01:45:27.424541485 +0000 UTC m=+51.138267435 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/26023146-86ca-4a77-b73c-429dd24c4e5d-registry-tls") pod "image-registry-5b5c665f7c-7mtmh" (UID: "26023146-86ca-4a77-b73c-429dd24c4e5d") : secret "image-registry-tls" not found Jun 15 01:45:27.481004 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:27.480961 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/26023146-86ca-4a77-b73c-429dd24c4e5d-registry-tls\") pod \"image-registry-5b5c665f7c-7mtmh\" (UID: \"26023146-86ca-4a77-b73c-429dd24c4e5d\") " pod="openshift-image-registry/image-registry-5b5c665f7c-7mtmh" Jun 15 01:45:27.481554 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:27.481044 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7f6afa77-1931-4efb-aeae-7a7fdef97959-metrics-tls\") pod \"dns-default-bmrhc\" (UID: \"7f6afa77-1931-4efb-aeae-7a7fdef97959\") " pod="openshift-dns/dns-default-bmrhc" Jun 15 01:45:27.481554 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:27.481139 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 15 01:45:27.481554 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:27.481153 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b5c665f7c-7mtmh: secret "image-registry-tls" not found Jun 15 01:45:27.481554 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:27.481204 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/26023146-86ca-4a77-b73c-429dd24c4e5d-registry-tls podName:26023146-86ca-4a77-b73c-429dd24c4e5d nodeName:}" failed. No retries permitted until 2026-06-15 01:45:43.481190668 +0000 UTC m=+67.194916617 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/26023146-86ca-4a77-b73c-429dd24c4e5d-registry-tls") pod "image-registry-5b5c665f7c-7mtmh" (UID: "26023146-86ca-4a77-b73c-429dd24c4e5d") : secret "image-registry-tls" not found Jun 15 01:45:27.481554 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:27.481218 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 15 01:45:27.481554 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:27.481331 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f6afa77-1931-4efb-aeae-7a7fdef97959-metrics-tls podName:7f6afa77-1931-4efb-aeae-7a7fdef97959 nodeName:}" failed. No retries permitted until 2026-06-15 01:45:43.481312071 +0000 UTC m=+67.195038022 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7f6afa77-1931-4efb-aeae-7a7fdef97959-metrics-tls") pod "dns-default-bmrhc" (UID: "7f6afa77-1931-4efb-aeae-7a7fdef97959") : secret "dns-default-metrics-tls" not found Jun 15 01:45:38.031433 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:38.031405 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cf286" Jun 15 01:45:42.478470 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:42.478426 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1719195-2a1b-4c96-abe3-c25bda154d3d-metrics-certs\") pod \"network-metrics-daemon-n4p82\" (UID: \"e1719195-2a1b-4c96-abe3-c25bda154d3d\") " pod="openshift-multus/network-metrics-daemon-n4p82" Jun 15 01:45:42.481454 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:42.481435 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 15 01:45:42.488770 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:42.488751 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 15 01:45:42.488846 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:42.488806 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1719195-2a1b-4c96-abe3-c25bda154d3d-metrics-certs podName:e1719195-2a1b-4c96-abe3-c25bda154d3d nodeName:}" failed. No retries permitted until 2026-06-15 01:46:46.488790801 +0000 UTC m=+130.202516751 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e1719195-2a1b-4c96-abe3-c25bda154d3d-metrics-certs") pod "network-metrics-daemon-n4p82" (UID: "e1719195-2a1b-4c96-abe3-c25bda154d3d") : secret "metrics-daemon-secret" not found Jun 15 01:45:42.579609 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:42.579562 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a6368c58-4355-4d1c-b621-75c5bf0e7394-original-pull-secret\") pod \"global-pull-secret-syncer-6272r\" (UID: \"a6368c58-4355-4d1c-b621-75c5bf0e7394\") " pod="kube-system/global-pull-secret-syncer-6272r" Jun 15 01:45:42.582558 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:42.582540 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 15 01:45:42.593316 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:42.593277 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a6368c58-4355-4d1c-b621-75c5bf0e7394-original-pull-secret\") pod \"global-pull-secret-syncer-6272r\" (UID: \"a6368c58-4355-4d1c-b621-75c5bf0e7394\") " pod="kube-system/global-pull-secret-syncer-6272r" Jun 15 01:45:42.670262 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:42.670227 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6272r" Jun 15 01:45:42.680563 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:42.680371 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnvlx\" (UniqueName: \"kubernetes.io/projected/a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1-kube-api-access-fnvlx\") pod \"network-check-target-f2kkk\" (UID: \"a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1\") " pod="openshift-network-diagnostics/network-check-target-f2kkk" Jun 15 01:45:42.683319 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:42.683299 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 15 01:45:42.693856 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:42.693834 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 15 01:45:42.704732 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:42.704710 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnvlx\" (UniqueName: \"kubernetes.io/projected/a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1-kube-api-access-fnvlx\") pod \"network-check-target-f2kkk\" (UID: \"a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1\") " pod="openshift-network-diagnostics/network-check-target-f2kkk" Jun 15 01:45:42.793592 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:42.793559 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6272r"] Jun 15 01:45:42.796141 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:45:42.796114 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6368c58_4355_4d1c_b621_75c5bf0e7394.slice/crio-dd3c6bc6abcf78b8aa3218acb91eed3e29f1f08e8ae059c8b2614dcd300a538c WatchSource:0}: Error finding container dd3c6bc6abcf78b8aa3218acb91eed3e29f1f08e8ae059c8b2614dcd300a538c: Status 404 returned error can't find the container with id dd3c6bc6abcf78b8aa3218acb91eed3e29f1f08e8ae059c8b2614dcd300a538c Jun 15 01:45:42.976799 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:42.976769 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7pbsq\"" Jun 15 01:45:42.984501 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:42.984479 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2kkk" Jun 15 01:45:43.107531 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:43.107503 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-f2kkk"] Jun 15 01:45:43.110077 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:45:43.110052 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3b6b1f6_ec13_486c_9fda_0ab5ebc475a1.slice/crio-5e2751caa1a5e65a041d2927c652483761d31d4cc9018819e3348b14d3e4f0b1 WatchSource:0}: Error finding container 5e2751caa1a5e65a041d2927c652483761d31d4cc9018819e3348b14d3e4f0b1: Status 404 returned error can't find the container with id 5e2751caa1a5e65a041d2927c652483761d31d4cc9018819e3348b14d3e4f0b1 Jun 15 01:45:43.110760 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:43.110669 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6272r" event={"ID":"a6368c58-4355-4d1c-b621-75c5bf0e7394","Type":"ContainerStarted","Data":"dd3c6bc6abcf78b8aa3218acb91eed3e29f1f08e8ae059c8b2614dcd300a538c"} Jun 15 01:45:43.485411 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:43.485373 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/26023146-86ca-4a77-b73c-429dd24c4e5d-registry-tls\") pod \"image-registry-5b5c665f7c-7mtmh\" (UID: \"26023146-86ca-4a77-b73c-429dd24c4e5d\") " pod="openshift-image-registry/image-registry-5b5c665f7c-7mtmh" Jun 15 01:45:43.485854 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:43.485431 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7f6afa77-1931-4efb-aeae-7a7fdef97959-metrics-tls\") pod \"dns-default-bmrhc\" (UID: \"7f6afa77-1931-4efb-aeae-7a7fdef97959\") " pod="openshift-dns/dns-default-bmrhc" Jun 15 01:45:43.485854 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:43.485537 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 15 01:45:43.485854 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:43.485550 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 15 01:45:43.485854 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:43.485576 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b5c665f7c-7mtmh: secret "image-registry-tls" not found Jun 15 01:45:43.485854 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:43.485603 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f6afa77-1931-4efb-aeae-7a7fdef97959-metrics-tls podName:7f6afa77-1931-4efb-aeae-7a7fdef97959 nodeName:}" failed. No retries permitted until 2026-06-15 01:46:15.485586414 +0000 UTC m=+99.199312576 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7f6afa77-1931-4efb-aeae-7a7fdef97959-metrics-tls") pod "dns-default-bmrhc" (UID: "7f6afa77-1931-4efb-aeae-7a7fdef97959") : secret "dns-default-metrics-tls" not found Jun 15 01:45:43.485854 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:45:43.485629 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/26023146-86ca-4a77-b73c-429dd24c4e5d-registry-tls podName:26023146-86ca-4a77-b73c-429dd24c4e5d nodeName:}" failed. No retries permitted until 2026-06-15 01:46:15.485612284 +0000 UTC m=+99.199338245 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/26023146-86ca-4a77-b73c-429dd24c4e5d-registry-tls") pod "image-registry-5b5c665f7c-7mtmh" (UID: "26023146-86ca-4a77-b73c-429dd24c4e5d") : secret "image-registry-tls" not found Jun 15 01:45:44.115145 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:44.115104 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-f2kkk" event={"ID":"a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1","Type":"ContainerStarted","Data":"5e2751caa1a5e65a041d2927c652483761d31d4cc9018819e3348b14d3e4f0b1"} Jun 15 01:45:48.126418 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:48.126377 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-f2kkk" event={"ID":"a3b6b1f6-ec13-486c-9fda-0ab5ebc475a1","Type":"ContainerStarted","Data":"61de630dacf1b9240a2bde315784af779af52f8f6e20bfc2098788200ee81ae0"} Jun 15 01:45:48.126858 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:48.126484 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-f2kkk" Jun 15 01:45:48.127864 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:48.127842 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6272r" event={"ID":"a6368c58-4355-4d1c-b621-75c5bf0e7394","Type":"ContainerStarted","Data":"2ff5bd5eb9d639b61e2878e8065c0b13c06592b5af3969e5c2003db9570542fc"} Jun 15 01:45:48.142850 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:48.142794 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-f2kkk" podStartSLOduration=66.852484832 podStartE2EDuration="1m11.142780283s" totalImagesPullingTime="4.290295451s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 01:44:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 01:45:48.141371694 +0000 UTC m=+71.855097666" watchObservedRunningTime="2026-06-15 01:45:48.142780283 +0000 UTC m=+71.856506255" Jun 15 01:45:48.155166 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:45:48.155119 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-6272r" podStartSLOduration=66.545038758 podStartE2EDuration="1m11.155105666s" totalImagesPullingTime="4.610066908s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 01:44:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 01:45:48.15470938 +0000 UTC m=+71.868435353" watchObservedRunningTime="2026-06-15 01:45:48.155105666 +0000 UTC m=+71.868831632" Jun 15 01:46:00.297333 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:46:00.297292 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-844c6877cd-8v6bp"] Jun 15 01:46:00.299854 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:46:00.299834 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-844c6877cd-8v6bp" Jun 15 01:46:00.302268 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:46:00.302244 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jun 15 01:46:00.302510 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:46:00.302487 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jun 15 01:46:00.302608 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:46:00.302538 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jun 15 01:46:00.303801 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:46:00.303782 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jun 15 01:46:00.307953 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:46:00.307934 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-844c6877cd-8v6bp"] Jun 15 01:46:00.402675 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:46:00.402636 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/af636ff6-b9f5-4871-a86c-9ab3788dcd76-klusterlet-config\") pod \"klusterlet-addon-workmgr-844c6877cd-8v6bp\" (UID: \"af636ff6-b9f5-4871-a86c-9ab3788dcd76\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-844c6877cd-8v6bp" Jun 15 01:46:00.402675 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:46:00.402674 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjrmw\" (UniqueName: \"kubernetes.io/projected/af636ff6-b9f5-4871-a86c-9ab3788dcd76-kube-api-access-bjrmw\") pod \"klusterlet-addon-workmgr-844c6877cd-8v6bp\" (UID: \"af636ff6-b9f5-4871-a86c-9ab3788dcd76\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-844c6877cd-8v6bp" Jun 15 01:46:00.402895 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:46:00.402705 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/af636ff6-b9f5-4871-a86c-9ab3788dcd76-tmp\") pod \"klusterlet-addon-workmgr-844c6877cd-8v6bp\" (UID: \"af636ff6-b9f5-4871-a86c-9ab3788dcd76\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-844c6877cd-8v6bp" Jun 15 01:46:00.503698 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:46:00.503652 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/af636ff6-b9f5-4871-a86c-9ab3788dcd76-klusterlet-config\") pod \"klusterlet-addon-workmgr-844c6877cd-8v6bp\" (UID: \"af636ff6-b9f5-4871-a86c-9ab3788dcd76\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-844c6877cd-8v6bp" Jun 15 01:46:00.503698 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:46:00.503698 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjrmw\" (UniqueName: \"kubernetes.io/projected/af636ff6-b9f5-4871-a86c-9ab3788dcd76-kube-api-access-bjrmw\") pod \"klusterlet-addon-workmgr-844c6877cd-8v6bp\" (UID: \"af636ff6-b9f5-4871-a86c-9ab3788dcd76\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-844c6877cd-8v6bp" Jun 15 01:46:00.503954 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:46:00.503727 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/af636ff6-b9f5-4871-a86c-9ab3788dcd76-tmp\") pod \"klusterlet-addon-workmgr-844c6877cd-8v6bp\" (UID: \"af636ff6-b9f5-4871-a86c-9ab3788dcd76\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-844c6877cd-8v6bp" Jun 15 01:46:00.504060 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:46:00.504043 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/af636ff6-b9f5-4871-a86c-9ab3788dcd76-tmp\") pod \"klusterlet-addon-workmgr-844c6877cd-8v6bp\" (UID: \"af636ff6-b9f5-4871-a86c-9ab3788dcd76\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-844c6877cd-8v6bp" Jun 15 01:46:00.506371 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:46:00.506346 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/af636ff6-b9f5-4871-a86c-9ab3788dcd76-klusterlet-config\") pod \"klusterlet-addon-workmgr-844c6877cd-8v6bp\" (UID: \"af636ff6-b9f5-4871-a86c-9ab3788dcd76\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-844c6877cd-8v6bp" Jun 15 01:46:00.512337 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:46:00.512316 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjrmw\" (UniqueName: \"kubernetes.io/projected/af636ff6-b9f5-4871-a86c-9ab3788dcd76-kube-api-access-bjrmw\") pod \"klusterlet-addon-workmgr-844c6877cd-8v6bp\" (UID: \"af636ff6-b9f5-4871-a86c-9ab3788dcd76\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-844c6877cd-8v6bp" Jun 15 01:46:00.609309 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:46:00.609201 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-844c6877cd-8v6bp" Jun 15 01:46:00.731581 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:46:00.731551 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-844c6877cd-8v6bp"] Jun 15 01:46:00.734651 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:46:00.734615 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf636ff6_b9f5_4871_a86c_9ab3788dcd76.slice/crio-9ec088efca571beb5c491c207f09a8fc35eb829ff7bd54b7aa5c4156081f69be WatchSource:0}: Error finding container 9ec088efca571beb5c491c207f09a8fc35eb829ff7bd54b7aa5c4156081f69be: Status 404 returned error can't find the container with id 9ec088efca571beb5c491c207f09a8fc35eb829ff7bd54b7aa5c4156081f69be Jun 15 01:46:01.159232 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:46:01.159198 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-844c6877cd-8v6bp" event={"ID":"af636ff6-b9f5-4871-a86c-9ab3788dcd76","Type":"ContainerStarted","Data":"9ec088efca571beb5c491c207f09a8fc35eb829ff7bd54b7aa5c4156081f69be"} Jun 15 01:46:15.509955 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:46:15.509915 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/26023146-86ca-4a77-b73c-429dd24c4e5d-registry-tls\") pod \"image-registry-5b5c665f7c-7mtmh\" (UID: \"26023146-86ca-4a77-b73c-429dd24c4e5d\") " pod="openshift-image-registry/image-registry-5b5c665f7c-7mtmh" Jun 15 01:46:15.510362 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:46:15.509986 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7f6afa77-1931-4efb-aeae-7a7fdef97959-metrics-tls\") pod \"dns-default-bmrhc\" (UID: \"7f6afa77-1931-4efb-aeae-7a7fdef97959\") " pod="openshift-dns/dns-default-bmrhc" Jun 15 01:46:15.510362 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:46:15.510074 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 15 01:46:15.510362 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:46:15.510077 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 15 01:46:15.510362 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:46:15.510117 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b5c665f7c-7mtmh: secret "image-registry-tls" not found Jun 15 01:46:15.510362 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:46:15.510169 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/26023146-86ca-4a77-b73c-429dd24c4e5d-registry-tls podName:26023146-86ca-4a77-b73c-429dd24c4e5d nodeName:}" failed. No retries permitted until 2026-06-15 01:47:19.510150934 +0000 UTC m=+163.223876883 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/26023146-86ca-4a77-b73c-429dd24c4e5d-registry-tls") pod "image-registry-5b5c665f7c-7mtmh" (UID: "26023146-86ca-4a77-b73c-429dd24c4e5d") : secret "image-registry-tls" not found Jun 15 01:46:15.510362 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:46:15.510186 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f6afa77-1931-4efb-aeae-7a7fdef97959-metrics-tls podName:7f6afa77-1931-4efb-aeae-7a7fdef97959 nodeName:}" failed. No retries permitted until 2026-06-15 01:47:19.510178225 +0000 UTC m=+163.223904175 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7f6afa77-1931-4efb-aeae-7a7fdef97959-metrics-tls") pod "dns-default-bmrhc" (UID: "7f6afa77-1931-4efb-aeae-7a7fdef97959") : secret "dns-default-metrics-tls" not found Jun 15 01:46:19.132906 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:46:19.132877 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-f2kkk" Jun 15 01:46:36.246811 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:46:36.246771 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-844c6877cd-8v6bp" event={"ID":"af636ff6-b9f5-4871-a86c-9ab3788dcd76","Type":"ContainerStarted","Data":"6a4fabf29175d8cfcb72c39aaef355ba70d2b2ef08a9479041fbead444b4bf5b"} Jun 15 01:46:36.247303 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:46:36.247040 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-844c6877cd-8v6bp" Jun 15 01:46:36.248708 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:46:36.248687 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-844c6877cd-8v6bp" Jun 15 01:46:36.262697 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:46:36.262655 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-844c6877cd-8v6bp" podStartSLOduration=1.668535594 podStartE2EDuration="36.262644014s" totalImagesPullingTime="34.59410842s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 01:46:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 01:46:36.261644921 +0000 UTC m=+119.975370892" watchObservedRunningTime="2026-06-15 01:46:36.262644014 +0000 UTC m=+119.976370034" Jun 15 01:46:46.528589 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:46:46.528549 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1719195-2a1b-4c96-abe3-c25bda154d3d-metrics-certs\") pod \"network-metrics-daemon-n4p82\" (UID: \"e1719195-2a1b-4c96-abe3-c25bda154d3d\") " pod="openshift-multus/network-metrics-daemon-n4p82" Jun 15 01:46:46.528997 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:46:46.528693 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 15 01:46:46.528997 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:46:46.528765 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1719195-2a1b-4c96-abe3-c25bda154d3d-metrics-certs podName:e1719195-2a1b-4c96-abe3-c25bda154d3d nodeName:}" failed. No retries permitted until 2026-06-15 01:48:48.528747616 +0000 UTC m=+252.242473571 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e1719195-2a1b-4c96-abe3-c25bda154d3d-metrics-certs") pod "network-metrics-daemon-n4p82" (UID: "e1719195-2a1b-4c96-abe3-c25bda154d3d") : secret "metrics-daemon-secret" not found Jun 15 01:47:14.612772 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:47:14.612723 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-5b5c665f7c-7mtmh" podUID="26023146-86ca-4a77-b73c-429dd24c4e5d" Jun 15 01:47:14.626069 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:47:14.626030 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-bmrhc" podUID="7f6afa77-1931-4efb-aeae-7a7fdef97959" Jun 15 01:47:15.337622 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:47:15.337586 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bmrhc" Jun 15 01:47:15.337804 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:47:15.337586 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b5c665f7c-7mtmh" Jun 15 01:47:15.772330 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:47:15.772292 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-n4p82" podUID="e1719195-2a1b-4c96-abe3-c25bda154d3d" Jun 15 01:47:19.570702 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:47:19.570644 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/26023146-86ca-4a77-b73c-429dd24c4e5d-registry-tls\") pod \"image-registry-5b5c665f7c-7mtmh\" (UID: \"26023146-86ca-4a77-b73c-429dd24c4e5d\") " pod="openshift-image-registry/image-registry-5b5c665f7c-7mtmh" Jun 15 01:47:19.571111 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:47:19.570727 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7f6afa77-1931-4efb-aeae-7a7fdef97959-metrics-tls\") pod \"dns-default-bmrhc\" (UID: \"7f6afa77-1931-4efb-aeae-7a7fdef97959\") " pod="openshift-dns/dns-default-bmrhc" Jun 15 01:47:19.571111 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:47:19.570794 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 15 01:47:19.571111 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:47:19.570812 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 15 01:47:19.571111 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:47:19.570815 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b5c665f7c-7mtmh: secret "image-registry-tls" not found Jun 15 01:47:19.571111 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:47:19.570862 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f6afa77-1931-4efb-aeae-7a7fdef97959-metrics-tls podName:7f6afa77-1931-4efb-aeae-7a7fdef97959 nodeName:}" failed. No retries permitted until 2026-06-15 01:49:21.570847804 +0000 UTC m=+285.284573754 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7f6afa77-1931-4efb-aeae-7a7fdef97959-metrics-tls") pod "dns-default-bmrhc" (UID: "7f6afa77-1931-4efb-aeae-7a7fdef97959") : secret "dns-default-metrics-tls" not found Jun 15 01:47:19.571111 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:47:19.570875 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/26023146-86ca-4a77-b73c-429dd24c4e5d-registry-tls podName:26023146-86ca-4a77-b73c-429dd24c4e5d nodeName:}" failed. No retries permitted until 2026-06-15 01:49:21.570869016 +0000 UTC m=+285.284594965 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/26023146-86ca-4a77-b73c-429dd24c4e5d-registry-tls") pod "image-registry-5b5c665f7c-7mtmh" (UID: "26023146-86ca-4a77-b73c-429dd24c4e5d") : secret "image-registry-tls" not found Jun 15 01:47:28.760591 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:47:28.760552 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4p82" Jun 15 01:48:10.579188 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:10.579149 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-58cf549cb4-mc482"] Jun 15 01:48:10.581114 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:10.581079 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-58cf549cb4-mc482" Jun 15 01:48:10.584522 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:10.584500 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 15 01:48:10.584522 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:10.584516 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jun 15 01:48:10.584681 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:10.584524 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jun 15 01:48:10.585277 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:10.585260 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jun 15 01:48:10.585375 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:10.585283 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jun 15 01:48:10.585375 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:10.585263 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 15 01:48:10.585599 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:10.585584 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-kfk6k\"" Jun 15 01:48:10.604323 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:10.604295 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-58cf549cb4-mc482"] Jun 15 01:48:10.670327 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:10.670288 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6947749cfc-k74jj"] Jun 15 01:48:10.672035 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:10.672014 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6947749cfc-k74jj" Jun 15 01:48:10.674669 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:10.674650 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-f8zzp\"" Jun 15 01:48:10.677680 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:10.677659 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-w8h9d"] Jun 15 01:48:10.679637 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:10.679622 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-w8h9d" Jun 15 01:48:10.682850 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:10.682832 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jun 15 01:48:10.683201 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:10.683163 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jun 15 01:48:10.683312 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:10.683293 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jun 15 01:48:10.683432 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:10.683410 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-85474\"" Jun 15 01:48:10.684856 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:10.684836 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6947749cfc-k74jj"] Jun 15 01:48:10.700422 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:10.700402 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-w8h9d"] Jun 15 01:48:10.737504 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:10.737474 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4979e05-89c5-482b-a895-4f1f9ae54e93-service-ca-bundle\") pod \"router-default-58cf549cb4-mc482\" (UID: \"d4979e05-89c5-482b-a895-4f1f9ae54e93\") " pod="openshift-ingress/router-default-58cf549cb4-mc482" Jun 15 01:48:10.737637 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:10.737529 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d4979e05-89c5-482b-a895-4f1f9ae54e93-default-certificate\") pod \"router-default-58cf549cb4-mc482\" (UID: \"d4979e05-89c5-482b-a895-4f1f9ae54e93\") " pod="openshift-ingress/router-default-58cf549cb4-mc482" Jun 15 01:48:10.737637 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:10.737551 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8ksc\" (UniqueName: \"kubernetes.io/projected/d4979e05-89c5-482b-a895-4f1f9ae54e93-kube-api-access-f8ksc\") pod \"router-default-58cf549cb4-mc482\" (UID: \"d4979e05-89c5-482b-a895-4f1f9ae54e93\") " pod="openshift-ingress/router-default-58cf549cb4-mc482" Jun 15 01:48:10.737637 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:10.737566 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4979e05-89c5-482b-a895-4f1f9ae54e93-metrics-certs\") pod \"router-default-58cf549cb4-mc482\" (UID: \"d4979e05-89c5-482b-a895-4f1f9ae54e93\") " pod="openshift-ingress/router-default-58cf549cb4-mc482" Jun 15 01:48:10.737637 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:10.737622 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d4979e05-89c5-482b-a895-4f1f9ae54e93-stats-auth\") pod \"router-default-58cf549cb4-mc482\" (UID: \"d4979e05-89c5-482b-a895-4f1f9ae54e93\") " pod="openshift-ingress/router-default-58cf549cb4-mc482" Jun 15 01:48:10.838045 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:10.837950 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4979e05-89c5-482b-a895-4f1f9ae54e93-service-ca-bundle\") pod \"router-default-58cf549cb4-mc482\" (UID: \"d4979e05-89c5-482b-a895-4f1f9ae54e93\") " pod="openshift-ingress/router-default-58cf549cb4-mc482" Jun 15 01:48:10.838045 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:10.838018 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d4979e05-89c5-482b-a895-4f1f9ae54e93-default-certificate\") pod \"router-default-58cf549cb4-mc482\" (UID: \"d4979e05-89c5-482b-a895-4f1f9ae54e93\") " pod="openshift-ingress/router-default-58cf549cb4-mc482" Jun 15 01:48:10.838265 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:10.838050 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhpf9\" (UniqueName: \"kubernetes.io/projected/a6f20e37-2612-4b8a-95a6-f4530b58144f-kube-api-access-lhpf9\") pod \"network-check-source-6947749cfc-k74jj\" (UID: \"a6f20e37-2612-4b8a-95a6-f4530b58144f\") " pod="openshift-network-diagnostics/network-check-source-6947749cfc-k74jj" Jun 15 01:48:10.838265 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:10.838079 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bp6l6\" (UniqueName: \"kubernetes.io/projected/a916f9e4-441d-4c65-bd95-be9c821970b3-kube-api-access-bp6l6\") pod \"cluster-samples-operator-9547488fd-w8h9d\" (UID: \"a916f9e4-441d-4c65-bd95-be9c821970b3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-w8h9d" Jun 15 01:48:10.838265 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:10.838152 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8ksc\" (UniqueName: \"kubernetes.io/projected/d4979e05-89c5-482b-a895-4f1f9ae54e93-kube-api-access-f8ksc\") pod \"router-default-58cf549cb4-mc482\" (UID: \"d4979e05-89c5-482b-a895-4f1f9ae54e93\") " pod="openshift-ingress/router-default-58cf549cb4-mc482" Jun 15 01:48:10.838265 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:48:10.838170 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d4979e05-89c5-482b-a895-4f1f9ae54e93-service-ca-bundle podName:d4979e05-89c5-482b-a895-4f1f9ae54e93 nodeName:}" failed. No retries permitted until 2026-06-15 01:48:11.338147642 +0000 UTC m=+215.051873596 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d4979e05-89c5-482b-a895-4f1f9ae54e93-service-ca-bundle") pod "router-default-58cf549cb4-mc482" (UID: "d4979e05-89c5-482b-a895-4f1f9ae54e93") : configmap references non-existent config key: service-ca.crt Jun 15 01:48:10.838265 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:10.838208 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a916f9e4-441d-4c65-bd95-be9c821970b3-samples-operator-tls\") pod \"cluster-samples-operator-9547488fd-w8h9d\" (UID: \"a916f9e4-441d-4c65-bd95-be9c821970b3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-w8h9d" Jun 15 01:48:10.838497 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:10.838332 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4979e05-89c5-482b-a895-4f1f9ae54e93-metrics-certs\") pod \"router-default-58cf549cb4-mc482\" (UID: \"d4979e05-89c5-482b-a895-4f1f9ae54e93\") " pod="openshift-ingress/router-default-58cf549cb4-mc482" Jun 15 01:48:10.838497 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:10.838397 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d4979e05-89c5-482b-a895-4f1f9ae54e93-stats-auth\") pod \"router-default-58cf549cb4-mc482\" (UID: \"d4979e05-89c5-482b-a895-4f1f9ae54e93\") " pod="openshift-ingress/router-default-58cf549cb4-mc482" Jun 15 01:48:10.838497 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:48:10.838415 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 15 01:48:10.838497 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:48:10.838478 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4979e05-89c5-482b-a895-4f1f9ae54e93-metrics-certs podName:d4979e05-89c5-482b-a895-4f1f9ae54e93 nodeName:}" failed. No retries permitted until 2026-06-15 01:48:11.338461112 +0000 UTC m=+215.052187065 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4979e05-89c5-482b-a895-4f1f9ae54e93-metrics-certs") pod "router-default-58cf549cb4-mc482" (UID: "d4979e05-89c5-482b-a895-4f1f9ae54e93") : secret "router-metrics-certs-default" not found Jun 15 01:48:10.840581 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:10.840554 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d4979e05-89c5-482b-a895-4f1f9ae54e93-default-certificate\") pod \"router-default-58cf549cb4-mc482\" (UID: \"d4979e05-89c5-482b-a895-4f1f9ae54e93\") " pod="openshift-ingress/router-default-58cf549cb4-mc482" Jun 15 01:48:10.840690 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:10.840624 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d4979e05-89c5-482b-a895-4f1f9ae54e93-stats-auth\") pod \"router-default-58cf549cb4-mc482\" (UID: \"d4979e05-89c5-482b-a895-4f1f9ae54e93\") " pod="openshift-ingress/router-default-58cf549cb4-mc482" Jun 15 01:48:10.847736 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:10.847710 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8ksc\" (UniqueName: \"kubernetes.io/projected/d4979e05-89c5-482b-a895-4f1f9ae54e93-kube-api-access-f8ksc\") pod \"router-default-58cf549cb4-mc482\" (UID: \"d4979e05-89c5-482b-a895-4f1f9ae54e93\") " pod="openshift-ingress/router-default-58cf549cb4-mc482" Jun 15 01:48:10.939056 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:10.939022 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhpf9\" (UniqueName: \"kubernetes.io/projected/a6f20e37-2612-4b8a-95a6-f4530b58144f-kube-api-access-lhpf9\") pod \"network-check-source-6947749cfc-k74jj\" (UID: \"a6f20e37-2612-4b8a-95a6-f4530b58144f\") " pod="openshift-network-diagnostics/network-check-source-6947749cfc-k74jj" Jun 15 01:48:10.939056 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:10.939060 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bp6l6\" (UniqueName: \"kubernetes.io/projected/a916f9e4-441d-4c65-bd95-be9c821970b3-kube-api-access-bp6l6\") pod \"cluster-samples-operator-9547488fd-w8h9d\" (UID: \"a916f9e4-441d-4c65-bd95-be9c821970b3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-w8h9d" Jun 15 01:48:10.939364 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:10.939079 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a916f9e4-441d-4c65-bd95-be9c821970b3-samples-operator-tls\") pod \"cluster-samples-operator-9547488fd-w8h9d\" (UID: \"a916f9e4-441d-4c65-bd95-be9c821970b3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-w8h9d" Jun 15 01:48:10.939364 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:48:10.939232 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 15 01:48:10.939364 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:48:10.939280 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a916f9e4-441d-4c65-bd95-be9c821970b3-samples-operator-tls podName:a916f9e4-441d-4c65-bd95-be9c821970b3 nodeName:}" failed. No retries permitted until 2026-06-15 01:48:11.4392672 +0000 UTC m=+215.152993150 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a916f9e4-441d-4c65-bd95-be9c821970b3-samples-operator-tls") pod "cluster-samples-operator-9547488fd-w8h9d" (UID: "a916f9e4-441d-4c65-bd95-be9c821970b3") : secret "samples-operator-tls" not found Jun 15 01:48:10.947810 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:10.947780 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhpf9\" (UniqueName: \"kubernetes.io/projected/a6f20e37-2612-4b8a-95a6-f4530b58144f-kube-api-access-lhpf9\") pod \"network-check-source-6947749cfc-k74jj\" (UID: \"a6f20e37-2612-4b8a-95a6-f4530b58144f\") " pod="openshift-network-diagnostics/network-check-source-6947749cfc-k74jj" Jun 15 01:48:10.947905 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:10.947843 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bp6l6\" (UniqueName: \"kubernetes.io/projected/a916f9e4-441d-4c65-bd95-be9c821970b3-kube-api-access-bp6l6\") pod \"cluster-samples-operator-9547488fd-w8h9d\" (UID: \"a916f9e4-441d-4c65-bd95-be9c821970b3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-w8h9d" Jun 15 01:48:10.980705 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:10.980679 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6947749cfc-k74jj" Jun 15 01:48:11.098512 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:11.098438 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6947749cfc-k74jj"] Jun 15 01:48:11.101898 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:48:11.101861 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6f20e37_2612_4b8a_95a6_f4530b58144f.slice/crio-d46e4999e5ac993f826fcd2bb83badb66d67d6a54bbfd5c35b027f37dff1f409 WatchSource:0}: Error finding container d46e4999e5ac993f826fcd2bb83badb66d67d6a54bbfd5c35b027f37dff1f409: Status 404 returned error can't find the container with id d46e4999e5ac993f826fcd2bb83badb66d67d6a54bbfd5c35b027f37dff1f409 Jun 15 01:48:11.342584 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:11.342546 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4979e05-89c5-482b-a895-4f1f9ae54e93-service-ca-bundle\") pod \"router-default-58cf549cb4-mc482\" (UID: \"d4979e05-89c5-482b-a895-4f1f9ae54e93\") " pod="openshift-ingress/router-default-58cf549cb4-mc482" Jun 15 01:48:11.342788 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:11.342613 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4979e05-89c5-482b-a895-4f1f9ae54e93-metrics-certs\") pod \"router-default-58cf549cb4-mc482\" (UID: \"d4979e05-89c5-482b-a895-4f1f9ae54e93\") " pod="openshift-ingress/router-default-58cf549cb4-mc482" Jun 15 01:48:11.342788 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:48:11.342702 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 15 01:48:11.342788 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:48:11.342735 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d4979e05-89c5-482b-a895-4f1f9ae54e93-service-ca-bundle podName:d4979e05-89c5-482b-a895-4f1f9ae54e93 nodeName:}" failed. No retries permitted until 2026-06-15 01:48:12.342714331 +0000 UTC m=+216.056440296 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d4979e05-89c5-482b-a895-4f1f9ae54e93-service-ca-bundle") pod "router-default-58cf549cb4-mc482" (UID: "d4979e05-89c5-482b-a895-4f1f9ae54e93") : configmap references non-existent config key: service-ca.crt Jun 15 01:48:11.342788 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:48:11.342763 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4979e05-89c5-482b-a895-4f1f9ae54e93-metrics-certs podName:d4979e05-89c5-482b-a895-4f1f9ae54e93 nodeName:}" failed. No retries permitted until 2026-06-15 01:48:12.342754214 +0000 UTC m=+216.056480170 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4979e05-89c5-482b-a895-4f1f9ae54e93-metrics-certs") pod "router-default-58cf549cb4-mc482" (UID: "d4979e05-89c5-482b-a895-4f1f9ae54e93") : secret "router-metrics-certs-default" not found Jun 15 01:48:11.443768 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:11.443734 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a916f9e4-441d-4c65-bd95-be9c821970b3-samples-operator-tls\") pod \"cluster-samples-operator-9547488fd-w8h9d\" (UID: \"a916f9e4-441d-4c65-bd95-be9c821970b3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-w8h9d" Jun 15 01:48:11.443996 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:48:11.443863 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 15 01:48:11.443996 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:48:11.443914 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a916f9e4-441d-4c65-bd95-be9c821970b3-samples-operator-tls podName:a916f9e4-441d-4c65-bd95-be9c821970b3 nodeName:}" failed. No retries permitted until 2026-06-15 01:48:12.443900633 +0000 UTC m=+216.157626583 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a916f9e4-441d-4c65-bd95-be9c821970b3-samples-operator-tls") pod "cluster-samples-operator-9547488fd-w8h9d" (UID: "a916f9e4-441d-4c65-bd95-be9c821970b3") : secret "samples-operator-tls" not found Jun 15 01:48:11.473242 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:11.473209 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6947749cfc-k74jj" event={"ID":"a6f20e37-2612-4b8a-95a6-f4530b58144f","Type":"ContainerStarted","Data":"01288537e3bea884b65e5066bed170d2bc7c14327324e62d656558ff4d7ca5d3"} Jun 15 01:48:11.473441 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:11.473270 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6947749cfc-k74jj" event={"ID":"a6f20e37-2612-4b8a-95a6-f4530b58144f","Type":"ContainerStarted","Data":"d46e4999e5ac993f826fcd2bb83badb66d67d6a54bbfd5c35b027f37dff1f409"} Jun 15 01:48:11.488853 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:11.488803 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6947749cfc-k74jj" podStartSLOduration=1.488787745 podStartE2EDuration="1.488787745s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 01:48:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 01:48:11.487899164 +0000 UTC m=+215.201625137" watchObservedRunningTime="2026-06-15 01:48:11.488787745 +0000 UTC m=+215.202513761" Jun 15 01:48:12.350366 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:12.350321 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4979e05-89c5-482b-a895-4f1f9ae54e93-service-ca-bundle\") pod \"router-default-58cf549cb4-mc482\" (UID: \"d4979e05-89c5-482b-a895-4f1f9ae54e93\") " pod="openshift-ingress/router-default-58cf549cb4-mc482" Jun 15 01:48:12.350748 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:12.350417 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4979e05-89c5-482b-a895-4f1f9ae54e93-metrics-certs\") pod \"router-default-58cf549cb4-mc482\" (UID: \"d4979e05-89c5-482b-a895-4f1f9ae54e93\") " pod="openshift-ingress/router-default-58cf549cb4-mc482" Jun 15 01:48:12.350748 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:48:12.350488 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d4979e05-89c5-482b-a895-4f1f9ae54e93-service-ca-bundle podName:d4979e05-89c5-482b-a895-4f1f9ae54e93 nodeName:}" failed. No retries permitted until 2026-06-15 01:48:14.350469056 +0000 UTC m=+218.064195007 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d4979e05-89c5-482b-a895-4f1f9ae54e93-service-ca-bundle") pod "router-default-58cf549cb4-mc482" (UID: "d4979e05-89c5-482b-a895-4f1f9ae54e93") : configmap references non-existent config key: service-ca.crt Jun 15 01:48:12.350748 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:48:12.350531 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 15 01:48:12.350748 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:48:12.350570 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4979e05-89c5-482b-a895-4f1f9ae54e93-metrics-certs podName:d4979e05-89c5-482b-a895-4f1f9ae54e93 nodeName:}" failed. No retries permitted until 2026-06-15 01:48:14.35055847 +0000 UTC m=+218.064284425 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4979e05-89c5-482b-a895-4f1f9ae54e93-metrics-certs") pod "router-default-58cf549cb4-mc482" (UID: "d4979e05-89c5-482b-a895-4f1f9ae54e93") : secret "router-metrics-certs-default" not found Jun 15 01:48:12.451036 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:12.451004 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a916f9e4-441d-4c65-bd95-be9c821970b3-samples-operator-tls\") pod \"cluster-samples-operator-9547488fd-w8h9d\" (UID: \"a916f9e4-441d-4c65-bd95-be9c821970b3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-w8h9d" Jun 15 01:48:12.451184 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:48:12.451129 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 15 01:48:12.451184 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:48:12.451178 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a916f9e4-441d-4c65-bd95-be9c821970b3-samples-operator-tls podName:a916f9e4-441d-4c65-bd95-be9c821970b3 nodeName:}" failed. No retries permitted until 2026-06-15 01:48:14.451165038 +0000 UTC m=+218.164890988 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a916f9e4-441d-4c65-bd95-be9c821970b3-samples-operator-tls") pod "cluster-samples-operator-9547488fd-w8h9d" (UID: "a916f9e4-441d-4c65-bd95-be9c821970b3") : secret "samples-operator-tls" not found Jun 15 01:48:14.365755 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:14.365715 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4979e05-89c5-482b-a895-4f1f9ae54e93-service-ca-bundle\") pod \"router-default-58cf549cb4-mc482\" (UID: \"d4979e05-89c5-482b-a895-4f1f9ae54e93\") " pod="openshift-ingress/router-default-58cf549cb4-mc482" Jun 15 01:48:14.366164 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:14.365785 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4979e05-89c5-482b-a895-4f1f9ae54e93-metrics-certs\") pod \"router-default-58cf549cb4-mc482\" (UID: \"d4979e05-89c5-482b-a895-4f1f9ae54e93\") " pod="openshift-ingress/router-default-58cf549cb4-mc482" Jun 15 01:48:14.366164 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:48:14.365864 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 15 01:48:14.366164 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:48:14.365892 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d4979e05-89c5-482b-a895-4f1f9ae54e93-service-ca-bundle podName:d4979e05-89c5-482b-a895-4f1f9ae54e93 nodeName:}" failed. No retries permitted until 2026-06-15 01:48:18.365874571 +0000 UTC m=+222.079600523 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d4979e05-89c5-482b-a895-4f1f9ae54e93-service-ca-bundle") pod "router-default-58cf549cb4-mc482" (UID: "d4979e05-89c5-482b-a895-4f1f9ae54e93") : configmap references non-existent config key: service-ca.crt Jun 15 01:48:14.366164 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:48:14.365918 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4979e05-89c5-482b-a895-4f1f9ae54e93-metrics-certs podName:d4979e05-89c5-482b-a895-4f1f9ae54e93 nodeName:}" failed. No retries permitted until 2026-06-15 01:48:18.365910352 +0000 UTC m=+222.079636302 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4979e05-89c5-482b-a895-4f1f9ae54e93-metrics-certs") pod "router-default-58cf549cb4-mc482" (UID: "d4979e05-89c5-482b-a895-4f1f9ae54e93") : secret "router-metrics-certs-default" not found Jun 15 01:48:14.466628 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:14.466587 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a916f9e4-441d-4c65-bd95-be9c821970b3-samples-operator-tls\") pod \"cluster-samples-operator-9547488fd-w8h9d\" (UID: \"a916f9e4-441d-4c65-bd95-be9c821970b3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-w8h9d" Jun 15 01:48:14.466817 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:48:14.466794 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 15 01:48:14.466912 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:48:14.466877 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a916f9e4-441d-4c65-bd95-be9c821970b3-samples-operator-tls podName:a916f9e4-441d-4c65-bd95-be9c821970b3 nodeName:}" failed. No retries permitted until 2026-06-15 01:48:18.466857716 +0000 UTC m=+222.180583666 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a916f9e4-441d-4c65-bd95-be9c821970b3-samples-operator-tls") pod "cluster-samples-operator-9547488fd-w8h9d" (UID: "a916f9e4-441d-4c65-bd95-be9c821970b3") : secret "samples-operator-tls" not found Jun 15 01:48:16.058999 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:16.058965 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-lgxjj"] Jun 15 01:48:16.061542 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:16.061527 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lgxjj" Jun 15 01:48:16.076235 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:16.076215 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 15 01:48:16.077357 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:16.077335 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jcjs4\"" Jun 15 01:48:16.077357 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:16.077340 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 15 01:48:16.077529 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:16.077368 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 15 01:48:16.077529 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:16.077344 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 15 01:48:16.103808 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:16.103772 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lgxjj"] Jun 15 01:48:16.179910 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:16.179875 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpw8c\" (UniqueName: \"kubernetes.io/projected/47a7fc6e-077c-4eb7-9850-a31a72bec0d1-kube-api-access-qpw8c\") pod \"insights-runtime-extractor-lgxjj\" (UID: \"47a7fc6e-077c-4eb7-9850-a31a72bec0d1\") " pod="openshift-insights/insights-runtime-extractor-lgxjj" Jun 15 01:48:16.180127 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:16.179947 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/47a7fc6e-077c-4eb7-9850-a31a72bec0d1-crio-socket\") pod \"insights-runtime-extractor-lgxjj\" (UID: \"47a7fc6e-077c-4eb7-9850-a31a72bec0d1\") " pod="openshift-insights/insights-runtime-extractor-lgxjj" Jun 15 01:48:16.180127 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:16.180054 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/47a7fc6e-077c-4eb7-9850-a31a72bec0d1-data-volume\") pod \"insights-runtime-extractor-lgxjj\" (UID: \"47a7fc6e-077c-4eb7-9850-a31a72bec0d1\") " pod="openshift-insights/insights-runtime-extractor-lgxjj" Jun 15 01:48:16.180127 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:16.180079 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/47a7fc6e-077c-4eb7-9850-a31a72bec0d1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lgxjj\" (UID: \"47a7fc6e-077c-4eb7-9850-a31a72bec0d1\") " pod="openshift-insights/insights-runtime-extractor-lgxjj" Jun 15 01:48:16.180259 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:16.180148 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/47a7fc6e-077c-4eb7-9850-a31a72bec0d1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lgxjj\" (UID: \"47a7fc6e-077c-4eb7-9850-a31a72bec0d1\") " pod="openshift-insights/insights-runtime-extractor-lgxjj" Jun 15 01:48:16.281212 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:16.281178 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/47a7fc6e-077c-4eb7-9850-a31a72bec0d1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lgxjj\" (UID: \"47a7fc6e-077c-4eb7-9850-a31a72bec0d1\") " pod="openshift-insights/insights-runtime-extractor-lgxjj" Jun 15 01:48:16.281400 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:16.281226 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpw8c\" (UniqueName: \"kubernetes.io/projected/47a7fc6e-077c-4eb7-9850-a31a72bec0d1-kube-api-access-qpw8c\") pod \"insights-runtime-extractor-lgxjj\" (UID: \"47a7fc6e-077c-4eb7-9850-a31a72bec0d1\") " pod="openshift-insights/insights-runtime-extractor-lgxjj" Jun 15 01:48:16.281400 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:16.281279 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/47a7fc6e-077c-4eb7-9850-a31a72bec0d1-crio-socket\") pod \"insights-runtime-extractor-lgxjj\" (UID: \"47a7fc6e-077c-4eb7-9850-a31a72bec0d1\") " pod="openshift-insights/insights-runtime-extractor-lgxjj" Jun 15 01:48:16.281400 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:48:16.281291 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 15 01:48:16.281400 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:48:16.281370 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47a7fc6e-077c-4eb7-9850-a31a72bec0d1-insights-runtime-extractor-tls podName:47a7fc6e-077c-4eb7-9850-a31a72bec0d1 nodeName:}" failed. No retries permitted until 2026-06-15 01:48:16.781351536 +0000 UTC m=+220.495077486 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/47a7fc6e-077c-4eb7-9850-a31a72bec0d1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lgxjj" (UID: "47a7fc6e-077c-4eb7-9850-a31a72bec0d1") : secret "insights-runtime-extractor-tls" not found Jun 15 01:48:16.281638 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:16.281406 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/47a7fc6e-077c-4eb7-9850-a31a72bec0d1-crio-socket\") pod \"insights-runtime-extractor-lgxjj\" (UID: \"47a7fc6e-077c-4eb7-9850-a31a72bec0d1\") " pod="openshift-insights/insights-runtime-extractor-lgxjj" Jun 15 01:48:16.281638 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:16.281411 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/47a7fc6e-077c-4eb7-9850-a31a72bec0d1-data-volume\") pod \"insights-runtime-extractor-lgxjj\" (UID: \"47a7fc6e-077c-4eb7-9850-a31a72bec0d1\") " pod="openshift-insights/insights-runtime-extractor-lgxjj" Jun 15 01:48:16.281638 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:16.281463 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/47a7fc6e-077c-4eb7-9850-a31a72bec0d1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lgxjj\" (UID: \"47a7fc6e-077c-4eb7-9850-a31a72bec0d1\") " pod="openshift-insights/insights-runtime-extractor-lgxjj" Jun 15 01:48:16.281638 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:48:16.281521 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 15 01:48:16.281638 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:48:16.281564 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/47a7fc6e-077c-4eb7-9850-a31a72bec0d1-kube-rbac-proxy-cm podName:47a7fc6e-077c-4eb7-9850-a31a72bec0d1 nodeName:}" failed. No retries permitted until 2026-06-15 01:48:16.781551983 +0000 UTC m=+220.495277936 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/47a7fc6e-077c-4eb7-9850-a31a72bec0d1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lgxjj" (UID: "47a7fc6e-077c-4eb7-9850-a31a72bec0d1") : configmap "kube-rbac-proxy" not found Jun 15 01:48:16.281890 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:16.281762 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/47a7fc6e-077c-4eb7-9850-a31a72bec0d1-data-volume\") pod \"insights-runtime-extractor-lgxjj\" (UID: \"47a7fc6e-077c-4eb7-9850-a31a72bec0d1\") " pod="openshift-insights/insights-runtime-extractor-lgxjj" Jun 15 01:48:16.296849 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:16.296779 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpw8c\" (UniqueName: \"kubernetes.io/projected/47a7fc6e-077c-4eb7-9850-a31a72bec0d1-kube-api-access-qpw8c\") pod \"insights-runtime-extractor-lgxjj\" (UID: \"47a7fc6e-077c-4eb7-9850-a31a72bec0d1\") " pod="openshift-insights/insights-runtime-extractor-lgxjj" Jun 15 01:48:16.785804 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:16.785772 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/47a7fc6e-077c-4eb7-9850-a31a72bec0d1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lgxjj\" (UID: \"47a7fc6e-077c-4eb7-9850-a31a72bec0d1\") " pod="openshift-insights/insights-runtime-extractor-lgxjj" Jun 15 01:48:16.785804 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:16.785806 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/47a7fc6e-077c-4eb7-9850-a31a72bec0d1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lgxjj\" (UID: \"47a7fc6e-077c-4eb7-9850-a31a72bec0d1\") " pod="openshift-insights/insights-runtime-extractor-lgxjj" Jun 15 01:48:16.786043 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:48:16.785886 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 15 01:48:16.786043 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:48:16.785972 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/47a7fc6e-077c-4eb7-9850-a31a72bec0d1-kube-rbac-proxy-cm podName:47a7fc6e-077c-4eb7-9850-a31a72bec0d1 nodeName:}" failed. No retries permitted until 2026-06-15 01:48:17.7859502 +0000 UTC m=+221.499676176 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/47a7fc6e-077c-4eb7-9850-a31a72bec0d1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lgxjj" (UID: "47a7fc6e-077c-4eb7-9850-a31a72bec0d1") : configmap "kube-rbac-proxy" not found Jun 15 01:48:16.786043 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:48:16.786000 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 15 01:48:16.786205 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:48:16.786074 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47a7fc6e-077c-4eb7-9850-a31a72bec0d1-insights-runtime-extractor-tls podName:47a7fc6e-077c-4eb7-9850-a31a72bec0d1 nodeName:}" failed. No retries permitted until 2026-06-15 01:48:17.786056613 +0000 UTC m=+221.499782564 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/47a7fc6e-077c-4eb7-9850-a31a72bec0d1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lgxjj" (UID: "47a7fc6e-077c-4eb7-9850-a31a72bec0d1") : secret "insights-runtime-extractor-tls" not found Jun 15 01:48:17.793170 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:17.793116 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/47a7fc6e-077c-4eb7-9850-a31a72bec0d1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lgxjj\" (UID: \"47a7fc6e-077c-4eb7-9850-a31a72bec0d1\") " pod="openshift-insights/insights-runtime-extractor-lgxjj" Jun 15 01:48:17.793170 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:17.793171 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/47a7fc6e-077c-4eb7-9850-a31a72bec0d1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lgxjj\" (UID: \"47a7fc6e-077c-4eb7-9850-a31a72bec0d1\") " pod="openshift-insights/insights-runtime-extractor-lgxjj" Jun 15 01:48:17.793718 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:48:17.793232 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 15 01:48:17.793718 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:48:17.793312 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/47a7fc6e-077c-4eb7-9850-a31a72bec0d1-kube-rbac-proxy-cm podName:47a7fc6e-077c-4eb7-9850-a31a72bec0d1 nodeName:}" failed. No retries permitted until 2026-06-15 01:48:19.793292891 +0000 UTC m=+223.507018843 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/47a7fc6e-077c-4eb7-9850-a31a72bec0d1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lgxjj" (UID: "47a7fc6e-077c-4eb7-9850-a31a72bec0d1") : configmap "kube-rbac-proxy" not found Jun 15 01:48:17.793718 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:48:17.793315 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 15 01:48:17.793718 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:48:17.793363 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47a7fc6e-077c-4eb7-9850-a31a72bec0d1-insights-runtime-extractor-tls podName:47a7fc6e-077c-4eb7-9850-a31a72bec0d1 nodeName:}" failed. No retries permitted until 2026-06-15 01:48:19.793351592 +0000 UTC m=+223.507077541 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/47a7fc6e-077c-4eb7-9850-a31a72bec0d1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lgxjj" (UID: "47a7fc6e-077c-4eb7-9850-a31a72bec0d1") : secret "insights-runtime-extractor-tls" not found Jun 15 01:48:18.246138 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:18.246111 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pmkl7_3cf2d35f-dae7-42c6-8f35-5b0ccb5e4750/dns-node-resolver/0.log" Jun 15 01:48:18.398134 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:18.398080 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4979e05-89c5-482b-a895-4f1f9ae54e93-metrics-certs\") pod \"router-default-58cf549cb4-mc482\" (UID: \"d4979e05-89c5-482b-a895-4f1f9ae54e93\") " pod="openshift-ingress/router-default-58cf549cb4-mc482" Jun 15 01:48:18.398270 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:18.398150 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4979e05-89c5-482b-a895-4f1f9ae54e93-service-ca-bundle\") pod \"router-default-58cf549cb4-mc482\" (UID: \"d4979e05-89c5-482b-a895-4f1f9ae54e93\") " pod="openshift-ingress/router-default-58cf549cb4-mc482" Jun 15 01:48:18.398270 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:48:18.398219 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 15 01:48:18.398337 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:48:18.398296 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4979e05-89c5-482b-a895-4f1f9ae54e93-metrics-certs podName:d4979e05-89c5-482b-a895-4f1f9ae54e93 nodeName:}" failed. No retries permitted until 2026-06-15 01:48:26.398275678 +0000 UTC m=+230.112001629 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4979e05-89c5-482b-a895-4f1f9ae54e93-metrics-certs") pod "router-default-58cf549cb4-mc482" (UID: "d4979e05-89c5-482b-a895-4f1f9ae54e93") : secret "router-metrics-certs-default" not found Jun 15 01:48:18.398337 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:48:18.398311 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d4979e05-89c5-482b-a895-4f1f9ae54e93-service-ca-bundle podName:d4979e05-89c5-482b-a895-4f1f9ae54e93 nodeName:}" failed. No retries permitted until 2026-06-15 01:48:26.398304817 +0000 UTC m=+230.112030767 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d4979e05-89c5-482b-a895-4f1f9ae54e93-service-ca-bundle") pod "router-default-58cf549cb4-mc482" (UID: "d4979e05-89c5-482b-a895-4f1f9ae54e93") : configmap references non-existent config key: service-ca.crt Jun 15 01:48:18.498700 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:18.498604 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a916f9e4-441d-4c65-bd95-be9c821970b3-samples-operator-tls\") pod \"cluster-samples-operator-9547488fd-w8h9d\" (UID: \"a916f9e4-441d-4c65-bd95-be9c821970b3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-w8h9d" Jun 15 01:48:18.498845 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:48:18.498712 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 15 01:48:18.498845 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:48:18.498760 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a916f9e4-441d-4c65-bd95-be9c821970b3-samples-operator-tls podName:a916f9e4-441d-4c65-bd95-be9c821970b3 nodeName:}" failed. No retries permitted until 2026-06-15 01:48:26.498747308 +0000 UTC m=+230.212473258 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a916f9e4-441d-4c65-bd95-be9c821970b3-samples-operator-tls") pod "cluster-samples-operator-9547488fd-w8h9d" (UID: "a916f9e4-441d-4c65-bd95-be9c821970b3") : secret "samples-operator-tls" not found Jun 15 01:48:19.123761 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:19.123728 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-7b994bc59-zdlrr"] Jun 15 01:48:19.126637 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:19.126620 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7b994bc59-zdlrr" Jun 15 01:48:19.129435 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:19.129415 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jun 15 01:48:19.129540 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:19.129440 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jun 15 01:48:19.129540 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:19.129440 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jun 15 01:48:19.130722 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:19.130705 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jun 15 01:48:19.130800 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:19.130765 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-rz4xp\"" Jun 15 01:48:19.134196 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:19.134177 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7b994bc59-zdlrr"] Jun 15 01:48:19.245713 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:19.245686 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-r8dqf_17743aa6-c9cb-458c-b3d3-16c6a774171a/node-ca/0.log" Jun 15 01:48:19.305363 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:19.305327 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zll5r\" (UniqueName: \"kubernetes.io/projected/e51926b2-0415-426a-8a41-840d53c81389-kube-api-access-zll5r\") pod \"service-ca-7b994bc59-zdlrr\" (UID: \"e51926b2-0415-426a-8a41-840d53c81389\") " pod="openshift-service-ca/service-ca-7b994bc59-zdlrr" Jun 15 01:48:19.305504 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:19.305371 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e51926b2-0415-426a-8a41-840d53c81389-signing-key\") pod \"service-ca-7b994bc59-zdlrr\" (UID: \"e51926b2-0415-426a-8a41-840d53c81389\") " pod="openshift-service-ca/service-ca-7b994bc59-zdlrr" Jun 15 01:48:19.305504 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:19.305476 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e51926b2-0415-426a-8a41-840d53c81389-signing-cabundle\") pod \"service-ca-7b994bc59-zdlrr\" (UID: \"e51926b2-0415-426a-8a41-840d53c81389\") " pod="openshift-service-ca/service-ca-7b994bc59-zdlrr" Jun 15 01:48:19.405985 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:19.405892 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zll5r\" (UniqueName: \"kubernetes.io/projected/e51926b2-0415-426a-8a41-840d53c81389-kube-api-access-zll5r\") pod \"service-ca-7b994bc59-zdlrr\" (UID: \"e51926b2-0415-426a-8a41-840d53c81389\") " pod="openshift-service-ca/service-ca-7b994bc59-zdlrr" Jun 15 01:48:19.405985 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:19.405944 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e51926b2-0415-426a-8a41-840d53c81389-signing-key\") pod \"service-ca-7b994bc59-zdlrr\" (UID: \"e51926b2-0415-426a-8a41-840d53c81389\") " pod="openshift-service-ca/service-ca-7b994bc59-zdlrr" Jun 15 01:48:19.406208 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:19.406045 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e51926b2-0415-426a-8a41-840d53c81389-signing-cabundle\") pod \"service-ca-7b994bc59-zdlrr\" (UID: \"e51926b2-0415-426a-8a41-840d53c81389\") " pod="openshift-service-ca/service-ca-7b994bc59-zdlrr" Jun 15 01:48:19.406760 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:19.406736 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e51926b2-0415-426a-8a41-840d53c81389-signing-cabundle\") pod \"service-ca-7b994bc59-zdlrr\" (UID: \"e51926b2-0415-426a-8a41-840d53c81389\") " pod="openshift-service-ca/service-ca-7b994bc59-zdlrr" Jun 15 01:48:19.408502 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:19.408482 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e51926b2-0415-426a-8a41-840d53c81389-signing-key\") pod \"service-ca-7b994bc59-zdlrr\" (UID: \"e51926b2-0415-426a-8a41-840d53c81389\") " pod="openshift-service-ca/service-ca-7b994bc59-zdlrr" Jun 15 01:48:19.414821 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:19.414798 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zll5r\" (UniqueName: \"kubernetes.io/projected/e51926b2-0415-426a-8a41-840d53c81389-kube-api-access-zll5r\") pod \"service-ca-7b994bc59-zdlrr\" (UID: \"e51926b2-0415-426a-8a41-840d53c81389\") " pod="openshift-service-ca/service-ca-7b994bc59-zdlrr" Jun 15 01:48:19.435768 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:19.435736 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7b994bc59-zdlrr" Jun 15 01:48:19.559186 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:19.559152 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7b994bc59-zdlrr"] Jun 15 01:48:19.562013 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:48:19.561982 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode51926b2_0415_426a_8a41_840d53c81389.slice/crio-c3e4c1a3d6536f94f0b5b753097dc765ebc3dfdf219ed399a06811f6a7ea561c WatchSource:0}: Error finding container c3e4c1a3d6536f94f0b5b753097dc765ebc3dfdf219ed399a06811f6a7ea561c: Status 404 returned error can't find the container with id c3e4c1a3d6536f94f0b5b753097dc765ebc3dfdf219ed399a06811f6a7ea561c Jun 15 01:48:19.646893 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:19.646868 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-48xtp_878da7f4-63c4-45b4-a4c6-fc6a72556516/serve-healthcheck-canary/0.log" Jun 15 01:48:19.810689 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:19.810593 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/47a7fc6e-077c-4eb7-9850-a31a72bec0d1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lgxjj\" (UID: \"47a7fc6e-077c-4eb7-9850-a31a72bec0d1\") " pod="openshift-insights/insights-runtime-extractor-lgxjj" Jun 15 01:48:19.810689 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:19.810646 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/47a7fc6e-077c-4eb7-9850-a31a72bec0d1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lgxjj\" (UID: \"47a7fc6e-077c-4eb7-9850-a31a72bec0d1\") " pod="openshift-insights/insights-runtime-extractor-lgxjj" Jun 15 01:48:19.810883 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:48:19.810723 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 15 01:48:19.810883 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:48:19.810775 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 15 01:48:19.810883 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:48:19.810790 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/47a7fc6e-077c-4eb7-9850-a31a72bec0d1-kube-rbac-proxy-cm podName:47a7fc6e-077c-4eb7-9850-a31a72bec0d1 nodeName:}" failed. No retries permitted until 2026-06-15 01:48:23.810774146 +0000 UTC m=+227.524500096 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/47a7fc6e-077c-4eb7-9850-a31a72bec0d1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lgxjj" (UID: "47a7fc6e-077c-4eb7-9850-a31a72bec0d1") : configmap "kube-rbac-proxy" not found Jun 15 01:48:19.810883 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:48:19.810818 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47a7fc6e-077c-4eb7-9850-a31a72bec0d1-insights-runtime-extractor-tls podName:47a7fc6e-077c-4eb7-9850-a31a72bec0d1 nodeName:}" failed. No retries permitted until 2026-06-15 01:48:23.810806067 +0000 UTC m=+227.524532018 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/47a7fc6e-077c-4eb7-9850-a31a72bec0d1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lgxjj" (UID: "47a7fc6e-077c-4eb7-9850-a31a72bec0d1") : secret "insights-runtime-extractor-tls" not found Jun 15 01:48:20.506259 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:20.506222 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7b994bc59-zdlrr" event={"ID":"e51926b2-0415-426a-8a41-840d53c81389","Type":"ContainerStarted","Data":"c3e4c1a3d6536f94f0b5b753097dc765ebc3dfdf219ed399a06811f6a7ea561c"} Jun 15 01:48:21.510449 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:21.510414 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7b994bc59-zdlrr" event={"ID":"e51926b2-0415-426a-8a41-840d53c81389","Type":"ContainerStarted","Data":"57ce56af38cb57bdaca7ec95d6598543735ca637eb01737456872b5dd8a16e92"} Jun 15 01:48:21.528135 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:21.528071 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-7b994bc59-zdlrr" podStartSLOduration=0.772929051 podStartE2EDuration="2.52805539s" totalImagesPullingTime="1.755126339s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 01:48:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 01:48:21.526666512 +0000 UTC m=+225.240392498" watchObservedRunningTime="2026-06-15 01:48:21.52805539 +0000 UTC m=+225.241781361" Jun 15 01:48:23.844306 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:23.844268 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/47a7fc6e-077c-4eb7-9850-a31a72bec0d1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lgxjj\" (UID: \"47a7fc6e-077c-4eb7-9850-a31a72bec0d1\") " pod="openshift-insights/insights-runtime-extractor-lgxjj" Jun 15 01:48:23.844859 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:48:23.844401 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 15 01:48:23.844938 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:48:23.844917 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47a7fc6e-077c-4eb7-9850-a31a72bec0d1-insights-runtime-extractor-tls podName:47a7fc6e-077c-4eb7-9850-a31a72bec0d1 nodeName:}" failed. No retries permitted until 2026-06-15 01:48:31.844894951 +0000 UTC m=+235.558620907 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/47a7fc6e-077c-4eb7-9850-a31a72bec0d1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lgxjj" (UID: "47a7fc6e-077c-4eb7-9850-a31a72bec0d1") : secret "insights-runtime-extractor-tls" not found Jun 15 01:48:23.845195 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:23.845165 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/47a7fc6e-077c-4eb7-9850-a31a72bec0d1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lgxjj\" (UID: \"47a7fc6e-077c-4eb7-9850-a31a72bec0d1\") " pod="openshift-insights/insights-runtime-extractor-lgxjj" Jun 15 01:48:23.845320 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:48:23.845262 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 15 01:48:23.845320 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:48:23.845318 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/47a7fc6e-077c-4eb7-9850-a31a72bec0d1-kube-rbac-proxy-cm podName:47a7fc6e-077c-4eb7-9850-a31a72bec0d1 nodeName:}" failed. No retries permitted until 2026-06-15 01:48:31.845301585 +0000 UTC m=+235.559027550 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/47a7fc6e-077c-4eb7-9850-a31a72bec0d1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lgxjj" (UID: "47a7fc6e-077c-4eb7-9850-a31a72bec0d1") : configmap "kube-rbac-proxy" not found Jun 15 01:48:26.467366 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:26.467318 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4979e05-89c5-482b-a895-4f1f9ae54e93-metrics-certs\") pod \"router-default-58cf549cb4-mc482\" (UID: \"d4979e05-89c5-482b-a895-4f1f9ae54e93\") " pod="openshift-ingress/router-default-58cf549cb4-mc482" Jun 15 01:48:26.467366 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:26.467373 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4979e05-89c5-482b-a895-4f1f9ae54e93-service-ca-bundle\") pod \"router-default-58cf549cb4-mc482\" (UID: \"d4979e05-89c5-482b-a895-4f1f9ae54e93\") " pod="openshift-ingress/router-default-58cf549cb4-mc482" Jun 15 01:48:26.467789 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:48:26.467471 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 15 01:48:26.467789 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:48:26.467542 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4979e05-89c5-482b-a895-4f1f9ae54e93-metrics-certs podName:d4979e05-89c5-482b-a895-4f1f9ae54e93 nodeName:}" failed. No retries permitted until 2026-06-15 01:48:42.467526703 +0000 UTC m=+246.181252653 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4979e05-89c5-482b-a895-4f1f9ae54e93-metrics-certs") pod "router-default-58cf549cb4-mc482" (UID: "d4979e05-89c5-482b-a895-4f1f9ae54e93") : secret "router-metrics-certs-default" not found Jun 15 01:48:26.467789 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:48:26.467556 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d4979e05-89c5-482b-a895-4f1f9ae54e93-service-ca-bundle podName:d4979e05-89c5-482b-a895-4f1f9ae54e93 nodeName:}" failed. No retries permitted until 2026-06-15 01:48:42.467550414 +0000 UTC m=+246.181276363 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d4979e05-89c5-482b-a895-4f1f9ae54e93-service-ca-bundle") pod "router-default-58cf549cb4-mc482" (UID: "d4979e05-89c5-482b-a895-4f1f9ae54e93") : configmap references non-existent config key: service-ca.crt Jun 15 01:48:26.568770 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:26.568730 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a916f9e4-441d-4c65-bd95-be9c821970b3-samples-operator-tls\") pod \"cluster-samples-operator-9547488fd-w8h9d\" (UID: \"a916f9e4-441d-4c65-bd95-be9c821970b3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-w8h9d" Jun 15 01:48:26.571387 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:26.571364 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a916f9e4-441d-4c65-bd95-be9c821970b3-samples-operator-tls\") pod \"cluster-samples-operator-9547488fd-w8h9d\" (UID: \"a916f9e4-441d-4c65-bd95-be9c821970b3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-w8h9d" Jun 15 01:48:26.588171 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:26.588132 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-w8h9d" Jun 15 01:48:26.715884 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:26.715846 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-w8h9d"] Jun 15 01:48:27.531513 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:27.531473 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-w8h9d" event={"ID":"a916f9e4-441d-4c65-bd95-be9c821970b3","Type":"ContainerStarted","Data":"ba71b51eb40501fb29958974060b8e5bb6d94332e91e82d4324e9ae87e632842"} Jun 15 01:48:28.535628 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:28.535545 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-w8h9d" event={"ID":"a916f9e4-441d-4c65-bd95-be9c821970b3","Type":"ContainerStarted","Data":"5c51c82028b041a81345bda1ad38f616a62e75aae93dde938badf2b08bab1e5d"} Jun 15 01:48:28.535965 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:28.535640 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-w8h9d" event={"ID":"a916f9e4-441d-4c65-bd95-be9c821970b3","Type":"ContainerStarted","Data":"476d8bdfe208e4ed5dca9d3cc57cfe4deaa6dde8b5fc574cf4cfa5f7ffc62951"} Jun 15 01:48:28.552080 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:28.552032 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-9547488fd-w8h9d" podStartSLOduration=17.089422496 podStartE2EDuration="18.552017425s" totalImagesPullingTime="1.462594929s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 01:48:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 01:48:28.550820434 +0000 UTC m=+232.264546407" watchObservedRunningTime="2026-06-15 01:48:28.552017425 +0000 UTC m=+232.265743396" Jun 15 01:48:31.913935 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:31.913890 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/47a7fc6e-077c-4eb7-9850-a31a72bec0d1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lgxjj\" (UID: \"47a7fc6e-077c-4eb7-9850-a31a72bec0d1\") " pod="openshift-insights/insights-runtime-extractor-lgxjj" Jun 15 01:48:31.914447 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:31.913941 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/47a7fc6e-077c-4eb7-9850-a31a72bec0d1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lgxjj\" (UID: \"47a7fc6e-077c-4eb7-9850-a31a72bec0d1\") " pod="openshift-insights/insights-runtime-extractor-lgxjj" Jun 15 01:48:31.914447 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:48:31.914176 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 15 01:48:31.914447 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:48:31.914262 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47a7fc6e-077c-4eb7-9850-a31a72bec0d1-insights-runtime-extractor-tls podName:47a7fc6e-077c-4eb7-9850-a31a72bec0d1 nodeName:}" failed. No retries permitted until 2026-06-15 01:48:47.914240785 +0000 UTC m=+251.627966758 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/47a7fc6e-077c-4eb7-9850-a31a72bec0d1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lgxjj" (UID: "47a7fc6e-077c-4eb7-9850-a31a72bec0d1") : secret "insights-runtime-extractor-tls" not found Jun 15 01:48:31.914618 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:31.914463 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/47a7fc6e-077c-4eb7-9850-a31a72bec0d1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lgxjj\" (UID: \"47a7fc6e-077c-4eb7-9850-a31a72bec0d1\") " pod="openshift-insights/insights-runtime-extractor-lgxjj" Jun 15 01:48:35.558495 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:35.558462 2578 generic.go:356] "Generic (PLEG): container finished" podID="af636ff6-b9f5-4871-a86c-9ab3788dcd76" containerID="6a4fabf29175d8cfcb72c39aaef355ba70d2b2ef08a9479041fbead444b4bf5b" exitCode=1 Jun 15 01:48:35.558944 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:35.558524 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-844c6877cd-8v6bp" event={"ID":"af636ff6-b9f5-4871-a86c-9ab3788dcd76","Type":"ContainerDied","Data":"6a4fabf29175d8cfcb72c39aaef355ba70d2b2ef08a9479041fbead444b4bf5b"} Jun 15 01:48:35.558944 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:35.558937 2578 scope.go:117] "RemoveContainer" containerID="6a4fabf29175d8cfcb72c39aaef355ba70d2b2ef08a9479041fbead444b4bf5b" Jun 15 01:48:36.247570 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:36.247529 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-844c6877cd-8v6bp" Jun 15 01:48:36.563485 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:36.563403 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-844c6877cd-8v6bp" event={"ID":"af636ff6-b9f5-4871-a86c-9ab3788dcd76","Type":"ContainerStarted","Data":"68bd1e799ee89f16ce3b87c06b64f4ca200477d98123ba6dfe0ae5aef408277e"} Jun 15 01:48:36.563894 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:36.563630 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-844c6877cd-8v6bp" Jun 15 01:48:36.564278 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:36.564260 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-844c6877cd-8v6bp" Jun 15 01:48:42.500760 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:42.500722 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4979e05-89c5-482b-a895-4f1f9ae54e93-metrics-certs\") pod \"router-default-58cf549cb4-mc482\" (UID: \"d4979e05-89c5-482b-a895-4f1f9ae54e93\") " pod="openshift-ingress/router-default-58cf549cb4-mc482" Jun 15 01:48:42.501353 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:42.500769 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4979e05-89c5-482b-a895-4f1f9ae54e93-service-ca-bundle\") pod \"router-default-58cf549cb4-mc482\" (UID: \"d4979e05-89c5-482b-a895-4f1f9ae54e93\") " pod="openshift-ingress/router-default-58cf549cb4-mc482" Jun 15 01:48:42.501480 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:42.501462 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4979e05-89c5-482b-a895-4f1f9ae54e93-service-ca-bundle\") pod \"router-default-58cf549cb4-mc482\" (UID: \"d4979e05-89c5-482b-a895-4f1f9ae54e93\") " pod="openshift-ingress/router-default-58cf549cb4-mc482" Jun 15 01:48:42.503415 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:42.503383 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4979e05-89c5-482b-a895-4f1f9ae54e93-metrics-certs\") pod \"router-default-58cf549cb4-mc482\" (UID: \"d4979e05-89c5-482b-a895-4f1f9ae54e93\") " pod="openshift-ingress/router-default-58cf549cb4-mc482" Jun 15 01:48:42.692515 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:42.692482 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-kfk6k\"" Jun 15 01:48:42.700817 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:42.700792 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-58cf549cb4-mc482" Jun 15 01:48:42.858907 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:42.858878 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-58cf549cb4-mc482"] Jun 15 01:48:42.859725 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:48:42.859704 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4979e05_89c5_482b_a895_4f1f9ae54e93.slice/crio-28df2aa5e96469a8885cc0bd62bece7ec5fd339443deb475424df4f41fc51c92 WatchSource:0}: Error finding container 28df2aa5e96469a8885cc0bd62bece7ec5fd339443deb475424df4f41fc51c92: Status 404 returned error can't find the container with id 28df2aa5e96469a8885cc0bd62bece7ec5fd339443deb475424df4f41fc51c92 Jun 15 01:48:43.585000 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:43.584964 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-58cf549cb4-mc482" event={"ID":"d4979e05-89c5-482b-a895-4f1f9ae54e93","Type":"ContainerStarted","Data":"c344a63d90cc5e14548f7e338f1fdfe8303e2ffbdb40465578e8f79b2655c8c8"} Jun 15 01:48:43.585425 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:43.585021 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-58cf549cb4-mc482" event={"ID":"d4979e05-89c5-482b-a895-4f1f9ae54e93","Type":"ContainerStarted","Data":"28df2aa5e96469a8885cc0bd62bece7ec5fd339443deb475424df4f41fc51c92"} Jun 15 01:48:43.604656 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:43.604608 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-58cf549cb4-mc482" podStartSLOduration=33.604593454 podStartE2EDuration="33.604593454s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 01:48:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 01:48:43.603132483 +0000 UTC m=+247.316858452" watchObservedRunningTime="2026-06-15 01:48:43.604593454 +0000 UTC m=+247.318319426" Jun 15 01:48:43.701665 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:43.701629 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-58cf549cb4-mc482" Jun 15 01:48:43.704448 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:43.704424 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-58cf549cb4-mc482" Jun 15 01:48:43.995223 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:43.995188 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5ff47d99dc-p2vfn"] Jun 15 01:48:43.999076 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:43.999056 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5ff47d99dc-p2vfn" Jun 15 01:48:44.001966 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:44.001940 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jun 15 01:48:44.002304 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:44.002278 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-s92rf\"" Jun 15 01:48:44.002304 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:44.002292 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jun 15 01:48:44.002304 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:44.002309 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jun 15 01:48:44.002533 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:44.002329 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jun 15 01:48:44.002533 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:44.002285 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jun 15 01:48:44.003243 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:44.003223 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jun 15 01:48:44.003571 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:44.003557 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jun 15 01:48:44.008467 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:44.008445 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5ff47d99dc-p2vfn"] Jun 15 01:48:44.113849 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:44.113811 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2feade0e-acaf-4b90-8998-0df4be7c3c14-console-config\") pod \"console-5ff47d99dc-p2vfn\" (UID: \"2feade0e-acaf-4b90-8998-0df4be7c3c14\") " pod="openshift-console/console-5ff47d99dc-p2vfn" Jun 15 01:48:44.113849 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:44.113852 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2feade0e-acaf-4b90-8998-0df4be7c3c14-oauth-serving-cert\") pod \"console-5ff47d99dc-p2vfn\" (UID: \"2feade0e-acaf-4b90-8998-0df4be7c3c14\") " pod="openshift-console/console-5ff47d99dc-p2vfn" Jun 15 01:48:44.114051 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:44.113874 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2feade0e-acaf-4b90-8998-0df4be7c3c14-service-ca\") pod \"console-5ff47d99dc-p2vfn\" (UID: \"2feade0e-acaf-4b90-8998-0df4be7c3c14\") " pod="openshift-console/console-5ff47d99dc-p2vfn" Jun 15 01:48:44.114051 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:44.113951 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2feade0e-acaf-4b90-8998-0df4be7c3c14-console-oauth-config\") pod \"console-5ff47d99dc-p2vfn\" (UID: \"2feade0e-acaf-4b90-8998-0df4be7c3c14\") " pod="openshift-console/console-5ff47d99dc-p2vfn" Jun 15 01:48:44.114051 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:44.114046 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpfgq\" (UniqueName: \"kubernetes.io/projected/2feade0e-acaf-4b90-8998-0df4be7c3c14-kube-api-access-lpfgq\") pod \"console-5ff47d99dc-p2vfn\" (UID: \"2feade0e-acaf-4b90-8998-0df4be7c3c14\") " pod="openshift-console/console-5ff47d99dc-p2vfn" Jun 15 01:48:44.114179 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:44.114066 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2feade0e-acaf-4b90-8998-0df4be7c3c14-console-serving-cert\") pod \"console-5ff47d99dc-p2vfn\" (UID: \"2feade0e-acaf-4b90-8998-0df4be7c3c14\") " pod="openshift-console/console-5ff47d99dc-p2vfn" Jun 15 01:48:44.214612 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:44.214581 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpfgq\" (UniqueName: \"kubernetes.io/projected/2feade0e-acaf-4b90-8998-0df4be7c3c14-kube-api-access-lpfgq\") pod \"console-5ff47d99dc-p2vfn\" (UID: \"2feade0e-acaf-4b90-8998-0df4be7c3c14\") " pod="openshift-console/console-5ff47d99dc-p2vfn" Jun 15 01:48:44.214763 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:44.214617 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2feade0e-acaf-4b90-8998-0df4be7c3c14-console-serving-cert\") pod \"console-5ff47d99dc-p2vfn\" (UID: \"2feade0e-acaf-4b90-8998-0df4be7c3c14\") " pod="openshift-console/console-5ff47d99dc-p2vfn" Jun 15 01:48:44.214829 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:44.214786 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2feade0e-acaf-4b90-8998-0df4be7c3c14-console-config\") pod \"console-5ff47d99dc-p2vfn\" (UID: \"2feade0e-acaf-4b90-8998-0df4be7c3c14\") " pod="openshift-console/console-5ff47d99dc-p2vfn" Jun 15 01:48:44.214887 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:44.214837 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2feade0e-acaf-4b90-8998-0df4be7c3c14-oauth-serving-cert\") pod \"console-5ff47d99dc-p2vfn\" (UID: \"2feade0e-acaf-4b90-8998-0df4be7c3c14\") " pod="openshift-console/console-5ff47d99dc-p2vfn" Jun 15 01:48:44.214887 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:44.214871 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2feade0e-acaf-4b90-8998-0df4be7c3c14-service-ca\") pod \"console-5ff47d99dc-p2vfn\" (UID: \"2feade0e-acaf-4b90-8998-0df4be7c3c14\") " pod="openshift-console/console-5ff47d99dc-p2vfn" Jun 15 01:48:44.214981 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:44.214903 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2feade0e-acaf-4b90-8998-0df4be7c3c14-console-oauth-config\") pod \"console-5ff47d99dc-p2vfn\" (UID: \"2feade0e-acaf-4b90-8998-0df4be7c3c14\") " pod="openshift-console/console-5ff47d99dc-p2vfn" Jun 15 01:48:44.215552 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:44.215513 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2feade0e-acaf-4b90-8998-0df4be7c3c14-service-ca\") pod \"console-5ff47d99dc-p2vfn\" (UID: \"2feade0e-acaf-4b90-8998-0df4be7c3c14\") " pod="openshift-console/console-5ff47d99dc-p2vfn" Jun 15 01:48:44.215552 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:44.215520 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2feade0e-acaf-4b90-8998-0df4be7c3c14-console-config\") pod \"console-5ff47d99dc-p2vfn\" (UID: \"2feade0e-acaf-4b90-8998-0df4be7c3c14\") " pod="openshift-console/console-5ff47d99dc-p2vfn" Jun 15 01:48:44.215728 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:44.215546 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2feade0e-acaf-4b90-8998-0df4be7c3c14-oauth-serving-cert\") pod \"console-5ff47d99dc-p2vfn\" (UID: \"2feade0e-acaf-4b90-8998-0df4be7c3c14\") " pod="openshift-console/console-5ff47d99dc-p2vfn" Jun 15 01:48:44.217285 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:44.217255 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2feade0e-acaf-4b90-8998-0df4be7c3c14-console-serving-cert\") pod \"console-5ff47d99dc-p2vfn\" (UID: \"2feade0e-acaf-4b90-8998-0df4be7c3c14\") " pod="openshift-console/console-5ff47d99dc-p2vfn" Jun 15 01:48:44.217376 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:44.217348 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2feade0e-acaf-4b90-8998-0df4be7c3c14-console-oauth-config\") pod \"console-5ff47d99dc-p2vfn\" (UID: \"2feade0e-acaf-4b90-8998-0df4be7c3c14\") " pod="openshift-console/console-5ff47d99dc-p2vfn" Jun 15 01:48:44.222972 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:44.222943 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpfgq\" (UniqueName: \"kubernetes.io/projected/2feade0e-acaf-4b90-8998-0df4be7c3c14-kube-api-access-lpfgq\") pod \"console-5ff47d99dc-p2vfn\" (UID: \"2feade0e-acaf-4b90-8998-0df4be7c3c14\") " pod="openshift-console/console-5ff47d99dc-p2vfn" Jun 15 01:48:44.312220 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:44.312127 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5ff47d99dc-p2vfn" Jun 15 01:48:44.443057 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:44.443014 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5ff47d99dc-p2vfn"] Jun 15 01:48:44.445629 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:48:44.445603 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2feade0e_acaf_4b90_8998_0df4be7c3c14.slice/crio-ba3b5eb7bae57d0a1b419ac0afd41c3f1aec073c2caacff09fd5ae4cb43307fe WatchSource:0}: Error finding container ba3b5eb7bae57d0a1b419ac0afd41c3f1aec073c2caacff09fd5ae4cb43307fe: Status 404 returned error can't find the container with id ba3b5eb7bae57d0a1b419ac0afd41c3f1aec073c2caacff09fd5ae4cb43307fe Jun 15 01:48:44.589722 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:44.589602 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5ff47d99dc-p2vfn" event={"ID":"2feade0e-acaf-4b90-8998-0df4be7c3c14","Type":"ContainerStarted","Data":"ba3b5eb7bae57d0a1b419ac0afd41c3f1aec073c2caacff09fd5ae4cb43307fe"} Jun 15 01:48:44.590283 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:44.589898 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-58cf549cb4-mc482" Jun 15 01:48:44.591212 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:44.591189 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-58cf549cb4-mc482" Jun 15 01:48:44.778996 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:44.778956 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-bzm9j"] Jun 15 01:48:44.783771 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:44.783748 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-bzm9j" Jun 15 01:48:44.789426 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:44.789405 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jun 15 01:48:44.789556 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:44.789436 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-cjfbh\"" Jun 15 01:48:44.794529 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:44.794505 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-bzm9j"] Jun 15 01:48:44.920905 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:44.920874 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/8f715911-523f-4405-be5a-9e902db5609a-tls-certificates\") pod \"prometheus-operator-admission-webhook-5cf667749c-bzm9j\" (UID: \"8f715911-523f-4405-be5a-9e902db5609a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-bzm9j" Jun 15 01:48:45.022017 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:45.021965 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/8f715911-523f-4405-be5a-9e902db5609a-tls-certificates\") pod \"prometheus-operator-admission-webhook-5cf667749c-bzm9j\" (UID: \"8f715911-523f-4405-be5a-9e902db5609a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-bzm9j" Jun 15 01:48:45.022214 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:48:45.022131 2578 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jun 15 01:48:45.022214 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:48:45.022210 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f715911-523f-4405-be5a-9e902db5609a-tls-certificates podName:8f715911-523f-4405-be5a-9e902db5609a nodeName:}" failed. No retries permitted until 2026-06-15 01:48:45.522192755 +0000 UTC m=+249.235918705 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/8f715911-523f-4405-be5a-9e902db5609a-tls-certificates") pod "prometheus-operator-admission-webhook-5cf667749c-bzm9j" (UID: "8f715911-523f-4405-be5a-9e902db5609a") : secret "prometheus-operator-admission-webhook-tls" not found Jun 15 01:48:45.526061 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:45.526022 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/8f715911-523f-4405-be5a-9e902db5609a-tls-certificates\") pod \"prometheus-operator-admission-webhook-5cf667749c-bzm9j\" (UID: \"8f715911-523f-4405-be5a-9e902db5609a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-bzm9j" Jun 15 01:48:45.528827 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:45.528795 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/8f715911-523f-4405-be5a-9e902db5609a-tls-certificates\") pod \"prometheus-operator-admission-webhook-5cf667749c-bzm9j\" (UID: \"8f715911-523f-4405-be5a-9e902db5609a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-bzm9j" Jun 15 01:48:45.692890 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:45.692853 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-bzm9j" Jun 15 01:48:45.835895 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:45.835862 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-bzm9j"] Jun 15 01:48:45.839280 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:48:45.839253 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f715911_523f_4405_be5a_9e902db5609a.slice/crio-2f760a02c4e4b2ae5a3367809024a2cfb4087a591a015b2cb043747d6fd8b287 WatchSource:0}: Error finding container 2f760a02c4e4b2ae5a3367809024a2cfb4087a591a015b2cb043747d6fd8b287: Status 404 returned error can't find the container with id 2f760a02c4e4b2ae5a3367809024a2cfb4087a591a015b2cb043747d6fd8b287 Jun 15 01:48:46.381203 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:46.381168 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7b5846fddc-qfjkz"] Jun 15 01:48:46.384768 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:46.384740 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b5846fddc-qfjkz" Jun 15 01:48:46.394135 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:46.394079 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jun 15 01:48:46.396592 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:46.396566 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b5846fddc-qfjkz"] Jun 15 01:48:46.435073 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:46.435026 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0ba66708-00f3-48e6-bc68-c087405f1b3f-console-config\") pod \"console-7b5846fddc-qfjkz\" (UID: \"0ba66708-00f3-48e6-bc68-c087405f1b3f\") " pod="openshift-console/console-7b5846fddc-qfjkz" Jun 15 01:48:46.435073 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:46.435074 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0ba66708-00f3-48e6-bc68-c087405f1b3f-console-serving-cert\") pod \"console-7b5846fddc-qfjkz\" (UID: \"0ba66708-00f3-48e6-bc68-c087405f1b3f\") " pod="openshift-console/console-7b5846fddc-qfjkz" Jun 15 01:48:46.435362 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:46.435146 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ba66708-00f3-48e6-bc68-c087405f1b3f-trusted-ca-bundle\") pod \"console-7b5846fddc-qfjkz\" (UID: \"0ba66708-00f3-48e6-bc68-c087405f1b3f\") " pod="openshift-console/console-7b5846fddc-qfjkz" Jun 15 01:48:46.435362 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:46.435279 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0ba66708-00f3-48e6-bc68-c087405f1b3f-service-ca\") pod \"console-7b5846fddc-qfjkz\" (UID: \"0ba66708-00f3-48e6-bc68-c087405f1b3f\") " pod="openshift-console/console-7b5846fddc-qfjkz" Jun 15 01:48:46.435362 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:46.435337 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0ba66708-00f3-48e6-bc68-c087405f1b3f-oauth-serving-cert\") pod \"console-7b5846fddc-qfjkz\" (UID: \"0ba66708-00f3-48e6-bc68-c087405f1b3f\") " pod="openshift-console/console-7b5846fddc-qfjkz" Jun 15 01:48:46.435498 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:46.435373 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8cdx\" (UniqueName: \"kubernetes.io/projected/0ba66708-00f3-48e6-bc68-c087405f1b3f-kube-api-access-s8cdx\") pod \"console-7b5846fddc-qfjkz\" (UID: \"0ba66708-00f3-48e6-bc68-c087405f1b3f\") " pod="openshift-console/console-7b5846fddc-qfjkz" Jun 15 01:48:46.435498 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:46.435446 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0ba66708-00f3-48e6-bc68-c087405f1b3f-console-oauth-config\") pod \"console-7b5846fddc-qfjkz\" (UID: \"0ba66708-00f3-48e6-bc68-c087405f1b3f\") " pod="openshift-console/console-7b5846fddc-qfjkz" Jun 15 01:48:46.536408 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:46.536371 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8cdx\" (UniqueName: \"kubernetes.io/projected/0ba66708-00f3-48e6-bc68-c087405f1b3f-kube-api-access-s8cdx\") pod \"console-7b5846fddc-qfjkz\" (UID: \"0ba66708-00f3-48e6-bc68-c087405f1b3f\") " pod="openshift-console/console-7b5846fddc-qfjkz" Jun 15 01:48:46.536619 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:46.536578 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0ba66708-00f3-48e6-bc68-c087405f1b3f-console-oauth-config\") pod \"console-7b5846fddc-qfjkz\" (UID: \"0ba66708-00f3-48e6-bc68-c087405f1b3f\") " pod="openshift-console/console-7b5846fddc-qfjkz" Jun 15 01:48:46.536695 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:46.536665 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0ba66708-00f3-48e6-bc68-c087405f1b3f-console-config\") pod \"console-7b5846fddc-qfjkz\" (UID: \"0ba66708-00f3-48e6-bc68-c087405f1b3f\") " pod="openshift-console/console-7b5846fddc-qfjkz" Jun 15 01:48:46.536750 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:46.536711 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0ba66708-00f3-48e6-bc68-c087405f1b3f-console-serving-cert\") pod \"console-7b5846fddc-qfjkz\" (UID: \"0ba66708-00f3-48e6-bc68-c087405f1b3f\") " pod="openshift-console/console-7b5846fddc-qfjkz" Jun 15 01:48:46.536804 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:46.536759 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ba66708-00f3-48e6-bc68-c087405f1b3f-trusted-ca-bundle\") pod \"console-7b5846fddc-qfjkz\" (UID: \"0ba66708-00f3-48e6-bc68-c087405f1b3f\") " pod="openshift-console/console-7b5846fddc-qfjkz" Jun 15 01:48:46.536877 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:46.536862 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0ba66708-00f3-48e6-bc68-c087405f1b3f-service-ca\") pod \"console-7b5846fddc-qfjkz\" (UID: \"0ba66708-00f3-48e6-bc68-c087405f1b3f\") " pod="openshift-console/console-7b5846fddc-qfjkz" Jun 15 01:48:46.536930 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:46.536907 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0ba66708-00f3-48e6-bc68-c087405f1b3f-oauth-serving-cert\") pod \"console-7b5846fddc-qfjkz\" (UID: \"0ba66708-00f3-48e6-bc68-c087405f1b3f\") " pod="openshift-console/console-7b5846fddc-qfjkz" Jun 15 01:48:46.538010 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:46.537867 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0ba66708-00f3-48e6-bc68-c087405f1b3f-console-config\") pod \"console-7b5846fddc-qfjkz\" (UID: \"0ba66708-00f3-48e6-bc68-c087405f1b3f\") " pod="openshift-console/console-7b5846fddc-qfjkz" Jun 15 01:48:46.538010 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:46.537824 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0ba66708-00f3-48e6-bc68-c087405f1b3f-oauth-serving-cert\") pod \"console-7b5846fddc-qfjkz\" (UID: \"0ba66708-00f3-48e6-bc68-c087405f1b3f\") " pod="openshift-console/console-7b5846fddc-qfjkz" Jun 15 01:48:46.538010 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:46.537952 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ba66708-00f3-48e6-bc68-c087405f1b3f-trusted-ca-bundle\") pod \"console-7b5846fddc-qfjkz\" (UID: \"0ba66708-00f3-48e6-bc68-c087405f1b3f\") " pod="openshift-console/console-7b5846fddc-qfjkz" Jun 15 01:48:46.538413 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:46.538398 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0ba66708-00f3-48e6-bc68-c087405f1b3f-service-ca\") pod \"console-7b5846fddc-qfjkz\" (UID: \"0ba66708-00f3-48e6-bc68-c087405f1b3f\") " pod="openshift-console/console-7b5846fddc-qfjkz" Jun 15 01:48:46.540424 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:46.540382 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0ba66708-00f3-48e6-bc68-c087405f1b3f-console-oauth-config\") pod \"console-7b5846fddc-qfjkz\" (UID: \"0ba66708-00f3-48e6-bc68-c087405f1b3f\") " pod="openshift-console/console-7b5846fddc-qfjkz" Jun 15 01:48:46.540526 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:46.540453 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0ba66708-00f3-48e6-bc68-c087405f1b3f-console-serving-cert\") pod \"console-7b5846fddc-qfjkz\" (UID: \"0ba66708-00f3-48e6-bc68-c087405f1b3f\") " pod="openshift-console/console-7b5846fddc-qfjkz" Jun 15 01:48:46.545178 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:46.545136 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8cdx\" (UniqueName: \"kubernetes.io/projected/0ba66708-00f3-48e6-bc68-c087405f1b3f-kube-api-access-s8cdx\") pod \"console-7b5846fddc-qfjkz\" (UID: \"0ba66708-00f3-48e6-bc68-c087405f1b3f\") " pod="openshift-console/console-7b5846fddc-qfjkz" Jun 15 01:48:46.598408 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:46.598369 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-bzm9j" event={"ID":"8f715911-523f-4405-be5a-9e902db5609a","Type":"ContainerStarted","Data":"2f760a02c4e4b2ae5a3367809024a2cfb4087a591a015b2cb043747d6fd8b287"} Jun 15 01:48:46.699671 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:46.699577 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b5846fddc-qfjkz" Jun 15 01:48:47.948473 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:47.948401 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/47a7fc6e-077c-4eb7-9850-a31a72bec0d1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lgxjj\" (UID: \"47a7fc6e-077c-4eb7-9850-a31a72bec0d1\") " pod="openshift-insights/insights-runtime-extractor-lgxjj" Jun 15 01:48:47.950930 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:47.950908 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/47a7fc6e-077c-4eb7-9850-a31a72bec0d1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lgxjj\" (UID: \"47a7fc6e-077c-4eb7-9850-a31a72bec0d1\") " pod="openshift-insights/insights-runtime-extractor-lgxjj" Jun 15 01:48:48.087186 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:48.087154 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b5846fddc-qfjkz"] Jun 15 01:48:48.090197 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:48:48.090163 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ba66708_00f3_48e6_bc68_c087405f1b3f.slice/crio-61c5b80cc680aebb52957cb25e839fd9957976f05f51db81e3e5b84432ff5c89 WatchSource:0}: Error finding container 61c5b80cc680aebb52957cb25e839fd9957976f05f51db81e3e5b84432ff5c89: Status 404 returned error can't find the container with id 61c5b80cc680aebb52957cb25e839fd9957976f05f51db81e3e5b84432ff5c89 Jun 15 01:48:48.172521 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:48.172487 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jcjs4\"" Jun 15 01:48:48.180694 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:48.180667 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lgxjj" Jun 15 01:48:48.315936 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:48.315899 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lgxjj"] Jun 15 01:48:48.320827 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:48:48.320804 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47a7fc6e_077c_4eb7_9850_a31a72bec0d1.slice/crio-f4babd4a25a4252201d7c0bb78f404d0186c4e363a06a7b323aff07b549323c0 WatchSource:0}: Error finding container f4babd4a25a4252201d7c0bb78f404d0186c4e363a06a7b323aff07b549323c0: Status 404 returned error can't find the container with id f4babd4a25a4252201d7c0bb78f404d0186c4e363a06a7b323aff07b549323c0 Jun 15 01:48:48.554839 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:48.554747 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1719195-2a1b-4c96-abe3-c25bda154d3d-metrics-certs\") pod \"network-metrics-daemon-n4p82\" (UID: \"e1719195-2a1b-4c96-abe3-c25bda154d3d\") " pod="openshift-multus/network-metrics-daemon-n4p82" Jun 15 01:48:48.557194 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:48.557175 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1719195-2a1b-4c96-abe3-c25bda154d3d-metrics-certs\") pod \"network-metrics-daemon-n4p82\" (UID: \"e1719195-2a1b-4c96-abe3-c25bda154d3d\") " pod="openshift-multus/network-metrics-daemon-n4p82" Jun 15 01:48:48.564011 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:48.563992 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-d5972\"" Jun 15 01:48:48.571301 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:48.571285 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4p82" Jun 15 01:48:48.606877 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:48.606839 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lgxjj" event={"ID":"47a7fc6e-077c-4eb7-9850-a31a72bec0d1","Type":"ContainerStarted","Data":"f4babd4a25a4252201d7c0bb78f404d0186c4e363a06a7b323aff07b549323c0"} Jun 15 01:48:48.608449 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:48.608418 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5ff47d99dc-p2vfn" event={"ID":"2feade0e-acaf-4b90-8998-0df4be7c3c14","Type":"ContainerStarted","Data":"ee00517773f3a660088b0a37f7dcd5fac6025c3d1a9b92190cf99bd292db97b8"} Jun 15 01:48:48.610053 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:48.610024 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-bzm9j" event={"ID":"8f715911-523f-4405-be5a-9e902db5609a","Type":"ContainerStarted","Data":"25068363b62d3f6f22a3d95d8f963e68e299cedc851d5ec89d9143a8881516d3"} Jun 15 01:48:48.610227 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:48.610076 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-bzm9j" Jun 15 01:48:48.611740 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:48.611711 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b5846fddc-qfjkz" event={"ID":"0ba66708-00f3-48e6-bc68-c087405f1b3f","Type":"ContainerStarted","Data":"8557cda748188c7ca7e3ffaa1d997fdc6c99598f4c8be767a6805578c2730db1"} Jun 15 01:48:48.611855 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:48.611756 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b5846fddc-qfjkz" event={"ID":"0ba66708-00f3-48e6-bc68-c087405f1b3f","Type":"ContainerStarted","Data":"61c5b80cc680aebb52957cb25e839fd9957976f05f51db81e3e5b84432ff5c89"} Jun 15 01:48:48.615741 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:48.615720 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-bzm9j" Jun 15 01:48:48.626498 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:48.626422 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5ff47d99dc-p2vfn" podStartSLOduration=2.103297946 podStartE2EDuration="5.626398467s" totalImagesPullingTime="3.523100521s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 01:48:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 01:48:48.624947876 +0000 UTC m=+252.338673868" watchObservedRunningTime="2026-06-15 01:48:48.626398467 +0000 UTC m=+252.340124440" Jun 15 01:48:48.645171 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:48.645121 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7b5846fddc-qfjkz" podStartSLOduration=2.645105082 podStartE2EDuration="2.645105082s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 01:48:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 01:48:48.644447678 +0000 UTC m=+252.358173651" watchObservedRunningTime="2026-06-15 01:48:48.645105082 +0000 UTC m=+252.358831051" Jun 15 01:48:48.658610 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:48.658550 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-bzm9j" podStartSLOduration=2.540481942 podStartE2EDuration="4.658531084s" totalImagesPullingTime="2.118049142s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 01:48:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 01:48:48.658518765 +0000 UTC m=+252.372244737" watchObservedRunningTime="2026-06-15 01:48:48.658531084 +0000 UTC m=+252.372257065" Jun 15 01:48:48.702850 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:48.702818 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-n4p82"] Jun 15 01:48:48.705966 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:48:48.705940 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1719195_2a1b_4c96_abe3_c25bda154d3d.slice/crio-159e917695c907165afef1f3bb50c840709326fcd3f6b14348b5a5e794a5db76 WatchSource:0}: Error finding container 159e917695c907165afef1f3bb50c840709326fcd3f6b14348b5a5e794a5db76: Status 404 returned error can't find the container with id 159e917695c907165afef1f3bb50c840709326fcd3f6b14348b5a5e794a5db76 Jun 15 01:48:49.616952 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:49.616895 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-n4p82" event={"ID":"e1719195-2a1b-4c96-abe3-c25bda154d3d","Type":"ContainerStarted","Data":"159e917695c907165afef1f3bb50c840709326fcd3f6b14348b5a5e794a5db76"} Jun 15 01:48:50.623923 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:50.623873 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-n4p82" event={"ID":"e1719195-2a1b-4c96-abe3-c25bda154d3d","Type":"ContainerStarted","Data":"c68619e28a858f66426b2251f1ed02fdf69a35a69c77ca769311f0103670fe91"} Jun 15 01:48:52.632783 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:52.632741 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lgxjj" event={"ID":"47a7fc6e-077c-4eb7-9850-a31a72bec0d1","Type":"ContainerStarted","Data":"790a62c15118302cdfb8ec0c54cca0aa511ea6dae79badb3fe3ef1e6a0a6f36d"} Jun 15 01:48:52.633166 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:52.632833 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lgxjj" event={"ID":"47a7fc6e-077c-4eb7-9850-a31a72bec0d1","Type":"ContainerStarted","Data":"73cc472afc7125e5a57e298fa7416bcd90dde03351a97d38459b5cb78074064c"} Jun 15 01:48:52.634536 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:52.634509 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-n4p82" event={"ID":"e1719195-2a1b-4c96-abe3-c25bda154d3d","Type":"ContainerStarted","Data":"2c3cafc8bf3151842024385bdec5d85698a2a046429744c47a9328bd3f91b498"} Jun 15 01:48:52.651910 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:52.651858 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-n4p82" podStartSLOduration=254.500948225 podStartE2EDuration="4m15.651844012s" totalImagesPullingTime="1.150895787s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 01:44:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 01:48:52.649727378 +0000 UTC m=+256.363453357" watchObservedRunningTime="2026-06-15 01:48:52.651844012 +0000 UTC m=+256.365569983" Jun 15 01:48:54.313015 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:54.312986 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5ff47d99dc-p2vfn" Jun 15 01:48:54.313316 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:54.313058 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5ff47d99dc-p2vfn" Jun 15 01:48:54.314378 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:54.314353 2578 patch_prober.go:28] interesting pod/console-5ff47d99dc-p2vfn container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.132.0.15:8443/health\": dial tcp 10.132.0.15:8443: connect: connection refused" start-of-body= Jun 15 01:48:54.314481 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:54.314395 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-5ff47d99dc-p2vfn" podUID="2feade0e-acaf-4b90-8998-0df4be7c3c14" containerName="console" probeResult="failure" output="Get \"https://10.132.0.15:8443/health\": dial tcp 10.132.0.15:8443: connect: connection refused" Jun 15 01:48:54.644461 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:54.644417 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lgxjj" event={"ID":"47a7fc6e-077c-4eb7-9850-a31a72bec0d1","Type":"ContainerStarted","Data":"c5a42f95db1c44c8ace4309cc71dac3868d65d65bc528b220b335f7d3d1487e1"} Jun 15 01:48:54.662562 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:54.662515 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-lgxjj" podStartSLOduration=32.832095015 podStartE2EDuration="38.66250126s" totalImagesPullingTime="5.830406245s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 01:48:16 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 01:48:54.660550802 +0000 UTC m=+258.374276810" watchObservedRunningTime="2026-06-15 01:48:54.66250126 +0000 UTC m=+258.376227231" Jun 15 01:48:55.170969 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:55.170934 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-nbzph"] Jun 15 01:48:55.174579 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:55.174550 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-6b8d965dd5-78tw8"] Jun 15 01:48:55.174715 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:55.174653 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-nbzph" Jun 15 01:48:55.177721 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:55.177697 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-6h6sj\"" Jun 15 01:48:55.177866 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:55.177705 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 15 01:48:55.178518 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:55.178125 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-78tw8" Jun 15 01:48:55.178518 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:55.178163 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 15 01:48:55.178518 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:55.178228 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 15 01:48:55.178518 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:55.178340 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 15 01:48:55.178518 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:55.178162 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 15 01:48:55.178838 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:55.178794 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 15 01:48:55.180794 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:55.180452 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jun 15 01:48:55.180794 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:55.180747 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jun 15 01:48:55.180972 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:55.180844 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-gvscm\"" Jun 15 01:48:55.192621 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:55.189455 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-6b8d965dd5-78tw8"] Jun 15 01:48:55.317900 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:55.317860 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c4bb542e-3182-427d-ae55-e6e0897abf87-metrics-client-ca\") pod \"openshift-state-metrics-6b8d965dd5-78tw8\" (UID: \"c4bb542e-3182-427d-ae55-e6e0897abf87\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-78tw8" Jun 15 01:48:55.317900 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:55.317915 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/46e7619f-0f36-4119-841d-e8a8713a5ee5-node-exporter-wtmp\") pod \"node-exporter-nbzph\" (UID: \"46e7619f-0f36-4119-841d-e8a8713a5ee5\") " pod="openshift-monitoring/node-exporter-nbzph" Jun 15 01:48:55.318444 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:55.317992 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/46e7619f-0f36-4119-841d-e8a8713a5ee5-root\") pod \"node-exporter-nbzph\" (UID: \"46e7619f-0f36-4119-841d-e8a8713a5ee5\") " pod="openshift-monitoring/node-exporter-nbzph" Jun 15 01:48:55.318444 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:55.318036 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/46e7619f-0f36-4119-841d-e8a8713a5ee5-sys\") pod \"node-exporter-nbzph\" (UID: \"46e7619f-0f36-4119-841d-e8a8713a5ee5\") " pod="openshift-monitoring/node-exporter-nbzph" Jun 15 01:48:55.318444 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:55.318061 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4wpm\" (UniqueName: \"kubernetes.io/projected/c4bb542e-3182-427d-ae55-e6e0897abf87-kube-api-access-q4wpm\") pod \"openshift-state-metrics-6b8d965dd5-78tw8\" (UID: \"c4bb542e-3182-427d-ae55-e6e0897abf87\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-78tw8" Jun 15 01:48:55.318444 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:55.318107 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mpkf\" (UniqueName: \"kubernetes.io/projected/46e7619f-0f36-4119-841d-e8a8713a5ee5-kube-api-access-9mpkf\") pod \"node-exporter-nbzph\" (UID: \"46e7619f-0f36-4119-841d-e8a8713a5ee5\") " pod="openshift-monitoring/node-exporter-nbzph" Jun 15 01:48:55.318444 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:55.318144 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/46e7619f-0f36-4119-841d-e8a8713a5ee5-node-exporter-textfile\") pod \"node-exporter-nbzph\" (UID: \"46e7619f-0f36-4119-841d-e8a8713a5ee5\") " pod="openshift-monitoring/node-exporter-nbzph" Jun 15 01:48:55.318444 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:55.318184 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/46e7619f-0f36-4119-841d-e8a8713a5ee5-node-exporter-accelerators-collector-config\") pod \"node-exporter-nbzph\" (UID: \"46e7619f-0f36-4119-841d-e8a8713a5ee5\") " pod="openshift-monitoring/node-exporter-nbzph" Jun 15 01:48:55.318444 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:55.318216 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/46e7619f-0f36-4119-841d-e8a8713a5ee5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nbzph\" (UID: \"46e7619f-0f36-4119-841d-e8a8713a5ee5\") " pod="openshift-monitoring/node-exporter-nbzph" Jun 15 01:48:55.318444 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:55.318258 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c4bb542e-3182-427d-ae55-e6e0897abf87-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6b8d965dd5-78tw8\" (UID: \"c4bb542e-3182-427d-ae55-e6e0897abf87\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-78tw8" Jun 15 01:48:55.318444 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:55.318285 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/46e7619f-0f36-4119-841d-e8a8713a5ee5-node-exporter-tls\") pod \"node-exporter-nbzph\" (UID: \"46e7619f-0f36-4119-841d-e8a8713a5ee5\") " pod="openshift-monitoring/node-exporter-nbzph" Jun 15 01:48:55.318444 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:55.318315 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/46e7619f-0f36-4119-841d-e8a8713a5ee5-metrics-client-ca\") pod \"node-exporter-nbzph\" (UID: \"46e7619f-0f36-4119-841d-e8a8713a5ee5\") " pod="openshift-monitoring/node-exporter-nbzph" Jun 15 01:48:55.318444 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:55.318373 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c4bb542e-3182-427d-ae55-e6e0897abf87-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-6b8d965dd5-78tw8\" (UID: \"c4bb542e-3182-427d-ae55-e6e0897abf87\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-78tw8" Jun 15 01:48:55.419799 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:55.419766 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/46e7619f-0f36-4119-841d-e8a8713a5ee5-node-exporter-tls\") pod \"node-exporter-nbzph\" (UID: \"46e7619f-0f36-4119-841d-e8a8713a5ee5\") " pod="openshift-monitoring/node-exporter-nbzph" Jun 15 01:48:55.419799 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:55.419811 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/46e7619f-0f36-4119-841d-e8a8713a5ee5-metrics-client-ca\") pod \"node-exporter-nbzph\" (UID: \"46e7619f-0f36-4119-841d-e8a8713a5ee5\") " pod="openshift-monitoring/node-exporter-nbzph" Jun 15 01:48:55.420040 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:55.419852 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c4bb542e-3182-427d-ae55-e6e0897abf87-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-6b8d965dd5-78tw8\" (UID: \"c4bb542e-3182-427d-ae55-e6e0897abf87\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-78tw8" Jun 15 01:48:55.420040 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:55.419917 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c4bb542e-3182-427d-ae55-e6e0897abf87-metrics-client-ca\") pod \"openshift-state-metrics-6b8d965dd5-78tw8\" (UID: \"c4bb542e-3182-427d-ae55-e6e0897abf87\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-78tw8" Jun 15 01:48:55.420040 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:55.419948 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/46e7619f-0f36-4119-841d-e8a8713a5ee5-node-exporter-wtmp\") pod \"node-exporter-nbzph\" (UID: \"46e7619f-0f36-4119-841d-e8a8713a5ee5\") " pod="openshift-monitoring/node-exporter-nbzph" Jun 15 01:48:55.420040 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:55.419968 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/46e7619f-0f36-4119-841d-e8a8713a5ee5-root\") pod \"node-exporter-nbzph\" (UID: \"46e7619f-0f36-4119-841d-e8a8713a5ee5\") " pod="openshift-monitoring/node-exporter-nbzph" Jun 15 01:48:55.420040 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:55.419993 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/46e7619f-0f36-4119-841d-e8a8713a5ee5-sys\") pod \"node-exporter-nbzph\" (UID: \"46e7619f-0f36-4119-841d-e8a8713a5ee5\") " pod="openshift-monitoring/node-exporter-nbzph" Jun 15 01:48:55.420040 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:55.420016 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4wpm\" (UniqueName: \"kubernetes.io/projected/c4bb542e-3182-427d-ae55-e6e0897abf87-kube-api-access-q4wpm\") pod \"openshift-state-metrics-6b8d965dd5-78tw8\" (UID: \"c4bb542e-3182-427d-ae55-e6e0897abf87\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-78tw8" Jun 15 01:48:55.420362 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:55.420047 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mpkf\" (UniqueName: \"kubernetes.io/projected/46e7619f-0f36-4119-841d-e8a8713a5ee5-kube-api-access-9mpkf\") pod \"node-exporter-nbzph\" (UID: \"46e7619f-0f36-4119-841d-e8a8713a5ee5\") " pod="openshift-monitoring/node-exporter-nbzph" Jun 15 01:48:55.420362 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:55.420100 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/46e7619f-0f36-4119-841d-e8a8713a5ee5-node-exporter-textfile\") pod \"node-exporter-nbzph\" (UID: \"46e7619f-0f36-4119-841d-e8a8713a5ee5\") " pod="openshift-monitoring/node-exporter-nbzph" Jun 15 01:48:55.420362 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:55.420129 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/46e7619f-0f36-4119-841d-e8a8713a5ee5-node-exporter-accelerators-collector-config\") pod \"node-exporter-nbzph\" (UID: \"46e7619f-0f36-4119-841d-e8a8713a5ee5\") " pod="openshift-monitoring/node-exporter-nbzph" Jun 15 01:48:55.420362 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:55.420156 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/46e7619f-0f36-4119-841d-e8a8713a5ee5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nbzph\" (UID: \"46e7619f-0f36-4119-841d-e8a8713a5ee5\") " pod="openshift-monitoring/node-exporter-nbzph" Jun 15 01:48:55.420362 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:55.420207 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c4bb542e-3182-427d-ae55-e6e0897abf87-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6b8d965dd5-78tw8\" (UID: \"c4bb542e-3182-427d-ae55-e6e0897abf87\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-78tw8" Jun 15 01:48:55.420362 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:48:55.420326 2578 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jun 15 01:48:55.420612 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:48:55.420393 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4bb542e-3182-427d-ae55-e6e0897abf87-openshift-state-metrics-tls podName:c4bb542e-3182-427d-ae55-e6e0897abf87 nodeName:}" failed. No retries permitted until 2026-06-15 01:48:55.920373178 +0000 UTC m=+259.634099132 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/c4bb542e-3182-427d-ae55-e6e0897abf87-openshift-state-metrics-tls") pod "openshift-state-metrics-6b8d965dd5-78tw8" (UID: "c4bb542e-3182-427d-ae55-e6e0897abf87") : secret "openshift-state-metrics-tls" not found Jun 15 01:48:55.420612 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:55.420590 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/46e7619f-0f36-4119-841d-e8a8713a5ee5-node-exporter-wtmp\") pod \"node-exporter-nbzph\" (UID: \"46e7619f-0f36-4119-841d-e8a8713a5ee5\") " pod="openshift-monitoring/node-exporter-nbzph" Jun 15 01:48:55.420748 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:55.420677 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/46e7619f-0f36-4119-841d-e8a8713a5ee5-root\") pod \"node-exporter-nbzph\" (UID: \"46e7619f-0f36-4119-841d-e8a8713a5ee5\") " pod="openshift-monitoring/node-exporter-nbzph" Jun 15 01:48:55.420801 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:55.420788 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/46e7619f-0f36-4119-841d-e8a8713a5ee5-metrics-client-ca\") pod \"node-exporter-nbzph\" (UID: \"46e7619f-0f36-4119-841d-e8a8713a5ee5\") " pod="openshift-monitoring/node-exporter-nbzph" Jun 15 01:48:55.421020 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:55.420968 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/46e7619f-0f36-4119-841d-e8a8713a5ee5-node-exporter-textfile\") pod \"node-exporter-nbzph\" (UID: \"46e7619f-0f36-4119-841d-e8a8713a5ee5\") " pod="openshift-monitoring/node-exporter-nbzph" Jun 15 01:48:55.421156 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:55.421075 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/46e7619f-0f36-4119-841d-e8a8713a5ee5-sys\") pod \"node-exporter-nbzph\" (UID: \"46e7619f-0f36-4119-841d-e8a8713a5ee5\") " pod="openshift-monitoring/node-exporter-nbzph" Jun 15 01:48:55.421215 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:55.421160 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/46e7619f-0f36-4119-841d-e8a8713a5ee5-node-exporter-accelerators-collector-config\") pod \"node-exporter-nbzph\" (UID: \"46e7619f-0f36-4119-841d-e8a8713a5ee5\") " pod="openshift-monitoring/node-exporter-nbzph" Jun 15 01:48:55.422001 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:55.421956 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c4bb542e-3182-427d-ae55-e6e0897abf87-metrics-client-ca\") pod \"openshift-state-metrics-6b8d965dd5-78tw8\" (UID: \"c4bb542e-3182-427d-ae55-e6e0897abf87\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-78tw8" Jun 15 01:48:55.423693 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:55.423649 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/46e7619f-0f36-4119-841d-e8a8713a5ee5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nbzph\" (UID: \"46e7619f-0f36-4119-841d-e8a8713a5ee5\") " pod="openshift-monitoring/node-exporter-nbzph" Jun 15 01:48:55.423933 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:55.423909 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/46e7619f-0f36-4119-841d-e8a8713a5ee5-node-exporter-tls\") pod \"node-exporter-nbzph\" (UID: \"46e7619f-0f36-4119-841d-e8a8713a5ee5\") " pod="openshift-monitoring/node-exporter-nbzph" Jun 15 01:48:55.424597 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:55.424574 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c4bb542e-3182-427d-ae55-e6e0897abf87-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-6b8d965dd5-78tw8\" (UID: \"c4bb542e-3182-427d-ae55-e6e0897abf87\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-78tw8" Jun 15 01:48:55.429397 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:55.429377 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4wpm\" (UniqueName: \"kubernetes.io/projected/c4bb542e-3182-427d-ae55-e6e0897abf87-kube-api-access-q4wpm\") pod \"openshift-state-metrics-6b8d965dd5-78tw8\" (UID: \"c4bb542e-3182-427d-ae55-e6e0897abf87\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-78tw8" Jun 15 01:48:55.429806 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:55.429788 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mpkf\" (UniqueName: \"kubernetes.io/projected/46e7619f-0f36-4119-841d-e8a8713a5ee5-kube-api-access-9mpkf\") pod \"node-exporter-nbzph\" (UID: \"46e7619f-0f36-4119-841d-e8a8713a5ee5\") " pod="openshift-monitoring/node-exporter-nbzph" Jun 15 01:48:55.496223 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:55.496188 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-nbzph" Jun 15 01:48:55.508803 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:48:55.508777 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46e7619f_0f36_4119_841d_e8a8713a5ee5.slice/crio-9646c2a2509e33736310f8d8cb34392cd250b6f5da5a60869a728f929fbb91dc WatchSource:0}: Error finding container 9646c2a2509e33736310f8d8cb34392cd250b6f5da5a60869a728f929fbb91dc: Status 404 returned error can't find the container with id 9646c2a2509e33736310f8d8cb34392cd250b6f5da5a60869a728f929fbb91dc Jun 15 01:48:55.649458 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:55.649422 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nbzph" event={"ID":"46e7619f-0f36-4119-841d-e8a8713a5ee5","Type":"ContainerStarted","Data":"9646c2a2509e33736310f8d8cb34392cd250b6f5da5a60869a728f929fbb91dc"} Jun 15 01:48:55.923725 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:55.923692 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c4bb542e-3182-427d-ae55-e6e0897abf87-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6b8d965dd5-78tw8\" (UID: \"c4bb542e-3182-427d-ae55-e6e0897abf87\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-78tw8" Jun 15 01:48:55.926195 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:55.926175 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c4bb542e-3182-427d-ae55-e6e0897abf87-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6b8d965dd5-78tw8\" (UID: \"c4bb542e-3182-427d-ae55-e6e0897abf87\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-78tw8" Jun 15 01:48:56.103848 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:56.103809 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-78tw8" Jun 15 01:48:56.273808 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:56.273776 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-6b8d965dd5-78tw8"] Jun 15 01:48:56.276919 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:48:56.276892 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4bb542e_3182_427d_ae55_e6e0897abf87.slice/crio-28cd88885f53e55ac85d4b0fb56c76a648994e4740fd3f24c24b413dce681044 WatchSource:0}: Error finding container 28cd88885f53e55ac85d4b0fb56c76a648994e4740fd3f24c24b413dce681044: Status 404 returned error can't find the container with id 28cd88885f53e55ac85d4b0fb56c76a648994e4740fd3f24c24b413dce681044 Jun 15 01:48:56.656859 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:56.656823 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-78tw8" event={"ID":"c4bb542e-3182-427d-ae55-e6e0897abf87","Type":"ContainerStarted","Data":"b61bdbb309211542c4ecd640e7d560570caeac523b874e73f3520f33a01c2e1b"} Jun 15 01:48:56.657299 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:56.656897 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-78tw8" event={"ID":"c4bb542e-3182-427d-ae55-e6e0897abf87","Type":"ContainerStarted","Data":"224c90c97415f5400fe6a527ddce0aa35d239cdf7bd8f2cb61e8db673ecb9dac"} Jun 15 01:48:56.657299 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:56.656928 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-78tw8" event={"ID":"c4bb542e-3182-427d-ae55-e6e0897abf87","Type":"ContainerStarted","Data":"28cd88885f53e55ac85d4b0fb56c76a648994e4740fd3f24c24b413dce681044"} Jun 15 01:48:56.659186 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:56.658592 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nbzph" event={"ID":"46e7619f-0f36-4119-841d-e8a8713a5ee5","Type":"ContainerStarted","Data":"e586c9ebb7df7ad078ef5e2e400ea6e93ad3e35a31a197df2c0507118cf22684"} Jun 15 01:48:56.700423 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:56.700382 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7b5846fddc-qfjkz" Jun 15 01:48:56.700583 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:56.700443 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7b5846fddc-qfjkz" Jun 15 01:48:56.701810 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:56.701785 2578 patch_prober.go:28] interesting pod/console-7b5846fddc-qfjkz container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.132.0.17:8443/health\": dial tcp 10.132.0.17:8443: connect: connection refused" start-of-body= Jun 15 01:48:56.701921 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:56.701825 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-7b5846fddc-qfjkz" podUID="0ba66708-00f3-48e6-bc68-c087405f1b3f" containerName="console" probeResult="failure" output="Get \"https://10.132.0.17:8443/health\": dial tcp 10.132.0.17:8443: connect: connection refused" Jun 15 01:48:57.165014 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:57.164976 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7d8dcf9d4c-n6wws"] Jun 15 01:48:57.169011 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:57.168976 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7d8dcf9d4c-n6wws" Jun 15 01:48:57.171801 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:57.171774 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-bpm7r9i17dt35\"" Jun 15 01:48:57.171801 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:57.171794 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-lf8d5\"" Jun 15 01:48:57.172011 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:57.171917 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jun 15 01:48:57.172118 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:57.172101 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jun 15 01:48:57.172208 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:57.172125 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jun 15 01:48:57.172208 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:57.172162 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jun 15 01:48:57.172208 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:57.172129 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jun 15 01:48:57.178702 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:57.178679 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7d8dcf9d4c-n6wws"] Jun 15 01:48:57.338958 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:57.338916 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fbcce109-1cea-4846-86ef-198e1bd347e4-secret-grpc-tls\") pod \"thanos-querier-7d8dcf9d4c-n6wws\" (UID: \"fbcce109-1cea-4846-86ef-198e1bd347e4\") " pod="openshift-monitoring/thanos-querier-7d8dcf9d4c-n6wws" Jun 15 01:48:57.339153 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:57.338966 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/fbcce109-1cea-4846-86ef-198e1bd347e4-secret-thanos-querier-tls\") pod \"thanos-querier-7d8dcf9d4c-n6wws\" (UID: \"fbcce109-1cea-4846-86ef-198e1bd347e4\") " pod="openshift-monitoring/thanos-querier-7d8dcf9d4c-n6wws" Jun 15 01:48:57.339153 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:57.338988 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fbcce109-1cea-4846-86ef-198e1bd347e4-metrics-client-ca\") pod \"thanos-querier-7d8dcf9d4c-n6wws\" (UID: \"fbcce109-1cea-4846-86ef-198e1bd347e4\") " pod="openshift-monitoring/thanos-querier-7d8dcf9d4c-n6wws" Jun 15 01:48:57.339153 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:57.339017 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/fbcce109-1cea-4846-86ef-198e1bd347e4-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7d8dcf9d4c-n6wws\" (UID: \"fbcce109-1cea-4846-86ef-198e1bd347e4\") " pod="openshift-monitoring/thanos-querier-7d8dcf9d4c-n6wws" Jun 15 01:48:57.339153 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:57.339080 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/fbcce109-1cea-4846-86ef-198e1bd347e4-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7d8dcf9d4c-n6wws\" (UID: \"fbcce109-1cea-4846-86ef-198e1bd347e4\") " pod="openshift-monitoring/thanos-querier-7d8dcf9d4c-n6wws" Jun 15 01:48:57.339319 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:57.339157 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fbcce109-1cea-4846-86ef-198e1bd347e4-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7d8dcf9d4c-n6wws\" (UID: \"fbcce109-1cea-4846-86ef-198e1bd347e4\") " pod="openshift-monitoring/thanos-querier-7d8dcf9d4c-n6wws" Jun 15 01:48:57.339319 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:57.339202 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fbcce109-1cea-4846-86ef-198e1bd347e4-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7d8dcf9d4c-n6wws\" (UID: \"fbcce109-1cea-4846-86ef-198e1bd347e4\") " pod="openshift-monitoring/thanos-querier-7d8dcf9d4c-n6wws" Jun 15 01:48:57.339319 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:57.339226 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzr9l\" (UniqueName: \"kubernetes.io/projected/fbcce109-1cea-4846-86ef-198e1bd347e4-kube-api-access-jzr9l\") pod \"thanos-querier-7d8dcf9d4c-n6wws\" (UID: \"fbcce109-1cea-4846-86ef-198e1bd347e4\") " pod="openshift-monitoring/thanos-querier-7d8dcf9d4c-n6wws" Jun 15 01:48:57.440296 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:57.440222 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/fbcce109-1cea-4846-86ef-198e1bd347e4-secret-thanos-querier-tls\") pod \"thanos-querier-7d8dcf9d4c-n6wws\" (UID: \"fbcce109-1cea-4846-86ef-198e1bd347e4\") " pod="openshift-monitoring/thanos-querier-7d8dcf9d4c-n6wws" Jun 15 01:48:57.440296 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:57.440267 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fbcce109-1cea-4846-86ef-198e1bd347e4-metrics-client-ca\") pod \"thanos-querier-7d8dcf9d4c-n6wws\" (UID: \"fbcce109-1cea-4846-86ef-198e1bd347e4\") " pod="openshift-monitoring/thanos-querier-7d8dcf9d4c-n6wws" Jun 15 01:48:57.440296 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:57.440286 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/fbcce109-1cea-4846-86ef-198e1bd347e4-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7d8dcf9d4c-n6wws\" (UID: \"fbcce109-1cea-4846-86ef-198e1bd347e4\") " pod="openshift-monitoring/thanos-querier-7d8dcf9d4c-n6wws" Jun 15 01:48:57.440534 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:57.440310 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/fbcce109-1cea-4846-86ef-198e1bd347e4-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7d8dcf9d4c-n6wws\" (UID: \"fbcce109-1cea-4846-86ef-198e1bd347e4\") " pod="openshift-monitoring/thanos-querier-7d8dcf9d4c-n6wws" Jun 15 01:48:57.440534 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:57.440358 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fbcce109-1cea-4846-86ef-198e1bd347e4-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7d8dcf9d4c-n6wws\" (UID: \"fbcce109-1cea-4846-86ef-198e1bd347e4\") " pod="openshift-monitoring/thanos-querier-7d8dcf9d4c-n6wws" Jun 15 01:48:57.440534 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:57.440401 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fbcce109-1cea-4846-86ef-198e1bd347e4-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7d8dcf9d4c-n6wws\" (UID: \"fbcce109-1cea-4846-86ef-198e1bd347e4\") " pod="openshift-monitoring/thanos-querier-7d8dcf9d4c-n6wws" Jun 15 01:48:57.440534 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:57.440431 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzr9l\" (UniqueName: \"kubernetes.io/projected/fbcce109-1cea-4846-86ef-198e1bd347e4-kube-api-access-jzr9l\") pod \"thanos-querier-7d8dcf9d4c-n6wws\" (UID: \"fbcce109-1cea-4846-86ef-198e1bd347e4\") " pod="openshift-monitoring/thanos-querier-7d8dcf9d4c-n6wws" Jun 15 01:48:57.440748 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:57.440550 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fbcce109-1cea-4846-86ef-198e1bd347e4-secret-grpc-tls\") pod \"thanos-querier-7d8dcf9d4c-n6wws\" (UID: \"fbcce109-1cea-4846-86ef-198e1bd347e4\") " pod="openshift-monitoring/thanos-querier-7d8dcf9d4c-n6wws" Jun 15 01:48:57.441328 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:57.441253 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fbcce109-1cea-4846-86ef-198e1bd347e4-metrics-client-ca\") pod \"thanos-querier-7d8dcf9d4c-n6wws\" (UID: \"fbcce109-1cea-4846-86ef-198e1bd347e4\") " pod="openshift-monitoring/thanos-querier-7d8dcf9d4c-n6wws" Jun 15 01:48:57.443503 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:57.443473 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fbcce109-1cea-4846-86ef-198e1bd347e4-secret-grpc-tls\") pod \"thanos-querier-7d8dcf9d4c-n6wws\" (UID: \"fbcce109-1cea-4846-86ef-198e1bd347e4\") " pod="openshift-monitoring/thanos-querier-7d8dcf9d4c-n6wws" Jun 15 01:48:57.443503 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:57.443492 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/fbcce109-1cea-4846-86ef-198e1bd347e4-secret-thanos-querier-tls\") pod \"thanos-querier-7d8dcf9d4c-n6wws\" (UID: \"fbcce109-1cea-4846-86ef-198e1bd347e4\") " pod="openshift-monitoring/thanos-querier-7d8dcf9d4c-n6wws" Jun 15 01:48:57.443624 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:57.443508 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fbcce109-1cea-4846-86ef-198e1bd347e4-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7d8dcf9d4c-n6wws\" (UID: \"fbcce109-1cea-4846-86ef-198e1bd347e4\") " pod="openshift-monitoring/thanos-querier-7d8dcf9d4c-n6wws" Jun 15 01:48:57.443624 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:57.443486 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fbcce109-1cea-4846-86ef-198e1bd347e4-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7d8dcf9d4c-n6wws\" (UID: \"fbcce109-1cea-4846-86ef-198e1bd347e4\") " pod="openshift-monitoring/thanos-querier-7d8dcf9d4c-n6wws" Jun 15 01:48:57.443784 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:57.443768 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/fbcce109-1cea-4846-86ef-198e1bd347e4-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7d8dcf9d4c-n6wws\" (UID: \"fbcce109-1cea-4846-86ef-198e1bd347e4\") " pod="openshift-monitoring/thanos-querier-7d8dcf9d4c-n6wws" Jun 15 01:48:57.443909 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:57.443889 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/fbcce109-1cea-4846-86ef-198e1bd347e4-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7d8dcf9d4c-n6wws\" (UID: \"fbcce109-1cea-4846-86ef-198e1bd347e4\") " pod="openshift-monitoring/thanos-querier-7d8dcf9d4c-n6wws" Jun 15 01:48:57.449322 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:57.449302 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzr9l\" (UniqueName: \"kubernetes.io/projected/fbcce109-1cea-4846-86ef-198e1bd347e4-kube-api-access-jzr9l\") pod \"thanos-querier-7d8dcf9d4c-n6wws\" (UID: \"fbcce109-1cea-4846-86ef-198e1bd347e4\") " pod="openshift-monitoring/thanos-querier-7d8dcf9d4c-n6wws" Jun 15 01:48:57.481348 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:57.481327 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7d8dcf9d4c-n6wws" Jun 15 01:48:57.625259 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:57.625224 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7d8dcf9d4c-n6wws"] Jun 15 01:48:57.628650 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:48:57.628618 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbcce109_1cea_4846_86ef_198e1bd347e4.slice/crio-a04094eeac883468ce0b4bfac63bd58aa6c3027f44d4b94c3bcdd95a24faa8a0 WatchSource:0}: Error finding container a04094eeac883468ce0b4bfac63bd58aa6c3027f44d4b94c3bcdd95a24faa8a0: Status 404 returned error can't find the container with id a04094eeac883468ce0b4bfac63bd58aa6c3027f44d4b94c3bcdd95a24faa8a0 Jun 15 01:48:57.664372 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:57.664331 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-78tw8" event={"ID":"c4bb542e-3182-427d-ae55-e6e0897abf87","Type":"ContainerStarted","Data":"48267b44e62a4e702a12ccc54cf47e2a32f7e0f4045216bc95d28bcc2adb3d2b"} Jun 15 01:48:57.665544 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:57.665507 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d8dcf9d4c-n6wws" event={"ID":"fbcce109-1cea-4846-86ef-198e1bd347e4","Type":"ContainerStarted","Data":"a04094eeac883468ce0b4bfac63bd58aa6c3027f44d4b94c3bcdd95a24faa8a0"} Jun 15 01:48:57.666845 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:57.666822 2578 generic.go:356] "Generic (PLEG): container finished" podID="46e7619f-0f36-4119-841d-e8a8713a5ee5" containerID="e586c9ebb7df7ad078ef5e2e400ea6e93ad3e35a31a197df2c0507118cf22684" exitCode=0 Jun 15 01:48:57.666957 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:57.666864 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nbzph" event={"ID":"46e7619f-0f36-4119-841d-e8a8713a5ee5","Type":"ContainerDied","Data":"e586c9ebb7df7ad078ef5e2e400ea6e93ad3e35a31a197df2c0507118cf22684"} Jun 15 01:48:57.682208 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:57.682165 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-78tw8" podStartSLOduration=1.8358433889999999 podStartE2EDuration="2.682148837s" totalImagesPullingTime="846.305448ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 01:48:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 01:48:57.681121836 +0000 UTC m=+261.394847809" watchObservedRunningTime="2026-06-15 01:48:57.682148837 +0000 UTC m=+261.395874809" Jun 15 01:48:58.673978 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:58.673934 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nbzph" event={"ID":"46e7619f-0f36-4119-841d-e8a8713a5ee5","Type":"ContainerStarted","Data":"d9991a84ed4e2a27c64f929052d1ba96602af3f59bf279d25e696601284f22fb"} Jun 15 01:48:58.674451 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:58.674004 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nbzph" event={"ID":"46e7619f-0f36-4119-841d-e8a8713a5ee5","Type":"ContainerStarted","Data":"826d3ed9d74a8b3babe836032dd97f5a6225ff6fddb3c35775f08ac98e622001"} Jun 15 01:48:58.695143 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:58.695069 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-nbzph" podStartSLOduration=2.597436964 podStartE2EDuration="3.695052567s" totalImagesPullingTime="1.022481664s" totalInitContainerRuntime="75.133939ms" isStatefulPod=true podCreationTimestamp="2026-06-15 01:48:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 01:48:58.692978324 +0000 UTC m=+262.406704296" watchObservedRunningTime="2026-06-15 01:48:58.695052567 +0000 UTC m=+262.408778539" Jun 15 01:48:59.680032 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:59.679996 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d8dcf9d4c-n6wws" event={"ID":"fbcce109-1cea-4846-86ef-198e1bd347e4","Type":"ContainerStarted","Data":"2762fe69cdbd125d5776ed250b4434936254ff160787c44726c37bb49b241b51"} Jun 15 01:48:59.680454 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:59.680071 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d8dcf9d4c-n6wws" event={"ID":"fbcce109-1cea-4846-86ef-198e1bd347e4","Type":"ContainerStarted","Data":"5cef615f4b9eb0acafe6d8f965cb180f995e12116e0842074ef059cfdc6790fd"} Jun 15 01:48:59.680454 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:59.680129 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d8dcf9d4c-n6wws" event={"ID":"fbcce109-1cea-4846-86ef-198e1bd347e4","Type":"ContainerStarted","Data":"6e2457faed9db1120c66faf600a27f760beff58598dd5ae8f5d2d0f5b95dcfc6"} Jun 15 01:48:59.970834 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:59.970745 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5ff47d99dc-p2vfn"] Jun 15 01:48:59.971879 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:59.971855 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-589576dfc7-9kvt9"] Jun 15 01:48:59.975027 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:59.975007 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-589576dfc7-9kvt9" Jun 15 01:48:59.977491 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:59.977466 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jun 15 01:48:59.977602 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:59.977509 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-d7lg9\"" Jun 15 01:48:59.991018 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:48:59.990997 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-589576dfc7-9kvt9"] Jun 15 01:49:00.065457 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:00.065419 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pxtw\" (UniqueName: \"kubernetes.io/projected/52ed7008-98b2-43b1-910c-b75cde0911e3-kube-api-access-4pxtw\") pod \"monitoring-plugin-589576dfc7-9kvt9\" (UID: \"52ed7008-98b2-43b1-910c-b75cde0911e3\") " pod="openshift-monitoring/monitoring-plugin-589576dfc7-9kvt9" Jun 15 01:49:00.065631 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:00.065501 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/52ed7008-98b2-43b1-910c-b75cde0911e3-monitoring-plugin-cert\") pod \"monitoring-plugin-589576dfc7-9kvt9\" (UID: \"52ed7008-98b2-43b1-910c-b75cde0911e3\") " pod="openshift-monitoring/monitoring-plugin-589576dfc7-9kvt9" Jun 15 01:49:00.166722 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:00.166683 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pxtw\" (UniqueName: \"kubernetes.io/projected/52ed7008-98b2-43b1-910c-b75cde0911e3-kube-api-access-4pxtw\") pod \"monitoring-plugin-589576dfc7-9kvt9\" (UID: \"52ed7008-98b2-43b1-910c-b75cde0911e3\") " pod="openshift-monitoring/monitoring-plugin-589576dfc7-9kvt9" Jun 15 01:49:00.166902 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:00.166741 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/52ed7008-98b2-43b1-910c-b75cde0911e3-monitoring-plugin-cert\") pod \"monitoring-plugin-589576dfc7-9kvt9\" (UID: \"52ed7008-98b2-43b1-910c-b75cde0911e3\") " pod="openshift-monitoring/monitoring-plugin-589576dfc7-9kvt9" Jun 15 01:49:00.169341 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:00.169316 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/52ed7008-98b2-43b1-910c-b75cde0911e3-monitoring-plugin-cert\") pod \"monitoring-plugin-589576dfc7-9kvt9\" (UID: \"52ed7008-98b2-43b1-910c-b75cde0911e3\") " pod="openshift-monitoring/monitoring-plugin-589576dfc7-9kvt9" Jun 15 01:49:00.174847 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:00.174827 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pxtw\" (UniqueName: \"kubernetes.io/projected/52ed7008-98b2-43b1-910c-b75cde0911e3-kube-api-access-4pxtw\") pod \"monitoring-plugin-589576dfc7-9kvt9\" (UID: \"52ed7008-98b2-43b1-910c-b75cde0911e3\") " pod="openshift-monitoring/monitoring-plugin-589576dfc7-9kvt9" Jun 15 01:49:00.284881 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:00.284799 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-589576dfc7-9kvt9" Jun 15 01:49:00.438275 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:00.438225 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-589576dfc7-9kvt9"] Jun 15 01:49:00.486404 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:49:00.486372 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52ed7008_98b2_43b1_910c_b75cde0911e3.slice/crio-b856c0d2056c5da2d22bfa86640acd07a4e90f18a10899c81d0726c97903e9f5 WatchSource:0}: Error finding container b856c0d2056c5da2d22bfa86640acd07a4e90f18a10899c81d0726c97903e9f5: Status 404 returned error can't find the container with id b856c0d2056c5da2d22bfa86640acd07a4e90f18a10899c81d0726c97903e9f5 Jun 15 01:49:00.687075 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:00.687040 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d8dcf9d4c-n6wws" event={"ID":"fbcce109-1cea-4846-86ef-198e1bd347e4","Type":"ContainerStarted","Data":"d344b836417e40baf9bdbe08c1608de88ec9e5a70dd969e12fccfd696f612eb8"} Jun 15 01:49:00.687490 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:00.687148 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d8dcf9d4c-n6wws" event={"ID":"fbcce109-1cea-4846-86ef-198e1bd347e4","Type":"ContainerStarted","Data":"a0d84d2b1a6acd3e575c825184d22491c907aab8e1970ccbfe330c612a69ad8f"} Jun 15 01:49:00.688628 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:00.688606 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-589576dfc7-9kvt9" event={"ID":"52ed7008-98b2-43b1-910c-b75cde0911e3","Type":"ContainerStarted","Data":"b856c0d2056c5da2d22bfa86640acd07a4e90f18a10899c81d0726c97903e9f5"} Jun 15 01:49:01.697313 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:01.697268 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d8dcf9d4c-n6wws" event={"ID":"fbcce109-1cea-4846-86ef-198e1bd347e4","Type":"ContainerStarted","Data":"c5f871d972c4e5f42072ec7cd0219b2f482e5f65685cea78c150f9afd2b1b9ee"} Jun 15 01:49:01.697794 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:01.697462 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7d8dcf9d4c-n6wws" Jun 15 01:49:01.720700 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:01.720631 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7d8dcf9d4c-n6wws" podStartSLOduration=2.020050812 podStartE2EDuration="4.720608478s" totalImagesPullingTime="2.700557666s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 01:48:57 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 01:49:01.719237854 +0000 UTC m=+265.432963831" watchObservedRunningTime="2026-06-15 01:49:01.720608478 +0000 UTC m=+265.434334453" Jun 15 01:49:02.702849 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:02.702817 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-589576dfc7-9kvt9" event={"ID":"52ed7008-98b2-43b1-910c-b75cde0911e3","Type":"ContainerStarted","Data":"ab3a78cb5fec25c793b763692de6c3338494fcef70c2d8bec652e7015ce875d1"} Jun 15 01:49:02.703282 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:02.703192 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-589576dfc7-9kvt9" Jun 15 01:49:02.707645 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:02.707622 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-589576dfc7-9kvt9" Jun 15 01:49:02.720381 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:02.720333 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-589576dfc7-9kvt9" podStartSLOduration=2.356424382 podStartE2EDuration="3.720318874s" totalImagesPullingTime="1.363894492s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 01:48:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 01:49:02.719177978 +0000 UTC m=+266.432903952" watchObservedRunningTime="2026-06-15 01:49:02.720318874 +0000 UTC m=+266.434044845" Jun 15 01:49:04.481041 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:04.480999 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b5846fddc-qfjkz"] Jun 15 01:49:04.510269 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:04.510236 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-654cbf94d4-nphzt"] Jun 15 01:49:04.513512 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:04.513495 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-654cbf94d4-nphzt" Jun 15 01:49:04.522258 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:04.522232 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-654cbf94d4-nphzt"] Jun 15 01:49:04.608508 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:04.608480 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c-trusted-ca-bundle\") pod \"console-654cbf94d4-nphzt\" (UID: \"47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c\") " pod="openshift-console/console-654cbf94d4-nphzt" Jun 15 01:49:04.608508 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:04.608507 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcr8w\" (UniqueName: \"kubernetes.io/projected/47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c-kube-api-access-mcr8w\") pod \"console-654cbf94d4-nphzt\" (UID: \"47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c\") " pod="openshift-console/console-654cbf94d4-nphzt" Jun 15 01:49:04.608721 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:04.608533 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c-console-config\") pod \"console-654cbf94d4-nphzt\" (UID: \"47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c\") " pod="openshift-console/console-654cbf94d4-nphzt" Jun 15 01:49:04.608721 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:04.608557 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c-service-ca\") pod \"console-654cbf94d4-nphzt\" (UID: \"47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c\") " pod="openshift-console/console-654cbf94d4-nphzt" Jun 15 01:49:04.608721 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:04.608610 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c-console-serving-cert\") pod \"console-654cbf94d4-nphzt\" (UID: \"47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c\") " pod="openshift-console/console-654cbf94d4-nphzt" Jun 15 01:49:04.608721 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:04.608672 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c-oauth-serving-cert\") pod \"console-654cbf94d4-nphzt\" (UID: \"47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c\") " pod="openshift-console/console-654cbf94d4-nphzt" Jun 15 01:49:04.608721 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:04.608712 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c-console-oauth-config\") pod \"console-654cbf94d4-nphzt\" (UID: \"47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c\") " pod="openshift-console/console-654cbf94d4-nphzt" Jun 15 01:49:04.709476 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:04.709446 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c-console-oauth-config\") pod \"console-654cbf94d4-nphzt\" (UID: \"47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c\") " pod="openshift-console/console-654cbf94d4-nphzt" Jun 15 01:49:04.709695 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:04.709523 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c-trusted-ca-bundle\") pod \"console-654cbf94d4-nphzt\" (UID: \"47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c\") " pod="openshift-console/console-654cbf94d4-nphzt" Jun 15 01:49:04.709768 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:04.709710 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcr8w\" (UniqueName: \"kubernetes.io/projected/47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c-kube-api-access-mcr8w\") pod \"console-654cbf94d4-nphzt\" (UID: \"47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c\") " pod="openshift-console/console-654cbf94d4-nphzt" Jun 15 01:49:04.709827 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:04.709768 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c-console-config\") pod \"console-654cbf94d4-nphzt\" (UID: \"47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c\") " pod="openshift-console/console-654cbf94d4-nphzt" Jun 15 01:49:04.709827 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:04.709818 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c-service-ca\") pod \"console-654cbf94d4-nphzt\" (UID: \"47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c\") " pod="openshift-console/console-654cbf94d4-nphzt" Jun 15 01:49:04.709988 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:04.709964 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c-console-serving-cert\") pod \"console-654cbf94d4-nphzt\" (UID: \"47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c\") " pod="openshift-console/console-654cbf94d4-nphzt" Jun 15 01:49:04.710061 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:04.710036 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c-oauth-serving-cert\") pod \"console-654cbf94d4-nphzt\" (UID: \"47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c\") " pod="openshift-console/console-654cbf94d4-nphzt" Jun 15 01:49:04.710522 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:04.710494 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c-trusted-ca-bundle\") pod \"console-654cbf94d4-nphzt\" (UID: \"47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c\") " pod="openshift-console/console-654cbf94d4-nphzt" Jun 15 01:49:04.710630 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:04.710524 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c-console-config\") pod \"console-654cbf94d4-nphzt\" (UID: \"47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c\") " pod="openshift-console/console-654cbf94d4-nphzt" Jun 15 01:49:04.710630 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:04.710524 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c-service-ca\") pod \"console-654cbf94d4-nphzt\" (UID: \"47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c\") " pod="openshift-console/console-654cbf94d4-nphzt" Jun 15 01:49:04.710820 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:04.710801 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c-oauth-serving-cert\") pod \"console-654cbf94d4-nphzt\" (UID: \"47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c\") " pod="openshift-console/console-654cbf94d4-nphzt" Jun 15 01:49:04.712335 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:04.712309 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c-console-oauth-config\") pod \"console-654cbf94d4-nphzt\" (UID: \"47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c\") " pod="openshift-console/console-654cbf94d4-nphzt" Jun 15 01:49:04.712474 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:04.712456 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c-console-serving-cert\") pod \"console-654cbf94d4-nphzt\" (UID: \"47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c\") " pod="openshift-console/console-654cbf94d4-nphzt" Jun 15 01:49:04.719419 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:04.719399 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcr8w\" (UniqueName: \"kubernetes.io/projected/47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c-kube-api-access-mcr8w\") pod \"console-654cbf94d4-nphzt\" (UID: \"47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c\") " pod="openshift-console/console-654cbf94d4-nphzt" Jun 15 01:49:04.823152 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:04.823035 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-654cbf94d4-nphzt" Jun 15 01:49:04.958167 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:04.958138 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-654cbf94d4-nphzt"] Jun 15 01:49:04.960550 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:49:04.960524 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47ce7ef0_7ae2_48b8_b4b9_a5c06b1ab94c.slice/crio-0ff9815eb4edcff61663cc9313be758c11928975f8e189d5509f851e480e5742 WatchSource:0}: Error finding container 0ff9815eb4edcff61663cc9313be758c11928975f8e189d5509f851e480e5742: Status 404 returned error can't find the container with id 0ff9815eb4edcff61663cc9313be758c11928975f8e189d5509f851e480e5742 Jun 15 01:49:05.715922 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:05.715885 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-654cbf94d4-nphzt" event={"ID":"47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c","Type":"ContainerStarted","Data":"412fef52ba8aee61cbc8897f328bbe18525e8b5f3009f4e81b57d05f3b3da554"} Jun 15 01:49:05.716374 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:05.715936 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-654cbf94d4-nphzt" event={"ID":"47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c","Type":"ContainerStarted","Data":"0ff9815eb4edcff61663cc9313be758c11928975f8e189d5509f851e480e5742"} Jun 15 01:49:05.736964 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:05.736915 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-654cbf94d4-nphzt" podStartSLOduration=1.736901107 podStartE2EDuration="1.736901107s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 01:49:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 01:49:05.735108663 +0000 UTC m=+269.448834631" watchObservedRunningTime="2026-06-15 01:49:05.736901107 +0000 UTC m=+269.450627078" Jun 15 01:49:07.708627 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:07.708598 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7d8dcf9d4c-n6wws" Jun 15 01:49:14.824038 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:14.823999 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-654cbf94d4-nphzt" Jun 15 01:49:14.824586 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:14.824062 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-654cbf94d4-nphzt" Jun 15 01:49:14.825394 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:14.825369 2578 patch_prober.go:28] interesting pod/console-654cbf94d4-nphzt container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.132.0.21:8443/health\": dial tcp 10.132.0.21:8443: connect: connection refused" start-of-body= Jun 15 01:49:14.825500 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:14.825416 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-654cbf94d4-nphzt" podUID="47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c" containerName="console" probeResult="failure" output="Get \"https://10.132.0.21:8443/health\": dial tcp 10.132.0.21:8443: connect: connection refused" Jun 15 01:49:18.338527 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:49:18.338486 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-5b5c665f7c-7mtmh" podUID="26023146-86ca-4a77-b73c-429dd24c4e5d" Jun 15 01:49:18.338527 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:49:18.338485 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-bmrhc" podUID="7f6afa77-1931-4efb-aeae-7a7fdef97959" Jun 15 01:49:18.766409 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:18.766376 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bmrhc" Jun 15 01:49:18.766577 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:18.766376 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b5c665f7c-7mtmh" Jun 15 01:49:21.653625 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:21.653589 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/26023146-86ca-4a77-b73c-429dd24c4e5d-registry-tls\") pod \"image-registry-5b5c665f7c-7mtmh\" (UID: \"26023146-86ca-4a77-b73c-429dd24c4e5d\") " pod="openshift-image-registry/image-registry-5b5c665f7c-7mtmh" Jun 15 01:49:21.653625 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:21.653630 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7f6afa77-1931-4efb-aeae-7a7fdef97959-metrics-tls\") pod \"dns-default-bmrhc\" (UID: \"7f6afa77-1931-4efb-aeae-7a7fdef97959\") " pod="openshift-dns/dns-default-bmrhc" Jun 15 01:49:21.656155 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:21.656131 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7f6afa77-1931-4efb-aeae-7a7fdef97959-metrics-tls\") pod \"dns-default-bmrhc\" (UID: \"7f6afa77-1931-4efb-aeae-7a7fdef97959\") " pod="openshift-dns/dns-default-bmrhc" Jun 15 01:49:21.656155 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:21.656153 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/26023146-86ca-4a77-b73c-429dd24c4e5d-registry-tls\") pod \"image-registry-5b5c665f7c-7mtmh\" (UID: \"26023146-86ca-4a77-b73c-429dd24c4e5d\") " pod="openshift-image-registry/image-registry-5b5c665f7c-7mtmh" Jun 15 01:49:21.770437 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:21.770405 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2blxm\"" Jun 15 01:49:21.770604 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:21.770459 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-h8zgg\"" Jun 15 01:49:21.777762 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:21.777743 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b5c665f7c-7mtmh" Jun 15 01:49:21.777830 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:21.777789 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bmrhc" Jun 15 01:49:21.926045 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:21.926022 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bmrhc"] Jun 15 01:49:21.928021 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:49:21.927992 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f6afa77_1931_4efb_aeae_7a7fdef97959.slice/crio-934898b1d1fb4f800e833002801e7bc364272d9a965ccf411c109bb078af2704 WatchSource:0}: Error finding container 934898b1d1fb4f800e833002801e7bc364272d9a965ccf411c109bb078af2704: Status 404 returned error can't find the container with id 934898b1d1fb4f800e833002801e7bc364272d9a965ccf411c109bb078af2704 Jun 15 01:49:21.947788 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:21.947770 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5b5c665f7c-7mtmh"] Jun 15 01:49:21.949933 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:49:21.949903 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26023146_86ca_4a77_b73c_429dd24c4e5d.slice/crio-c5a6698ed239092f93cb6a81df11c447824c418d2ff049678a7cd41d94e690c0 WatchSource:0}: Error finding container c5a6698ed239092f93cb6a81df11c447824c418d2ff049678a7cd41d94e690c0: Status 404 returned error can't find the container with id c5a6698ed239092f93cb6a81df11c447824c418d2ff049678a7cd41d94e690c0 Jun 15 01:49:22.781769 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:22.781728 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bmrhc" event={"ID":"7f6afa77-1931-4efb-aeae-7a7fdef97959","Type":"ContainerStarted","Data":"934898b1d1fb4f800e833002801e7bc364272d9a965ccf411c109bb078af2704"} Jun 15 01:49:22.783761 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:22.783721 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b5c665f7c-7mtmh" event={"ID":"26023146-86ca-4a77-b73c-429dd24c4e5d","Type":"ContainerStarted","Data":"020be08ab9f31454c2da315159336da9be166173b738a911bffa7688908f3932"} Jun 15 01:49:22.783932 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:22.783788 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b5c665f7c-7mtmh" event={"ID":"26023146-86ca-4a77-b73c-429dd24c4e5d","Type":"ContainerStarted","Data":"c5a6698ed239092f93cb6a81df11c447824c418d2ff049678a7cd41d94e690c0"} Jun 15 01:49:22.783932 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:22.783866 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5b5c665f7c-7mtmh" Jun 15 01:49:22.806694 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:22.806545 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5b5c665f7c-7mtmh" podStartSLOduration=285.806510018 podStartE2EDuration="4m45.806510018s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 01:44:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 01:49:22.803240672 +0000 UTC m=+286.516966646" watchObservedRunningTime="2026-06-15 01:49:22.806510018 +0000 UTC m=+286.520235991" Jun 15 01:49:24.794348 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:24.794313 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bmrhc" event={"ID":"7f6afa77-1931-4efb-aeae-7a7fdef97959","Type":"ContainerStarted","Data":"66080e51d5c0b58b672b0a3adb0f701a58ac654a3de055b94830426756f99acc"} Jun 15 01:49:24.794770 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:24.794366 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bmrhc" event={"ID":"7f6afa77-1931-4efb-aeae-7a7fdef97959","Type":"ContainerStarted","Data":"8dc3f314c5501fab90455890cd5ec6d2ab4dcc19bb9ab73253112d40910760fd"} Jun 15 01:49:24.794770 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:24.794473 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-bmrhc" Jun 15 01:49:24.812191 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:24.812145 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-bmrhc" podStartSLOduration=251.948482302 podStartE2EDuration="4m13.812130083s" totalImagesPullingTime="1.863647781s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 01:45:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 01:49:24.810577983 +0000 UTC m=+288.524303974" watchObservedRunningTime="2026-06-15 01:49:24.812130083 +0000 UTC m=+288.525856054" Jun 15 01:49:24.824186 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:24.824155 2578 patch_prober.go:28] interesting pod/console-654cbf94d4-nphzt container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.132.0.21:8443/health\": dial tcp 10.132.0.21:8443: connect: connection refused" start-of-body= Jun 15 01:49:24.824319 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:24.824200 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-654cbf94d4-nphzt" podUID="47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c" containerName="console" probeResult="failure" output="Get \"https://10.132.0.21:8443/health\": dial tcp 10.132.0.21:8443: connect: connection refused" Jun 15 01:49:24.992224 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:24.992181 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5ff47d99dc-p2vfn" podUID="2feade0e-acaf-4b90-8998-0df4be7c3c14" containerName="console" containerID="cri-o://ee00517773f3a660088b0a37f7dcd5fac6025c3d1a9b92190cf99bd292db97b8" gracePeriod=15 Jun 15 01:49:29.500653 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:29.500605 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-7b5846fddc-qfjkz" podUID="0ba66708-00f3-48e6-bc68-c087405f1b3f" containerName="console" containerID="cri-o://8557cda748188c7ca7e3ffaa1d997fdc6c99598f4c8be767a6805578c2730db1" gracePeriod=15 Jun 15 01:49:34.800850 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:34.800813 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-bmrhc" Jun 15 01:49:34.823373 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:34.823341 2578 patch_prober.go:28] interesting pod/console-654cbf94d4-nphzt container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.132.0.21:8443/health\": dial tcp 10.132.0.21:8443: connect: connection refused" start-of-body= Jun 15 01:49:34.823517 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:34.823384 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-654cbf94d4-nphzt" podUID="47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c" containerName="console" probeResult="failure" output="Get \"https://10.132.0.21:8443/health\": dial tcp 10.132.0.21:8443: connect: connection refused" Jun 15 01:49:36.714861 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:36.714825 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cf286_64f557b0-78d8-45a6-9f54-756417231d6c/ovn-acl-logging/0.log" Jun 15 01:49:36.717936 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:36.717915 2578 kubelet.go:1648] "Image garbage collection succeeded" Jun 15 01:49:36.720853 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:36.720835 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cf286_64f557b0-78d8-45a6-9f54-756417231d6c/ovn-acl-logging/0.log" Jun 15 01:49:38.347494 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.347470 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5ff47d99dc-p2vfn" Jun 15 01:49:38.468289 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.468264 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b5846fddc-qfjkz" Jun 15 01:49:38.504714 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.504687 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2feade0e-acaf-4b90-8998-0df4be7c3c14-service-ca\") pod \"2feade0e-acaf-4b90-8998-0df4be7c3c14\" (UID: \"2feade0e-acaf-4b90-8998-0df4be7c3c14\") " Jun 15 01:49:38.504714 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.504718 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lpfgq\" (UniqueName: \"kubernetes.io/projected/2feade0e-acaf-4b90-8998-0df4be7c3c14-kube-api-access-lpfgq\") pod \"2feade0e-acaf-4b90-8998-0df4be7c3c14\" (UID: \"2feade0e-acaf-4b90-8998-0df4be7c3c14\") " Jun 15 01:49:38.504923 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.504735 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2feade0e-acaf-4b90-8998-0df4be7c3c14-console-serving-cert\") pod \"2feade0e-acaf-4b90-8998-0df4be7c3c14\" (UID: \"2feade0e-acaf-4b90-8998-0df4be7c3c14\") " Jun 15 01:49:38.504923 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.504785 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2feade0e-acaf-4b90-8998-0df4be7c3c14-console-oauth-config\") pod \"2feade0e-acaf-4b90-8998-0df4be7c3c14\" (UID: \"2feade0e-acaf-4b90-8998-0df4be7c3c14\") " Jun 15 01:49:38.504923 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.504823 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2feade0e-acaf-4b90-8998-0df4be7c3c14-oauth-serving-cert\") pod \"2feade0e-acaf-4b90-8998-0df4be7c3c14\" (UID: \"2feade0e-acaf-4b90-8998-0df4be7c3c14\") " Jun 15 01:49:38.504923 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.504842 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2feade0e-acaf-4b90-8998-0df4be7c3c14-console-config\") pod \"2feade0e-acaf-4b90-8998-0df4be7c3c14\" (UID: \"2feade0e-acaf-4b90-8998-0df4be7c3c14\") " Jun 15 01:49:38.505273 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.505208 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2feade0e-acaf-4b90-8998-0df4be7c3c14-service-ca" (OuterVolumeSpecName: "service-ca") pod "2feade0e-acaf-4b90-8998-0df4be7c3c14" (UID: "2feade0e-acaf-4b90-8998-0df4be7c3c14"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 01:49:38.505410 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.505293 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2feade0e-acaf-4b90-8998-0df4be7c3c14-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "2feade0e-acaf-4b90-8998-0df4be7c3c14" (UID: "2feade0e-acaf-4b90-8998-0df4be7c3c14"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 01:49:38.505410 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.505313 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2feade0e-acaf-4b90-8998-0df4be7c3c14-console-config" (OuterVolumeSpecName: "console-config") pod "2feade0e-acaf-4b90-8998-0df4be7c3c14" (UID: "2feade0e-acaf-4b90-8998-0df4be7c3c14"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 01:49:38.507074 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.507051 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2feade0e-acaf-4b90-8998-0df4be7c3c14-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "2feade0e-acaf-4b90-8998-0df4be7c3c14" (UID: "2feade0e-acaf-4b90-8998-0df4be7c3c14"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 15 01:49:38.507203 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.507103 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2feade0e-acaf-4b90-8998-0df4be7c3c14-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "2feade0e-acaf-4b90-8998-0df4be7c3c14" (UID: "2feade0e-acaf-4b90-8998-0df4be7c3c14"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 15 01:49:38.507358 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.507341 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2feade0e-acaf-4b90-8998-0df4be7c3c14-kube-api-access-lpfgq" (OuterVolumeSpecName: "kube-api-access-lpfgq") pod "2feade0e-acaf-4b90-8998-0df4be7c3c14" (UID: "2feade0e-acaf-4b90-8998-0df4be7c3c14"). InnerVolumeSpecName "kube-api-access-lpfgq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 01:49:38.605788 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.605754 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0ba66708-00f3-48e6-bc68-c087405f1b3f-console-oauth-config\") pod \"0ba66708-00f3-48e6-bc68-c087405f1b3f\" (UID: \"0ba66708-00f3-48e6-bc68-c087405f1b3f\") " Jun 15 01:49:38.605788 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.605800 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0ba66708-00f3-48e6-bc68-c087405f1b3f-console-config\") pod \"0ba66708-00f3-48e6-bc68-c087405f1b3f\" (UID: \"0ba66708-00f3-48e6-bc68-c087405f1b3f\") " Jun 15 01:49:38.606036 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.605857 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ba66708-00f3-48e6-bc68-c087405f1b3f-trusted-ca-bundle\") pod \"0ba66708-00f3-48e6-bc68-c087405f1b3f\" (UID: \"0ba66708-00f3-48e6-bc68-c087405f1b3f\") " Jun 15 01:49:38.606036 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.605968 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s8cdx\" (UniqueName: \"kubernetes.io/projected/0ba66708-00f3-48e6-bc68-c087405f1b3f-kube-api-access-s8cdx\") pod \"0ba66708-00f3-48e6-bc68-c087405f1b3f\" (UID: \"0ba66708-00f3-48e6-bc68-c087405f1b3f\") " Jun 15 01:49:38.606036 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.606015 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0ba66708-00f3-48e6-bc68-c087405f1b3f-oauth-serving-cert\") pod \"0ba66708-00f3-48e6-bc68-c087405f1b3f\" (UID: \"0ba66708-00f3-48e6-bc68-c087405f1b3f\") " Jun 15 01:49:38.606224 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.606046 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0ba66708-00f3-48e6-bc68-c087405f1b3f-service-ca\") pod \"0ba66708-00f3-48e6-bc68-c087405f1b3f\" (UID: \"0ba66708-00f3-48e6-bc68-c087405f1b3f\") " Jun 15 01:49:38.606224 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.606156 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0ba66708-00f3-48e6-bc68-c087405f1b3f-console-serving-cert\") pod \"0ba66708-00f3-48e6-bc68-c087405f1b3f\" (UID: \"0ba66708-00f3-48e6-bc68-c087405f1b3f\") " Jun 15 01:49:38.606392 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.606245 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ba66708-00f3-48e6-bc68-c087405f1b3f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "0ba66708-00f3-48e6-bc68-c087405f1b3f" (UID: "0ba66708-00f3-48e6-bc68-c087405f1b3f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 01:49:38.606515 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.606423 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2feade0e-acaf-4b90-8998-0df4be7c3c14-oauth-serving-cert\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:49:38.606515 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.606442 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2feade0e-acaf-4b90-8998-0df4be7c3c14-console-config\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:49:38.606515 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.606446 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ba66708-00f3-48e6-bc68-c087405f1b3f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0ba66708-00f3-48e6-bc68-c087405f1b3f" (UID: "0ba66708-00f3-48e6-bc68-c087405f1b3f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 01:49:38.606515 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.606461 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ba66708-00f3-48e6-bc68-c087405f1b3f-service-ca" (OuterVolumeSpecName: "service-ca") pod "0ba66708-00f3-48e6-bc68-c087405f1b3f" (UID: "0ba66708-00f3-48e6-bc68-c087405f1b3f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 01:49:38.606515 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.606456 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2feade0e-acaf-4b90-8998-0df4be7c3c14-service-ca\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:49:38.606515 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.606514 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lpfgq\" (UniqueName: \"kubernetes.io/projected/2feade0e-acaf-4b90-8998-0df4be7c3c14-kube-api-access-lpfgq\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:49:38.606724 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.606531 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2feade0e-acaf-4b90-8998-0df4be7c3c14-console-serving-cert\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:49:38.606724 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.606547 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2feade0e-acaf-4b90-8998-0df4be7c3c14-console-oauth-config\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:49:38.606724 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.606589 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ba66708-00f3-48e6-bc68-c087405f1b3f-console-config" (OuterVolumeSpecName: "console-config") pod "0ba66708-00f3-48e6-bc68-c087405f1b3f" (UID: "0ba66708-00f3-48e6-bc68-c087405f1b3f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 01:49:38.608156 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.608134 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ba66708-00f3-48e6-bc68-c087405f1b3f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0ba66708-00f3-48e6-bc68-c087405f1b3f" (UID: "0ba66708-00f3-48e6-bc68-c087405f1b3f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 15 01:49:38.608237 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.608197 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ba66708-00f3-48e6-bc68-c087405f1b3f-kube-api-access-s8cdx" (OuterVolumeSpecName: "kube-api-access-s8cdx") pod "0ba66708-00f3-48e6-bc68-c087405f1b3f" (UID: "0ba66708-00f3-48e6-bc68-c087405f1b3f"). InnerVolumeSpecName "kube-api-access-s8cdx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 01:49:38.608309 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.608286 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ba66708-00f3-48e6-bc68-c087405f1b3f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0ba66708-00f3-48e6-bc68-c087405f1b3f" (UID: "0ba66708-00f3-48e6-bc68-c087405f1b3f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 15 01:49:38.707056 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.706966 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0ba66708-00f3-48e6-bc68-c087405f1b3f-console-oauth-config\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:49:38.707056 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.706995 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0ba66708-00f3-48e6-bc68-c087405f1b3f-console-config\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:49:38.707056 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.707004 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ba66708-00f3-48e6-bc68-c087405f1b3f-trusted-ca-bundle\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:49:38.707056 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.707014 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s8cdx\" (UniqueName: \"kubernetes.io/projected/0ba66708-00f3-48e6-bc68-c087405f1b3f-kube-api-access-s8cdx\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:49:38.707056 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.707023 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0ba66708-00f3-48e6-bc68-c087405f1b3f-oauth-serving-cert\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:49:38.707056 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.707032 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0ba66708-00f3-48e6-bc68-c087405f1b3f-service-ca\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:49:38.707056 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.707040 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0ba66708-00f3-48e6-bc68-c087405f1b3f-console-serving-cert\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:49:38.848166 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.848128 2578 generic.go:356] "Generic (PLEG): container finished" podID="2feade0e-acaf-4b90-8998-0df4be7c3c14" containerID="ee00517773f3a660088b0a37f7dcd5fac6025c3d1a9b92190cf99bd292db97b8" exitCode=0 Jun 15 01:49:38.848346 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.848183 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5ff47d99dc-p2vfn" Jun 15 01:49:38.848346 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.848217 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5ff47d99dc-p2vfn" event={"ID":"2feade0e-acaf-4b90-8998-0df4be7c3c14","Type":"ContainerDied","Data":"ee00517773f3a660088b0a37f7dcd5fac6025c3d1a9b92190cf99bd292db97b8"} Jun 15 01:49:38.848346 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.848269 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5ff47d99dc-p2vfn" event={"ID":"2feade0e-acaf-4b90-8998-0df4be7c3c14","Type":"ContainerDied","Data":"ba3b5eb7bae57d0a1b419ac0afd41c3f1aec073c2caacff09fd5ae4cb43307fe"} Jun 15 01:49:38.848346 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.848289 2578 scope.go:117] "RemoveContainer" containerID="ee00517773f3a660088b0a37f7dcd5fac6025c3d1a9b92190cf99bd292db97b8" Jun 15 01:49:38.849711 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.849673 2578 generic.go:356] "Generic (PLEG): container finished" podID="0ba66708-00f3-48e6-bc68-c087405f1b3f" containerID="8557cda748188c7ca7e3ffaa1d997fdc6c99598f4c8be767a6805578c2730db1" exitCode=0 Jun 15 01:49:38.849711 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.849703 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b5846fddc-qfjkz" event={"ID":"0ba66708-00f3-48e6-bc68-c087405f1b3f","Type":"ContainerDied","Data":"8557cda748188c7ca7e3ffaa1d997fdc6c99598f4c8be767a6805578c2730db1"} Jun 15 01:49:38.849882 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.849731 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b5846fddc-qfjkz" Jun 15 01:49:38.849882 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.849744 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b5846fddc-qfjkz" event={"ID":"0ba66708-00f3-48e6-bc68-c087405f1b3f","Type":"ContainerDied","Data":"61c5b80cc680aebb52957cb25e839fd9957976f05f51db81e3e5b84432ff5c89"} Jun 15 01:49:38.856950 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.856932 2578 scope.go:117] "RemoveContainer" containerID="ee00517773f3a660088b0a37f7dcd5fac6025c3d1a9b92190cf99bd292db97b8" Jun 15 01:49:38.857262 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:49:38.857229 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee00517773f3a660088b0a37f7dcd5fac6025c3d1a9b92190cf99bd292db97b8\": container with ID starting with ee00517773f3a660088b0a37f7dcd5fac6025c3d1a9b92190cf99bd292db97b8 not found: ID does not exist" containerID="ee00517773f3a660088b0a37f7dcd5fac6025c3d1a9b92190cf99bd292db97b8" Jun 15 01:49:38.857356 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.857267 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee00517773f3a660088b0a37f7dcd5fac6025c3d1a9b92190cf99bd292db97b8"} err="failed to get container status \"ee00517773f3a660088b0a37f7dcd5fac6025c3d1a9b92190cf99bd292db97b8\": rpc error: code = NotFound desc = could not find container \"ee00517773f3a660088b0a37f7dcd5fac6025c3d1a9b92190cf99bd292db97b8\": container with ID starting with ee00517773f3a660088b0a37f7dcd5fac6025c3d1a9b92190cf99bd292db97b8 not found: ID does not exist" Jun 15 01:49:38.857356 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.857298 2578 scope.go:117] "RemoveContainer" containerID="8557cda748188c7ca7e3ffaa1d997fdc6c99598f4c8be767a6805578c2730db1" Jun 15 01:49:38.865033 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.865014 2578 scope.go:117] "RemoveContainer" containerID="8557cda748188c7ca7e3ffaa1d997fdc6c99598f4c8be767a6805578c2730db1" Jun 15 01:49:38.865270 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.865251 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5ff47d99dc-p2vfn"] Jun 15 01:49:38.865325 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:49:38.865307 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8557cda748188c7ca7e3ffaa1d997fdc6c99598f4c8be767a6805578c2730db1\": container with ID starting with 8557cda748188c7ca7e3ffaa1d997fdc6c99598f4c8be767a6805578c2730db1 not found: ID does not exist" containerID="8557cda748188c7ca7e3ffaa1d997fdc6c99598f4c8be767a6805578c2730db1" Jun 15 01:49:38.865366 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.865330 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8557cda748188c7ca7e3ffaa1d997fdc6c99598f4c8be767a6805578c2730db1"} err="failed to get container status \"8557cda748188c7ca7e3ffaa1d997fdc6c99598f4c8be767a6805578c2730db1\": rpc error: code = NotFound desc = could not find container \"8557cda748188c7ca7e3ffaa1d997fdc6c99598f4c8be767a6805578c2730db1\": container with ID starting with 8557cda748188c7ca7e3ffaa1d997fdc6c99598f4c8be767a6805578c2730db1 not found: ID does not exist" Jun 15 01:49:38.869050 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.869029 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5ff47d99dc-p2vfn"] Jun 15 01:49:38.879026 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.879007 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b5846fddc-qfjkz"] Jun 15 01:49:38.884864 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:38.884841 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7b5846fddc-qfjkz"] Jun 15 01:49:40.765424 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:40.765386 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ba66708-00f3-48e6-bc68-c087405f1b3f" path="/var/lib/kubelet/pods/0ba66708-00f3-48e6-bc68-c087405f1b3f/volumes" Jun 15 01:49:40.765921 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:40.765903 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2feade0e-acaf-4b90-8998-0df4be7c3c14" path="/var/lib/kubelet/pods/2feade0e-acaf-4b90-8998-0df4be7c3c14/volumes" Jun 15 01:49:41.782310 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:41.782275 2578 patch_prober.go:28] interesting pod/image-registry-5b5c665f7c-7mtmh container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 15 01:49:41.782691 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:41.782361 2578 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-5b5c665f7c-7mtmh" podUID="26023146-86ca-4a77-b73c-429dd24c4e5d" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 15 01:49:43.178482 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:43.178449 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bmrhc_7f6afa77-1931-4efb-aeae-7a7fdef97959/dns/0.log" Jun 15 01:49:43.378355 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:43.378326 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bmrhc_7f6afa77-1931-4efb-aeae-7a7fdef97959/kube-rbac-proxy/0.log" Jun 15 01:49:43.794679 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:43.794650 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5b5c665f7c-7mtmh" Jun 15 01:49:44.778270 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:44.778238 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pmkl7_3cf2d35f-dae7-42c6-8f35-5b0ccb5e4750/dns-node-resolver/0.log" Jun 15 01:49:44.827555 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:44.827529 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-654cbf94d4-nphzt" Jun 15 01:49:44.831501 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:44.831476 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-654cbf94d4-nphzt" Jun 15 01:49:44.979393 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:44.979367 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5b5c665f7c-7mtmh_26023146-86ca-4a77-b73c-429dd24c4e5d/registry/0.log" Jun 15 01:49:45.778674 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:45.778643 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-r8dqf_17743aa6-c9cb-458c-b3d3-16c6a774171a/node-ca/0.log" Jun 15 01:49:45.978567 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:45.978525 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-58cf549cb4-mc482_d4979e05-89c5-482b-a895-4f1f9ae54e93/router/0.log" Jun 15 01:49:46.179994 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:49:46.179962 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-48xtp_878da7f4-63c4-45b4-a4c6-fc6a72556516/serve-healthcheck-canary/0.log" Jun 15 01:50:11.576962 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:11.576925 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5b5c665f7c-7mtmh"] Jun 15 01:50:33.736176 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:33.736096 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-654cbf94d4-nphzt"] Jun 15 01:50:36.597347 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:36.597284 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5b5c665f7c-7mtmh" podUID="26023146-86ca-4a77-b73c-429dd24c4e5d" containerName="registry" containerID="cri-o://020be08ab9f31454c2da315159336da9be166173b738a911bffa7688908f3932" gracePeriod=30 Jun 15 01:50:37.849118 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:37.849070 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b5c665f7c-7mtmh" Jun 15 01:50:37.984267 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:37.984159 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/26023146-86ca-4a77-b73c-429dd24c4e5d-installation-pull-secrets\") pod \"26023146-86ca-4a77-b73c-429dd24c4e5d\" (UID: \"26023146-86ca-4a77-b73c-429dd24c4e5d\") " Jun 15 01:50:37.984267 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:37.984238 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/26023146-86ca-4a77-b73c-429dd24c4e5d-bound-sa-token\") pod \"26023146-86ca-4a77-b73c-429dd24c4e5d\" (UID: \"26023146-86ca-4a77-b73c-429dd24c4e5d\") " Jun 15 01:50:37.984267 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:37.984263 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/26023146-86ca-4a77-b73c-429dd24c4e5d-ca-trust-extracted\") pod \"26023146-86ca-4a77-b73c-429dd24c4e5d\" (UID: \"26023146-86ca-4a77-b73c-429dd24c4e5d\") " Jun 15 01:50:37.984565 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:37.984309 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/26023146-86ca-4a77-b73c-429dd24c4e5d-image-registry-private-configuration\") pod \"26023146-86ca-4a77-b73c-429dd24c4e5d\" (UID: \"26023146-86ca-4a77-b73c-429dd24c4e5d\") " Jun 15 01:50:37.984565 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:37.984342 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/26023146-86ca-4a77-b73c-429dd24c4e5d-trusted-ca\") pod \"26023146-86ca-4a77-b73c-429dd24c4e5d\" (UID: \"26023146-86ca-4a77-b73c-429dd24c4e5d\") " Jun 15 01:50:37.984565 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:37.984387 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/26023146-86ca-4a77-b73c-429dd24c4e5d-registry-certificates\") pod \"26023146-86ca-4a77-b73c-429dd24c4e5d\" (UID: \"26023146-86ca-4a77-b73c-429dd24c4e5d\") " Jun 15 01:50:37.984565 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:37.984418 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/26023146-86ca-4a77-b73c-429dd24c4e5d-registry-tls\") pod \"26023146-86ca-4a77-b73c-429dd24c4e5d\" (UID: \"26023146-86ca-4a77-b73c-429dd24c4e5d\") " Jun 15 01:50:37.984565 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:37.984463 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l87tg\" (UniqueName: \"kubernetes.io/projected/26023146-86ca-4a77-b73c-429dd24c4e5d-kube-api-access-l87tg\") pod \"26023146-86ca-4a77-b73c-429dd24c4e5d\" (UID: \"26023146-86ca-4a77-b73c-429dd24c4e5d\") " Jun 15 01:50:37.984892 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:37.984865 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26023146-86ca-4a77-b73c-429dd24c4e5d-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "26023146-86ca-4a77-b73c-429dd24c4e5d" (UID: "26023146-86ca-4a77-b73c-429dd24c4e5d"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 01:50:37.985009 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:37.984856 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26023146-86ca-4a77-b73c-429dd24c4e5d-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "26023146-86ca-4a77-b73c-429dd24c4e5d" (UID: "26023146-86ca-4a77-b73c-429dd24c4e5d"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 01:50:37.987016 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:37.986982 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26023146-86ca-4a77-b73c-429dd24c4e5d-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "26023146-86ca-4a77-b73c-429dd24c4e5d" (UID: "26023146-86ca-4a77-b73c-429dd24c4e5d"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 01:50:37.987296 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:37.987253 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26023146-86ca-4a77-b73c-429dd24c4e5d-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "26023146-86ca-4a77-b73c-429dd24c4e5d" (UID: "26023146-86ca-4a77-b73c-429dd24c4e5d"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 15 01:50:37.987296 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:37.987267 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26023146-86ca-4a77-b73c-429dd24c4e5d-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "26023146-86ca-4a77-b73c-429dd24c4e5d" (UID: "26023146-86ca-4a77-b73c-429dd24c4e5d"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 01:50:37.987615 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:37.987595 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26023146-86ca-4a77-b73c-429dd24c4e5d-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "26023146-86ca-4a77-b73c-429dd24c4e5d" (UID: "26023146-86ca-4a77-b73c-429dd24c4e5d"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 15 01:50:37.987661 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:37.987615 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26023146-86ca-4a77-b73c-429dd24c4e5d-kube-api-access-l87tg" (OuterVolumeSpecName: "kube-api-access-l87tg") pod "26023146-86ca-4a77-b73c-429dd24c4e5d" (UID: "26023146-86ca-4a77-b73c-429dd24c4e5d"). InnerVolumeSpecName "kube-api-access-l87tg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 01:50:37.993429 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:37.993407 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26023146-86ca-4a77-b73c-429dd24c4e5d-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "26023146-86ca-4a77-b73c-429dd24c4e5d" (UID: "26023146-86ca-4a77-b73c-429dd24c4e5d"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 01:50:38.081638 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:38.081604 2578 generic.go:356] "Generic (PLEG): container finished" podID="26023146-86ca-4a77-b73c-429dd24c4e5d" containerID="020be08ab9f31454c2da315159336da9be166173b738a911bffa7688908f3932" exitCode=0 Jun 15 01:50:38.081813 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:38.081647 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b5c665f7c-7mtmh" Jun 15 01:50:38.081813 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:38.081680 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b5c665f7c-7mtmh" event={"ID":"26023146-86ca-4a77-b73c-429dd24c4e5d","Type":"ContainerDied","Data":"020be08ab9f31454c2da315159336da9be166173b738a911bffa7688908f3932"} Jun 15 01:50:38.081813 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:38.081721 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b5c665f7c-7mtmh" event={"ID":"26023146-86ca-4a77-b73c-429dd24c4e5d","Type":"ContainerDied","Data":"c5a6698ed239092f93cb6a81df11c447824c418d2ff049678a7cd41d94e690c0"} Jun 15 01:50:38.081813 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:38.081741 2578 scope.go:117] "RemoveContainer" containerID="020be08ab9f31454c2da315159336da9be166173b738a911bffa7688908f3932" Jun 15 01:50:38.085024 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:38.085005 2578 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/26023146-86ca-4a77-b73c-429dd24c4e5d-image-registry-private-configuration\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:50:38.085122 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:38.085025 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/26023146-86ca-4a77-b73c-429dd24c4e5d-trusted-ca\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:50:38.085122 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:38.085036 2578 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/26023146-86ca-4a77-b73c-429dd24c4e5d-registry-certificates\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:50:38.085122 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:38.085045 2578 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/26023146-86ca-4a77-b73c-429dd24c4e5d-registry-tls\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:50:38.085122 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:38.085054 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l87tg\" (UniqueName: \"kubernetes.io/projected/26023146-86ca-4a77-b73c-429dd24c4e5d-kube-api-access-l87tg\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:50:38.085122 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:38.085063 2578 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/26023146-86ca-4a77-b73c-429dd24c4e5d-installation-pull-secrets\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:50:38.085122 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:38.085071 2578 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/26023146-86ca-4a77-b73c-429dd24c4e5d-bound-sa-token\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:50:38.085122 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:38.085079 2578 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/26023146-86ca-4a77-b73c-429dd24c4e5d-ca-trust-extracted\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:50:38.090800 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:38.090779 2578 scope.go:117] "RemoveContainer" containerID="020be08ab9f31454c2da315159336da9be166173b738a911bffa7688908f3932" Jun 15 01:50:38.091059 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:50:38.091041 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"020be08ab9f31454c2da315159336da9be166173b738a911bffa7688908f3932\": container with ID starting with 020be08ab9f31454c2da315159336da9be166173b738a911bffa7688908f3932 not found: ID does not exist" containerID="020be08ab9f31454c2da315159336da9be166173b738a911bffa7688908f3932" Jun 15 01:50:38.091133 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:38.091068 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"020be08ab9f31454c2da315159336da9be166173b738a911bffa7688908f3932"} err="failed to get container status \"020be08ab9f31454c2da315159336da9be166173b738a911bffa7688908f3932\": rpc error: code = NotFound desc = could not find container \"020be08ab9f31454c2da315159336da9be166173b738a911bffa7688908f3932\": container with ID starting with 020be08ab9f31454c2da315159336da9be166173b738a911bffa7688908f3932 not found: ID does not exist" Jun 15 01:50:38.103094 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:38.103060 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5b5c665f7c-7mtmh"] Jun 15 01:50:38.107292 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:38.107272 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5b5c665f7c-7mtmh"] Jun 15 01:50:38.765657 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:38.765623 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26023146-86ca-4a77-b73c-429dd24c4e5d" path="/var/lib/kubelet/pods/26023146-86ca-4a77-b73c-429dd24c4e5d/volumes" Jun 15 01:50:58.760423 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:58.760359 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-654cbf94d4-nphzt" podUID="47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c" containerName="console" containerID="cri-o://412fef52ba8aee61cbc8897f328bbe18525e8b5f3009f4e81b57d05f3b3da554" gracePeriod=15 Jun 15 01:50:59.010996 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:59.010937 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-654cbf94d4-nphzt" Jun 15 01:50:59.043164 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:59.043136 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c-console-oauth-config\") pod \"47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c\" (UID: \"47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c\") " Jun 15 01:50:59.043343 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:59.043189 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c-console-serving-cert\") pod \"47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c\" (UID: \"47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c\") " Jun 15 01:50:59.043343 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:59.043218 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c-oauth-serving-cert\") pod \"47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c\" (UID: \"47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c\") " Jun 15 01:50:59.043343 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:59.043243 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c-trusted-ca-bundle\") pod \"47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c\" (UID: \"47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c\") " Jun 15 01:50:59.043343 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:59.043284 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c-console-config\") pod \"47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c\" (UID: \"47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c\") " Jun 15 01:50:59.043343 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:59.043308 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mcr8w\" (UniqueName: \"kubernetes.io/projected/47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c-kube-api-access-mcr8w\") pod \"47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c\" (UID: \"47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c\") " Jun 15 01:50:59.043343 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:59.043335 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c-service-ca\") pod \"47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c\" (UID: \"47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c\") " Jun 15 01:50:59.043792 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:59.043674 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c" (UID: "47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 01:50:59.043928 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:59.043796 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c-service-ca" (OuterVolumeSpecName: "service-ca") pod "47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c" (UID: "47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 01:50:59.043989 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:59.043956 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c" (UID: "47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 01:50:59.044046 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:59.044020 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c-console-config" (OuterVolumeSpecName: "console-config") pod "47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c" (UID: "47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 01:50:59.045546 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:59.045520 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c" (UID: "47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 15 01:50:59.045666 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:59.045638 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c-kube-api-access-mcr8w" (OuterVolumeSpecName: "kube-api-access-mcr8w") pod "47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c" (UID: "47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c"). InnerVolumeSpecName "kube-api-access-mcr8w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 01:50:59.045748 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:59.045724 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c" (UID: "47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 15 01:50:59.144596 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:59.144564 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c-trusted-ca-bundle\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:50:59.144596 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:59.144592 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c-console-config\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:50:59.144596 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:59.144602 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mcr8w\" (UniqueName: \"kubernetes.io/projected/47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c-kube-api-access-mcr8w\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:50:59.144845 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:59.144611 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c-service-ca\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:50:59.144845 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:59.144620 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c-console-oauth-config\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:50:59.144845 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:59.144629 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c-console-serving-cert\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:50:59.144845 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:59.144639 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c-oauth-serving-cert\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:50:59.157944 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:59.157910 2578 generic.go:356] "Generic (PLEG): container finished" podID="47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c" containerID="412fef52ba8aee61cbc8897f328bbe18525e8b5f3009f4e81b57d05f3b3da554" exitCode=0 Jun 15 01:50:59.158127 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:59.157965 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-654cbf94d4-nphzt" Jun 15 01:50:59.158127 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:59.158001 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-654cbf94d4-nphzt" event={"ID":"47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c","Type":"ContainerDied","Data":"412fef52ba8aee61cbc8897f328bbe18525e8b5f3009f4e81b57d05f3b3da554"} Jun 15 01:50:59.158127 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:59.158049 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-654cbf94d4-nphzt" event={"ID":"47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c","Type":"ContainerDied","Data":"0ff9815eb4edcff61663cc9313be758c11928975f8e189d5509f851e480e5742"} Jun 15 01:50:59.158127 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:59.158069 2578 scope.go:117] "RemoveContainer" containerID="412fef52ba8aee61cbc8897f328bbe18525e8b5f3009f4e81b57d05f3b3da554" Jun 15 01:50:59.167004 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:59.166984 2578 scope.go:117] "RemoveContainer" containerID="412fef52ba8aee61cbc8897f328bbe18525e8b5f3009f4e81b57d05f3b3da554" Jun 15 01:50:59.167293 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:50:59.167273 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"412fef52ba8aee61cbc8897f328bbe18525e8b5f3009f4e81b57d05f3b3da554\": container with ID starting with 412fef52ba8aee61cbc8897f328bbe18525e8b5f3009f4e81b57d05f3b3da554 not found: ID does not exist" containerID="412fef52ba8aee61cbc8897f328bbe18525e8b5f3009f4e81b57d05f3b3da554" Jun 15 01:50:59.167366 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:59.167307 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"412fef52ba8aee61cbc8897f328bbe18525e8b5f3009f4e81b57d05f3b3da554"} err="failed to get container status \"412fef52ba8aee61cbc8897f328bbe18525e8b5f3009f4e81b57d05f3b3da554\": rpc error: code = NotFound desc = could not find container \"412fef52ba8aee61cbc8897f328bbe18525e8b5f3009f4e81b57d05f3b3da554\": container with ID starting with 412fef52ba8aee61cbc8897f328bbe18525e8b5f3009f4e81b57d05f3b3da554 not found: ID does not exist" Jun 15 01:50:59.178040 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:59.178012 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-654cbf94d4-nphzt"] Jun 15 01:50:59.186002 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:50:59.185974 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-654cbf94d4-nphzt"] Jun 15 01:51:00.765045 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:00.765009 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c" path="/var/lib/kubelet/pods/47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c/volumes" Jun 15 01:51:07.696215 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:07.696179 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqfspr"] Jun 15 01:51:07.696663 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:07.696474 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26023146-86ca-4a77-b73c-429dd24c4e5d" containerName="registry" Jun 15 01:51:07.696663 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:07.696484 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="26023146-86ca-4a77-b73c-429dd24c4e5d" containerName="registry" Jun 15 01:51:07.696663 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:07.696496 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ba66708-00f3-48e6-bc68-c087405f1b3f" containerName="console" Jun 15 01:51:07.696663 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:07.696501 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ba66708-00f3-48e6-bc68-c087405f1b3f" containerName="console" Jun 15 01:51:07.696663 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:07.696513 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c" containerName="console" Jun 15 01:51:07.696663 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:07.696519 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c" containerName="console" Jun 15 01:51:07.696663 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:07.696527 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2feade0e-acaf-4b90-8998-0df4be7c3c14" containerName="console" Jun 15 01:51:07.696663 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:07.696532 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2feade0e-acaf-4b90-8998-0df4be7c3c14" containerName="console" Jun 15 01:51:07.696663 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:07.696574 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="2feade0e-acaf-4b90-8998-0df4be7c3c14" containerName="console" Jun 15 01:51:07.696663 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:07.696580 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="47ce7ef0-7ae2-48b8-b4b9-a5c06b1ab94c" containerName="console" Jun 15 01:51:07.696663 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:07.696586 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ba66708-00f3-48e6-bc68-c087405f1b3f" containerName="console" Jun 15 01:51:07.696663 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:07.696591 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="26023146-86ca-4a77-b73c-429dd24c4e5d" containerName="registry" Jun 15 01:51:07.701014 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:07.700996 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqfspr" Jun 15 01:51:07.704798 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:07.704767 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 15 01:51:07.705075 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:07.705057 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ncvbf\"" Jun 15 01:51:07.706204 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:07.706173 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 15 01:51:07.722872 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:07.722846 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqfspr"] Jun 15 01:51:07.806883 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:07.806853 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2bef6af9-1d9e-4457-8121-1345fd5d6277-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqfspr\" (UID: \"2bef6af9-1d9e-4457-8121-1345fd5d6277\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqfspr" Jun 15 01:51:07.807053 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:07.806894 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5vnw\" (UniqueName: \"kubernetes.io/projected/2bef6af9-1d9e-4457-8121-1345fd5d6277-kube-api-access-c5vnw\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqfspr\" (UID: \"2bef6af9-1d9e-4457-8121-1345fd5d6277\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqfspr" Jun 15 01:51:07.807053 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:07.806950 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2bef6af9-1d9e-4457-8121-1345fd5d6277-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqfspr\" (UID: \"2bef6af9-1d9e-4457-8121-1345fd5d6277\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqfspr" Jun 15 01:51:07.907897 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:07.907853 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2bef6af9-1d9e-4457-8121-1345fd5d6277-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqfspr\" (UID: \"2bef6af9-1d9e-4457-8121-1345fd5d6277\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqfspr" Jun 15 01:51:07.908128 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:07.907933 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2bef6af9-1d9e-4457-8121-1345fd5d6277-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqfspr\" (UID: \"2bef6af9-1d9e-4457-8121-1345fd5d6277\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqfspr" Jun 15 01:51:07.908128 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:07.907970 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5vnw\" (UniqueName: \"kubernetes.io/projected/2bef6af9-1d9e-4457-8121-1345fd5d6277-kube-api-access-c5vnw\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqfspr\" (UID: \"2bef6af9-1d9e-4457-8121-1345fd5d6277\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqfspr" Jun 15 01:51:07.908284 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:07.908261 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2bef6af9-1d9e-4457-8121-1345fd5d6277-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqfspr\" (UID: \"2bef6af9-1d9e-4457-8121-1345fd5d6277\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqfspr" Jun 15 01:51:07.908335 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:07.908294 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2bef6af9-1d9e-4457-8121-1345fd5d6277-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqfspr\" (UID: \"2bef6af9-1d9e-4457-8121-1345fd5d6277\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqfspr" Jun 15 01:51:07.916724 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:07.916695 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5vnw\" (UniqueName: \"kubernetes.io/projected/2bef6af9-1d9e-4457-8121-1345fd5d6277-kube-api-access-c5vnw\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqfspr\" (UID: \"2bef6af9-1d9e-4457-8121-1345fd5d6277\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqfspr" Jun 15 01:51:08.011098 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:08.011011 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqfspr" Jun 15 01:51:08.145066 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:08.145041 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqfspr"] Jun 15 01:51:08.147349 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:51:08.147319 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bef6af9_1d9e_4457_8121_1345fd5d6277.slice/crio-e8bb3eeeca463873ef4899ce1ec37c8ee9b4d3b7fb5ce153fe7f0837ae3e45e4 WatchSource:0}: Error finding container e8bb3eeeca463873ef4899ce1ec37c8ee9b4d3b7fb5ce153fe7f0837ae3e45e4: Status 404 returned error can't find the container with id e8bb3eeeca463873ef4899ce1ec37c8ee9b4d3b7fb5ce153fe7f0837ae3e45e4 Jun 15 01:51:08.149188 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:08.149173 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 15 01:51:08.192949 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:08.192919 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqfspr" event={"ID":"2bef6af9-1d9e-4457-8121-1345fd5d6277","Type":"ContainerStarted","Data":"e8bb3eeeca463873ef4899ce1ec37c8ee9b4d3b7fb5ce153fe7f0837ae3e45e4"} Jun 15 01:51:15.224373 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:15.224336 2578 generic.go:356] "Generic (PLEG): container finished" podID="2bef6af9-1d9e-4457-8121-1345fd5d6277" containerID="bdc0ba3fc044e0f6a83f1f28fd3bafbc7dbb1948c6560d5b141e44f48779d659" exitCode=0 Jun 15 01:51:15.224769 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:15.224418 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqfspr" event={"ID":"2bef6af9-1d9e-4457-8121-1345fd5d6277","Type":"ContainerDied","Data":"bdc0ba3fc044e0f6a83f1f28fd3bafbc7dbb1948c6560d5b141e44f48779d659"} Jun 15 01:51:18.238474 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:18.238441 2578 generic.go:356] "Generic (PLEG): container finished" podID="2bef6af9-1d9e-4457-8121-1345fd5d6277" containerID="f647eb441d45a41dad262a8c8dde6d0d4303d67d07ed6eea04b5c7676319df33" exitCode=0 Jun 15 01:51:18.238876 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:18.238509 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqfspr" event={"ID":"2bef6af9-1d9e-4457-8121-1345fd5d6277","Type":"ContainerDied","Data":"f647eb441d45a41dad262a8c8dde6d0d4303d67d07ed6eea04b5c7676319df33"} Jun 15 01:51:26.277167 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:26.277123 2578 generic.go:356] "Generic (PLEG): container finished" podID="2bef6af9-1d9e-4457-8121-1345fd5d6277" containerID="90218b71ac6211b495e691e256719dfada443bee452f88807a692822be521a3b" exitCode=0 Jun 15 01:51:26.277567 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:26.277210 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqfspr" event={"ID":"2bef6af9-1d9e-4457-8121-1345fd5d6277","Type":"ContainerDied","Data":"90218b71ac6211b495e691e256719dfada443bee452f88807a692822be521a3b"} Jun 15 01:51:27.406479 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:27.406449 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqfspr" Jun 15 01:51:27.468040 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:27.468009 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c5vnw\" (UniqueName: \"kubernetes.io/projected/2bef6af9-1d9e-4457-8121-1345fd5d6277-kube-api-access-c5vnw\") pod \"2bef6af9-1d9e-4457-8121-1345fd5d6277\" (UID: \"2bef6af9-1d9e-4457-8121-1345fd5d6277\") " Jun 15 01:51:27.468040 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:27.468048 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2bef6af9-1d9e-4457-8121-1345fd5d6277-util\") pod \"2bef6af9-1d9e-4457-8121-1345fd5d6277\" (UID: \"2bef6af9-1d9e-4457-8121-1345fd5d6277\") " Jun 15 01:51:27.468314 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:27.468123 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2bef6af9-1d9e-4457-8121-1345fd5d6277-bundle\") pod \"2bef6af9-1d9e-4457-8121-1345fd5d6277\" (UID: \"2bef6af9-1d9e-4457-8121-1345fd5d6277\") " Jun 15 01:51:27.468825 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:27.468795 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bef6af9-1d9e-4457-8121-1345fd5d6277-bundle" (OuterVolumeSpecName: "bundle") pod "2bef6af9-1d9e-4457-8121-1345fd5d6277" (UID: "2bef6af9-1d9e-4457-8121-1345fd5d6277"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 01:51:27.470411 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:27.470391 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2bef6af9-1d9e-4457-8121-1345fd5d6277-kube-api-access-c5vnw" (OuterVolumeSpecName: "kube-api-access-c5vnw") pod "2bef6af9-1d9e-4457-8121-1345fd5d6277" (UID: "2bef6af9-1d9e-4457-8121-1345fd5d6277"). InnerVolumeSpecName "kube-api-access-c5vnw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 01:51:27.473352 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:27.473324 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bef6af9-1d9e-4457-8121-1345fd5d6277-util" (OuterVolumeSpecName: "util") pod "2bef6af9-1d9e-4457-8121-1345fd5d6277" (UID: "2bef6af9-1d9e-4457-8121-1345fd5d6277"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 01:51:27.568909 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:27.568822 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c5vnw\" (UniqueName: \"kubernetes.io/projected/2bef6af9-1d9e-4457-8121-1345fd5d6277-kube-api-access-c5vnw\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:51:27.568909 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:27.568853 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2bef6af9-1d9e-4457-8121-1345fd5d6277-util\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:51:27.568909 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:27.568862 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2bef6af9-1d9e-4457-8121-1345fd5d6277-bundle\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:51:28.286285 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:28.286245 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqfspr" event={"ID":"2bef6af9-1d9e-4457-8121-1345fd5d6277","Type":"ContainerDied","Data":"e8bb3eeeca463873ef4899ce1ec37c8ee9b4d3b7fb5ce153fe7f0837ae3e45e4"} Jun 15 01:51:28.286285 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:28.286282 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dqfspr" Jun 15 01:51:28.286285 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:28.286293 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e8bb3eeeca463873ef4899ce1ec37c8ee9b4d3b7fb5ce153fe7f0837ae3e45e4" Jun 15 01:51:36.930286 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:36.930241 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-f7tdr"] Jun 15 01:51:36.930697 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:36.930558 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bef6af9-1d9e-4457-8121-1345fd5d6277" containerName="pull" Jun 15 01:51:36.930697 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:36.930570 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bef6af9-1d9e-4457-8121-1345fd5d6277" containerName="pull" Jun 15 01:51:36.930697 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:36.930581 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bef6af9-1d9e-4457-8121-1345fd5d6277" containerName="util" Jun 15 01:51:36.930697 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:36.930590 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bef6af9-1d9e-4457-8121-1345fd5d6277" containerName="util" Jun 15 01:51:36.930697 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:36.930595 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bef6af9-1d9e-4457-8121-1345fd5d6277" containerName="extract" Jun 15 01:51:36.930697 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:36.930601 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bef6af9-1d9e-4457-8121-1345fd5d6277" containerName="extract" Jun 15 01:51:36.930697 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:36.930661 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bef6af9-1d9e-4457-8121-1345fd5d6277" containerName="extract" Jun 15 01:51:36.934945 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:36.934927 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-f7tdr" Jun 15 01:51:36.937814 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:36.937780 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-88qbz\"" Jun 15 01:51:36.937814 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:36.937803 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jun 15 01:51:36.937978 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:36.937822 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jun 15 01:51:36.943947 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:36.943922 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-f7tdr"] Jun 15 01:51:37.040022 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:37.039937 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f8acae3-a7be-4a95-a9c9-8b32a44112cf-tmp\") pod \"cert-manager-operator-controller-manager-57757f7cfc-f7tdr\" (UID: \"8f8acae3-a7be-4a95-a9c9-8b32a44112cf\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-f7tdr" Jun 15 01:51:37.040022 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:37.040002 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6w9cz\" (UniqueName: \"kubernetes.io/projected/8f8acae3-a7be-4a95-a9c9-8b32a44112cf-kube-api-access-6w9cz\") pod \"cert-manager-operator-controller-manager-57757f7cfc-f7tdr\" (UID: \"8f8acae3-a7be-4a95-a9c9-8b32a44112cf\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-f7tdr" Jun 15 01:51:37.140792 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:37.140748 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f8acae3-a7be-4a95-a9c9-8b32a44112cf-tmp\") pod \"cert-manager-operator-controller-manager-57757f7cfc-f7tdr\" (UID: \"8f8acae3-a7be-4a95-a9c9-8b32a44112cf\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-f7tdr" Jun 15 01:51:37.140969 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:37.140838 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6w9cz\" (UniqueName: \"kubernetes.io/projected/8f8acae3-a7be-4a95-a9c9-8b32a44112cf-kube-api-access-6w9cz\") pod \"cert-manager-operator-controller-manager-57757f7cfc-f7tdr\" (UID: \"8f8acae3-a7be-4a95-a9c9-8b32a44112cf\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-f7tdr" Jun 15 01:51:37.141196 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:37.141175 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f8acae3-a7be-4a95-a9c9-8b32a44112cf-tmp\") pod \"cert-manager-operator-controller-manager-57757f7cfc-f7tdr\" (UID: \"8f8acae3-a7be-4a95-a9c9-8b32a44112cf\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-f7tdr" Jun 15 01:51:37.149328 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:37.149303 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6w9cz\" (UniqueName: \"kubernetes.io/projected/8f8acae3-a7be-4a95-a9c9-8b32a44112cf-kube-api-access-6w9cz\") pod \"cert-manager-operator-controller-manager-57757f7cfc-f7tdr\" (UID: \"8f8acae3-a7be-4a95-a9c9-8b32a44112cf\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-f7tdr" Jun 15 01:51:37.244808 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:37.244761 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-f7tdr" Jun 15 01:51:37.386860 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:37.386833 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-f7tdr"] Jun 15 01:51:37.389813 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:51:37.389786 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f8acae3_a7be_4a95_a9c9_8b32a44112cf.slice/crio-d5321fe16ed3d52c8720b11f21d8e11787bb7aa2bd1e971fbff03a2b3d26f34b WatchSource:0}: Error finding container d5321fe16ed3d52c8720b11f21d8e11787bb7aa2bd1e971fbff03a2b3d26f34b: Status 404 returned error can't find the container with id d5321fe16ed3d52c8720b11f21d8e11787bb7aa2bd1e971fbff03a2b3d26f34b Jun 15 01:51:38.326825 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:38.326785 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-f7tdr" event={"ID":"8f8acae3-a7be-4a95-a9c9-8b32a44112cf","Type":"ContainerStarted","Data":"d5321fe16ed3d52c8720b11f21d8e11787bb7aa2bd1e971fbff03a2b3d26f34b"} Jun 15 01:51:39.333721 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:39.333674 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-f7tdr" event={"ID":"8f8acae3-a7be-4a95-a9c9-8b32a44112cf","Type":"ContainerStarted","Data":"9369eba05c08c437c2ca2ff9e050ef48d24031006489017ab823d5f59845f625"} Jun 15 01:51:39.356518 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:39.356452 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-f7tdr" podStartSLOduration=1.487941389 podStartE2EDuration="3.356431565s" totalImagesPullingTime="1.868490176s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 01:51:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 01:51:39.356225778 +0000 UTC m=+423.069951750" watchObservedRunningTime="2026-06-15 01:51:39.356431565 +0000 UTC m=+423.070157538" Jun 15 01:51:42.652326 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:42.652294 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-7dc76cbb94-qrlfx"] Jun 15 01:51:42.654428 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:42.654413 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-7dc76cbb94-qrlfx" Jun 15 01:51:42.656985 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:42.656963 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jun 15 01:51:42.657131 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:42.656983 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jun 15 01:51:42.658203 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:42.658186 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-5gt4c\"" Jun 15 01:51:42.663135 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:42.663113 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-7dc76cbb94-qrlfx"] Jun 15 01:51:42.788957 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:42.788928 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4744657c-3975-4565-97dd-64a4c7468492-bound-sa-token\") pod \"cert-manager-webhook-7dc76cbb94-qrlfx\" (UID: \"4744657c-3975-4565-97dd-64a4c7468492\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-qrlfx" Jun 15 01:51:42.789127 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:42.788991 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r975l\" (UniqueName: \"kubernetes.io/projected/4744657c-3975-4565-97dd-64a4c7468492-kube-api-access-r975l\") pod \"cert-manager-webhook-7dc76cbb94-qrlfx\" (UID: \"4744657c-3975-4565-97dd-64a4c7468492\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-qrlfx" Jun 15 01:51:42.890344 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:42.890306 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r975l\" (UniqueName: \"kubernetes.io/projected/4744657c-3975-4565-97dd-64a4c7468492-kube-api-access-r975l\") pod \"cert-manager-webhook-7dc76cbb94-qrlfx\" (UID: \"4744657c-3975-4565-97dd-64a4c7468492\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-qrlfx" Jun 15 01:51:42.890492 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:42.890386 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4744657c-3975-4565-97dd-64a4c7468492-bound-sa-token\") pod \"cert-manager-webhook-7dc76cbb94-qrlfx\" (UID: \"4744657c-3975-4565-97dd-64a4c7468492\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-qrlfx" Jun 15 01:51:42.899500 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:42.899469 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4744657c-3975-4565-97dd-64a4c7468492-bound-sa-token\") pod \"cert-manager-webhook-7dc76cbb94-qrlfx\" (UID: \"4744657c-3975-4565-97dd-64a4c7468492\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-qrlfx" Jun 15 01:51:42.899626 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:42.899574 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r975l\" (UniqueName: \"kubernetes.io/projected/4744657c-3975-4565-97dd-64a4c7468492-kube-api-access-r975l\") pod \"cert-manager-webhook-7dc76cbb94-qrlfx\" (UID: \"4744657c-3975-4565-97dd-64a4c7468492\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-qrlfx" Jun 15 01:51:42.972709 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:42.972606 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-7dc76cbb94-qrlfx" Jun 15 01:51:43.098231 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:43.098058 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-7dc76cbb94-qrlfx"] Jun 15 01:51:43.100283 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:51:43.100257 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4744657c_3975_4565_97dd_64a4c7468492.slice/crio-28d7ea5638b14be8082cf5ebff96274c3cbd7220c0c5d97b5fe7e470ffd56a6f WatchSource:0}: Error finding container 28d7ea5638b14be8082cf5ebff96274c3cbd7220c0c5d97b5fe7e470ffd56a6f: Status 404 returned error can't find the container with id 28d7ea5638b14be8082cf5ebff96274c3cbd7220c0c5d97b5fe7e470ffd56a6f Jun 15 01:51:43.349906 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:43.349815 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-7dc76cbb94-qrlfx" event={"ID":"4744657c-3975-4565-97dd-64a4c7468492","Type":"ContainerStarted","Data":"28d7ea5638b14be8082cf5ebff96274c3cbd7220c0c5d97b5fe7e470ffd56a6f"} Jun 15 01:51:43.439334 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:43.439300 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhbbzp"] Jun 15 01:51:43.442024 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:43.442004 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhbbzp" Jun 15 01:51:43.444663 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:43.444644 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 15 01:51:43.445738 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:43.445721 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ncvbf\"" Jun 15 01:51:43.445823 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:43.445742 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 15 01:51:43.449847 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:43.449828 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhbbzp"] Jun 15 01:51:43.595856 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:43.595803 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d4f3aad2-5590-4fc5-bb0e-b628ec826a97-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhbbzp\" (UID: \"d4f3aad2-5590-4fc5-bb0e-b628ec826a97\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhbbzp" Jun 15 01:51:43.595856 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:43.595863 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fzd7\" (UniqueName: \"kubernetes.io/projected/d4f3aad2-5590-4fc5-bb0e-b628ec826a97-kube-api-access-6fzd7\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhbbzp\" (UID: \"d4f3aad2-5590-4fc5-bb0e-b628ec826a97\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhbbzp" Jun 15 01:51:43.596075 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:43.595942 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d4f3aad2-5590-4fc5-bb0e-b628ec826a97-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhbbzp\" (UID: \"d4f3aad2-5590-4fc5-bb0e-b628ec826a97\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhbbzp" Jun 15 01:51:43.616146 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:43.613921 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-5d99449ffd-vbv8h"] Jun 15 01:51:43.617328 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:43.617301 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5d99449ffd-vbv8h" Jun 15 01:51:43.620010 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:43.619987 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-llrs7\"" Jun 15 01:51:43.625220 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:43.625198 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5d99449ffd-vbv8h"] Jun 15 01:51:43.697026 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:43.696984 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fzd7\" (UniqueName: \"kubernetes.io/projected/d4f3aad2-5590-4fc5-bb0e-b628ec826a97-kube-api-access-6fzd7\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhbbzp\" (UID: \"d4f3aad2-5590-4fc5-bb0e-b628ec826a97\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhbbzp" Jun 15 01:51:43.697470 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:43.697035 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d4f3aad2-5590-4fc5-bb0e-b628ec826a97-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhbbzp\" (UID: \"d4f3aad2-5590-4fc5-bb0e-b628ec826a97\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhbbzp" Jun 15 01:51:43.697470 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:43.697169 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/436aca7a-adc0-43fd-a0a3-5ec8b9e86f0a-bound-sa-token\") pod \"cert-manager-cainjector-5d99449ffd-vbv8h\" (UID: \"436aca7a-adc0-43fd-a0a3-5ec8b9e86f0a\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-vbv8h" Jun 15 01:51:43.697470 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:43.697274 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tckt2\" (UniqueName: \"kubernetes.io/projected/436aca7a-adc0-43fd-a0a3-5ec8b9e86f0a-kube-api-access-tckt2\") pod \"cert-manager-cainjector-5d99449ffd-vbv8h\" (UID: \"436aca7a-adc0-43fd-a0a3-5ec8b9e86f0a\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-vbv8h" Jun 15 01:51:43.697470 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:43.697361 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d4f3aad2-5590-4fc5-bb0e-b628ec826a97-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhbbzp\" (UID: \"d4f3aad2-5590-4fc5-bb0e-b628ec826a97\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhbbzp" Jun 15 01:51:43.697470 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:43.697362 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d4f3aad2-5590-4fc5-bb0e-b628ec826a97-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhbbzp\" (UID: \"d4f3aad2-5590-4fc5-bb0e-b628ec826a97\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhbbzp" Jun 15 01:51:43.697643 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:43.697593 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d4f3aad2-5590-4fc5-bb0e-b628ec826a97-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhbbzp\" (UID: \"d4f3aad2-5590-4fc5-bb0e-b628ec826a97\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhbbzp" Jun 15 01:51:43.705821 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:43.705796 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fzd7\" (UniqueName: \"kubernetes.io/projected/d4f3aad2-5590-4fc5-bb0e-b628ec826a97-kube-api-access-6fzd7\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhbbzp\" (UID: \"d4f3aad2-5590-4fc5-bb0e-b628ec826a97\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhbbzp" Jun 15 01:51:43.752229 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:43.752200 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhbbzp" Jun 15 01:51:43.798515 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:43.798479 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/436aca7a-adc0-43fd-a0a3-5ec8b9e86f0a-bound-sa-token\") pod \"cert-manager-cainjector-5d99449ffd-vbv8h\" (UID: \"436aca7a-adc0-43fd-a0a3-5ec8b9e86f0a\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-vbv8h" Jun 15 01:51:43.798925 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:43.798564 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tckt2\" (UniqueName: \"kubernetes.io/projected/436aca7a-adc0-43fd-a0a3-5ec8b9e86f0a-kube-api-access-tckt2\") pod \"cert-manager-cainjector-5d99449ffd-vbv8h\" (UID: \"436aca7a-adc0-43fd-a0a3-5ec8b9e86f0a\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-vbv8h" Jun 15 01:51:43.811198 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:43.810883 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tckt2\" (UniqueName: \"kubernetes.io/projected/436aca7a-adc0-43fd-a0a3-5ec8b9e86f0a-kube-api-access-tckt2\") pod \"cert-manager-cainjector-5d99449ffd-vbv8h\" (UID: \"436aca7a-adc0-43fd-a0a3-5ec8b9e86f0a\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-vbv8h" Jun 15 01:51:43.815324 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:43.815297 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/436aca7a-adc0-43fd-a0a3-5ec8b9e86f0a-bound-sa-token\") pod \"cert-manager-cainjector-5d99449ffd-vbv8h\" (UID: \"436aca7a-adc0-43fd-a0a3-5ec8b9e86f0a\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-vbv8h" Jun 15 01:51:43.906570 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:43.906538 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhbbzp"] Jun 15 01:51:43.911539 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:51:43.911480 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4f3aad2_5590_4fc5_bb0e_b628ec826a97.slice/crio-df7f26953e55701f18b26a3a08143b711e452de64332d0e3073b4012a1186ed1 WatchSource:0}: Error finding container df7f26953e55701f18b26a3a08143b711e452de64332d0e3073b4012a1186ed1: Status 404 returned error can't find the container with id df7f26953e55701f18b26a3a08143b711e452de64332d0e3073b4012a1186ed1 Jun 15 01:51:43.926496 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:43.926472 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5d99449ffd-vbv8h" Jun 15 01:51:44.077955 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:44.077923 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5d99449ffd-vbv8h"] Jun 15 01:51:44.081058 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:51:44.081027 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod436aca7a_adc0_43fd_a0a3_5ec8b9e86f0a.slice/crio-5259c59037129fb67ecc9e2602480cabab41ce04c013ad0a686626c3b62895f3 WatchSource:0}: Error finding container 5259c59037129fb67ecc9e2602480cabab41ce04c013ad0a686626c3b62895f3: Status 404 returned error can't find the container with id 5259c59037129fb67ecc9e2602480cabab41ce04c013ad0a686626c3b62895f3 Jun 15 01:51:44.356313 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:44.356218 2578 generic.go:356] "Generic (PLEG): container finished" podID="d4f3aad2-5590-4fc5-bb0e-b628ec826a97" containerID="1cbdd892932683622f3989cdc53800ef17ccc76f2d67c851ff737cf43978adae" exitCode=0 Jun 15 01:51:44.356481 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:44.356306 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhbbzp" event={"ID":"d4f3aad2-5590-4fc5-bb0e-b628ec826a97","Type":"ContainerDied","Data":"1cbdd892932683622f3989cdc53800ef17ccc76f2d67c851ff737cf43978adae"} Jun 15 01:51:44.356481 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:44.356367 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhbbzp" event={"ID":"d4f3aad2-5590-4fc5-bb0e-b628ec826a97","Type":"ContainerStarted","Data":"df7f26953e55701f18b26a3a08143b711e452de64332d0e3073b4012a1186ed1"} Jun 15 01:51:44.357654 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:44.357622 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5d99449ffd-vbv8h" event={"ID":"436aca7a-adc0-43fd-a0a3-5ec8b9e86f0a","Type":"ContainerStarted","Data":"5259c59037129fb67ecc9e2602480cabab41ce04c013ad0a686626c3b62895f3"} Jun 15 01:51:46.371851 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:46.371801 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5d99449ffd-vbv8h" event={"ID":"436aca7a-adc0-43fd-a0a3-5ec8b9e86f0a","Type":"ContainerStarted","Data":"fc8a1cf5a62e41a64e2437f4c110a0eae624152edac9a94c99636c771dbd0f07"} Jun 15 01:51:46.373705 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:46.373675 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-7dc76cbb94-qrlfx" event={"ID":"4744657c-3975-4565-97dd-64a4c7468492","Type":"ContainerStarted","Data":"03565da8886d3ba150ecc65f1817f18c85c275749f7a3c61dd0061948b347399"} Jun 15 01:51:46.373858 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:46.373783 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-7dc76cbb94-qrlfx" Jun 15 01:51:46.390147 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:46.390022 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-5d99449ffd-vbv8h" podStartSLOduration=1.24329925 podStartE2EDuration="3.390008126s" totalImagesPullingTime="2.146708876s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 01:51:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 01:51:46.389042276 +0000 UTC m=+430.102768249" watchObservedRunningTime="2026-06-15 01:51:46.390008126 +0000 UTC m=+430.103734097" Jun 15 01:51:46.407559 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:46.406446 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-7dc76cbb94-qrlfx" podStartSLOduration=1.666713105 podStartE2EDuration="4.406422024s" totalImagesPullingTime="2.739708919s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 01:51:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 01:51:46.403941381 +0000 UTC m=+430.117667354" watchObservedRunningTime="2026-06-15 01:51:46.406422024 +0000 UTC m=+430.120147997" Jun 15 01:51:48.385489 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:48.385448 2578 generic.go:356] "Generic (PLEG): container finished" podID="d4f3aad2-5590-4fc5-bb0e-b628ec826a97" containerID="8cabf1f217cae238d0aa92e0f42a85ddb4238cd5153a0657526df819fb22d396" exitCode=0 Jun 15 01:51:48.385917 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:48.385538 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhbbzp" event={"ID":"d4f3aad2-5590-4fc5-bb0e-b628ec826a97","Type":"ContainerDied","Data":"8cabf1f217cae238d0aa92e0f42a85ddb4238cd5153a0657526df819fb22d396"} Jun 15 01:51:49.392054 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:49.392016 2578 generic.go:356] "Generic (PLEG): container finished" podID="d4f3aad2-5590-4fc5-bb0e-b628ec826a97" containerID="1a805935746ae52b0cb118f5ae03b52413d37c00edf36aa66244819dc383258a" exitCode=0 Jun 15 01:51:49.392453 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:49.392125 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhbbzp" event={"ID":"d4f3aad2-5590-4fc5-bb0e-b628ec826a97","Type":"ContainerDied","Data":"1a805935746ae52b0cb118f5ae03b52413d37c00edf36aa66244819dc383258a"} Jun 15 01:51:50.523077 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:50.523050 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhbbzp" Jun 15 01:51:50.657937 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:50.657848 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6fzd7\" (UniqueName: \"kubernetes.io/projected/d4f3aad2-5590-4fc5-bb0e-b628ec826a97-kube-api-access-6fzd7\") pod \"d4f3aad2-5590-4fc5-bb0e-b628ec826a97\" (UID: \"d4f3aad2-5590-4fc5-bb0e-b628ec826a97\") " Jun 15 01:51:50.657937 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:50.657928 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d4f3aad2-5590-4fc5-bb0e-b628ec826a97-util\") pod \"d4f3aad2-5590-4fc5-bb0e-b628ec826a97\" (UID: \"d4f3aad2-5590-4fc5-bb0e-b628ec826a97\") " Jun 15 01:51:50.658216 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:50.657986 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d4f3aad2-5590-4fc5-bb0e-b628ec826a97-bundle\") pod \"d4f3aad2-5590-4fc5-bb0e-b628ec826a97\" (UID: \"d4f3aad2-5590-4fc5-bb0e-b628ec826a97\") " Jun 15 01:51:50.658398 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:50.658372 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4f3aad2-5590-4fc5-bb0e-b628ec826a97-bundle" (OuterVolumeSpecName: "bundle") pod "d4f3aad2-5590-4fc5-bb0e-b628ec826a97" (UID: "d4f3aad2-5590-4fc5-bb0e-b628ec826a97"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 01:51:50.660146 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:50.660120 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4f3aad2-5590-4fc5-bb0e-b628ec826a97-kube-api-access-6fzd7" (OuterVolumeSpecName: "kube-api-access-6fzd7") pod "d4f3aad2-5590-4fc5-bb0e-b628ec826a97" (UID: "d4f3aad2-5590-4fc5-bb0e-b628ec826a97"). InnerVolumeSpecName "kube-api-access-6fzd7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 01:51:50.662967 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:50.662937 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4f3aad2-5590-4fc5-bb0e-b628ec826a97-util" (OuterVolumeSpecName: "util") pod "d4f3aad2-5590-4fc5-bb0e-b628ec826a97" (UID: "d4f3aad2-5590-4fc5-bb0e-b628ec826a97"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 01:51:50.759248 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:50.759217 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d4f3aad2-5590-4fc5-bb0e-b628ec826a97-bundle\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:51:50.759248 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:50.759245 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6fzd7\" (UniqueName: \"kubernetes.io/projected/d4f3aad2-5590-4fc5-bb0e-b628ec826a97-kube-api-access-6fzd7\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:51:50.759248 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:50.759254 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d4f3aad2-5590-4fc5-bb0e-b628ec826a97-util\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:51:51.403141 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:51.403108 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhbbzp" event={"ID":"d4f3aad2-5590-4fc5-bb0e-b628ec826a97","Type":"ContainerDied","Data":"df7f26953e55701f18b26a3a08143b711e452de64332d0e3073b4012a1186ed1"} Jun 15 01:51:51.403341 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:51.403147 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhbbzp" Jun 15 01:51:51.403341 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:51.403155 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="df7f26953e55701f18b26a3a08143b711e452de64332d0e3073b4012a1186ed1" Jun 15 01:51:52.382739 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:51:52.382708 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-7dc76cbb94-qrlfx" Jun 15 01:52:08.319240 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:08.319205 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd9r5fz"] Jun 15 01:52:08.319705 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:08.319496 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4f3aad2-5590-4fc5-bb0e-b628ec826a97" containerName="pull" Jun 15 01:52:08.319705 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:08.319506 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4f3aad2-5590-4fc5-bb0e-b628ec826a97" containerName="pull" Jun 15 01:52:08.319705 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:08.319512 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4f3aad2-5590-4fc5-bb0e-b628ec826a97" containerName="extract" Jun 15 01:52:08.319705 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:08.319517 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4f3aad2-5590-4fc5-bb0e-b628ec826a97" containerName="extract" Jun 15 01:52:08.319705 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:08.319527 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4f3aad2-5590-4fc5-bb0e-b628ec826a97" containerName="util" Jun 15 01:52:08.319705 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:08.319533 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4f3aad2-5590-4fc5-bb0e-b628ec826a97" containerName="util" Jun 15 01:52:08.319705 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:08.319583 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4f3aad2-5590-4fc5-bb0e-b628ec826a97" containerName="extract" Jun 15 01:52:08.322737 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:08.322716 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd9r5fz" Jun 15 01:52:08.325271 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:08.325247 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 15 01:52:08.325384 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:08.325293 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 15 01:52:08.326353 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:08.326331 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ncvbf\"" Jun 15 01:52:08.330482 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:08.330455 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd9r5fz"] Jun 15 01:52:08.400560 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:08.400525 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2nm4\" (UniqueName: \"kubernetes.io/projected/6fa10e87-9d8f-4e73-ad78-6d167189c0df-kube-api-access-j2nm4\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd9r5fz\" (UID: \"6fa10e87-9d8f-4e73-ad78-6d167189c0df\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd9r5fz" Jun 15 01:52:08.400560 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:08.400568 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6fa10e87-9d8f-4e73-ad78-6d167189c0df-util\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd9r5fz\" (UID: \"6fa10e87-9d8f-4e73-ad78-6d167189c0df\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd9r5fz" Jun 15 01:52:08.400796 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:08.400652 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6fa10e87-9d8f-4e73-ad78-6d167189c0df-bundle\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd9r5fz\" (UID: \"6fa10e87-9d8f-4e73-ad78-6d167189c0df\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd9r5fz" Jun 15 01:52:08.501211 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:08.501174 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6fa10e87-9d8f-4e73-ad78-6d167189c0df-bundle\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd9r5fz\" (UID: \"6fa10e87-9d8f-4e73-ad78-6d167189c0df\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd9r5fz" Jun 15 01:52:08.501414 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:08.501224 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2nm4\" (UniqueName: \"kubernetes.io/projected/6fa10e87-9d8f-4e73-ad78-6d167189c0df-kube-api-access-j2nm4\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd9r5fz\" (UID: \"6fa10e87-9d8f-4e73-ad78-6d167189c0df\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd9r5fz" Jun 15 01:52:08.501414 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:08.501257 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6fa10e87-9d8f-4e73-ad78-6d167189c0df-util\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd9r5fz\" (UID: \"6fa10e87-9d8f-4e73-ad78-6d167189c0df\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd9r5fz" Jun 15 01:52:08.501650 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:08.501628 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6fa10e87-9d8f-4e73-ad78-6d167189c0df-bundle\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd9r5fz\" (UID: \"6fa10e87-9d8f-4e73-ad78-6d167189c0df\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd9r5fz" Jun 15 01:52:08.501731 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:08.501669 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6fa10e87-9d8f-4e73-ad78-6d167189c0df-util\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd9r5fz\" (UID: \"6fa10e87-9d8f-4e73-ad78-6d167189c0df\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd9r5fz" Jun 15 01:52:08.509747 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:08.509712 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2nm4\" (UniqueName: \"kubernetes.io/projected/6fa10e87-9d8f-4e73-ad78-6d167189c0df-kube-api-access-j2nm4\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd9r5fz\" (UID: \"6fa10e87-9d8f-4e73-ad78-6d167189c0df\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd9r5fz" Jun 15 01:52:08.633097 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:08.633068 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd9r5fz" Jun 15 01:52:08.760202 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:08.760171 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd9r5fz"] Jun 15 01:52:08.762168 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:52:08.762144 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fa10e87_9d8f_4e73_ad78_6d167189c0df.slice/crio-eac7d42eb44e20a900c10768af3a399575c1cefb95ce06cf4cf8785d4b30dae0 WatchSource:0}: Error finding container eac7d42eb44e20a900c10768af3a399575c1cefb95ce06cf4cf8785d4b30dae0: Status 404 returned error can't find the container with id eac7d42eb44e20a900c10768af3a399575c1cefb95ce06cf4cf8785d4b30dae0 Jun 15 01:52:09.484254 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:09.484215 2578 generic.go:356] "Generic (PLEG): container finished" podID="6fa10e87-9d8f-4e73-ad78-6d167189c0df" containerID="d2f52342689fb51b5918dc4b1bf209f8003c4a72f77e8e71a70db58768388be1" exitCode=0 Jun 15 01:52:09.484640 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:09.484300 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd9r5fz" event={"ID":"6fa10e87-9d8f-4e73-ad78-6d167189c0df","Type":"ContainerDied","Data":"d2f52342689fb51b5918dc4b1bf209f8003c4a72f77e8e71a70db58768388be1"} Jun 15 01:52:09.484640 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:09.484350 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd9r5fz" event={"ID":"6fa10e87-9d8f-4e73-ad78-6d167189c0df","Type":"ContainerStarted","Data":"eac7d42eb44e20a900c10768af3a399575c1cefb95ce06cf4cf8785d4b30dae0"} Jun 15 01:52:10.495243 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:10.495207 2578 generic.go:356] "Generic (PLEG): container finished" podID="6fa10e87-9d8f-4e73-ad78-6d167189c0df" containerID="b292109489f28e3be40a70f7de8b37b5b80106fb90d4d84dd221e9a5df88af9c" exitCode=0 Jun 15 01:52:10.495705 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:10.495274 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd9r5fz" event={"ID":"6fa10e87-9d8f-4e73-ad78-6d167189c0df","Type":"ContainerDied","Data":"b292109489f28e3be40a70f7de8b37b5b80106fb90d4d84dd221e9a5df88af9c"} Jun 15 01:52:11.501984 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:11.501941 2578 generic.go:356] "Generic (PLEG): container finished" podID="6fa10e87-9d8f-4e73-ad78-6d167189c0df" containerID="da681b20a2a8fb7d0d8694506369d5c5e58bd5f17c82bd644105a6cdea45523a" exitCode=0 Jun 15 01:52:11.502457 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:11.502026 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd9r5fz" event={"ID":"6fa10e87-9d8f-4e73-ad78-6d167189c0df","Type":"ContainerDied","Data":"da681b20a2a8fb7d0d8694506369d5c5e58bd5f17c82bd644105a6cdea45523a"} Jun 15 01:52:12.629730 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:12.629694 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd9r5fz" Jun 15 01:52:12.736936 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:12.736904 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6fa10e87-9d8f-4e73-ad78-6d167189c0df-util\") pod \"6fa10e87-9d8f-4e73-ad78-6d167189c0df\" (UID: \"6fa10e87-9d8f-4e73-ad78-6d167189c0df\") " Jun 15 01:52:12.737132 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:12.736950 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j2nm4\" (UniqueName: \"kubernetes.io/projected/6fa10e87-9d8f-4e73-ad78-6d167189c0df-kube-api-access-j2nm4\") pod \"6fa10e87-9d8f-4e73-ad78-6d167189c0df\" (UID: \"6fa10e87-9d8f-4e73-ad78-6d167189c0df\") " Jun 15 01:52:12.737132 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:12.736977 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6fa10e87-9d8f-4e73-ad78-6d167189c0df-bundle\") pod \"6fa10e87-9d8f-4e73-ad78-6d167189c0df\" (UID: \"6fa10e87-9d8f-4e73-ad78-6d167189c0df\") " Jun 15 01:52:12.737727 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:12.737647 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fa10e87-9d8f-4e73-ad78-6d167189c0df-bundle" (OuterVolumeSpecName: "bundle") pod "6fa10e87-9d8f-4e73-ad78-6d167189c0df" (UID: "6fa10e87-9d8f-4e73-ad78-6d167189c0df"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 01:52:12.739259 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:12.739231 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fa10e87-9d8f-4e73-ad78-6d167189c0df-kube-api-access-j2nm4" (OuterVolumeSpecName: "kube-api-access-j2nm4") pod "6fa10e87-9d8f-4e73-ad78-6d167189c0df" (UID: "6fa10e87-9d8f-4e73-ad78-6d167189c0df"). InnerVolumeSpecName "kube-api-access-j2nm4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 01:52:12.743797 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:12.743772 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fa10e87-9d8f-4e73-ad78-6d167189c0df-util" (OuterVolumeSpecName: "util") pod "6fa10e87-9d8f-4e73-ad78-6d167189c0df" (UID: "6fa10e87-9d8f-4e73-ad78-6d167189c0df"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 01:52:12.837787 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:12.837748 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6fa10e87-9d8f-4e73-ad78-6d167189c0df-util\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:52:12.837787 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:12.837778 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j2nm4\" (UniqueName: \"kubernetes.io/projected/6fa10e87-9d8f-4e73-ad78-6d167189c0df-kube-api-access-j2nm4\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:52:12.837787 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:12.837791 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6fa10e87-9d8f-4e73-ad78-6d167189c0df-bundle\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:52:13.512344 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:13.512312 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd9r5fz" event={"ID":"6fa10e87-9d8f-4e73-ad78-6d167189c0df","Type":"ContainerDied","Data":"eac7d42eb44e20a900c10768af3a399575c1cefb95ce06cf4cf8785d4b30dae0"} Jun 15 01:52:13.512344 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:13.512339 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd9r5fz" Jun 15 01:52:13.512560 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:13.512358 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eac7d42eb44e20a900c10768af3a399575c1cefb95ce06cf4cf8785d4b30dae0" Jun 15 01:52:21.201858 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:21.201819 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7d494fbff9-9ndp7"] Jun 15 01:52:21.202286 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:21.202163 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fa10e87-9d8f-4e73-ad78-6d167189c0df" containerName="extract" Jun 15 01:52:21.202286 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:21.202175 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fa10e87-9d8f-4e73-ad78-6d167189c0df" containerName="extract" Jun 15 01:52:21.202286 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:21.202187 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fa10e87-9d8f-4e73-ad78-6d167189c0df" containerName="pull" Jun 15 01:52:21.202286 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:21.202193 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fa10e87-9d8f-4e73-ad78-6d167189c0df" containerName="pull" Jun 15 01:52:21.202286 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:21.202201 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fa10e87-9d8f-4e73-ad78-6d167189c0df" containerName="util" Jun 15 01:52:21.202286 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:21.202207 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fa10e87-9d8f-4e73-ad78-6d167189c0df" containerName="util" Jun 15 01:52:21.202286 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:21.202252 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fa10e87-9d8f-4e73-ad78-6d167189c0df" containerName="extract" Jun 15 01:52:21.205208 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:21.205186 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7d494fbff9-9ndp7" Jun 15 01:52:21.208286 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:21.208256 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-8ftgf\"" Jun 15 01:52:21.208412 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:21.208309 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 15 01:52:21.208558 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:21.208540 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 15 01:52:21.208624 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:21.208563 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 15 01:52:21.208731 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:21.208706 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 15 01:52:21.223636 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:21.223613 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7d494fbff9-9ndp7"] Jun 15 01:52:21.302616 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:21.302577 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/729a1b01-1fa0-44fb-88e2-c23c5cad4deb-webhook-cert\") pod \"opendatahub-operator-controller-manager-7d494fbff9-9ndp7\" (UID: \"729a1b01-1fa0-44fb-88e2-c23c5cad4deb\") " pod="opendatahub/opendatahub-operator-controller-manager-7d494fbff9-9ndp7" Jun 15 01:52:21.302793 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:21.302629 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/729a1b01-1fa0-44fb-88e2-c23c5cad4deb-tmp\") pod \"opendatahub-operator-controller-manager-7d494fbff9-9ndp7\" (UID: \"729a1b01-1fa0-44fb-88e2-c23c5cad4deb\") " pod="opendatahub/opendatahub-operator-controller-manager-7d494fbff9-9ndp7" Jun 15 01:52:21.302793 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:21.302709 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztzhg\" (UniqueName: \"kubernetes.io/projected/729a1b01-1fa0-44fb-88e2-c23c5cad4deb-kube-api-access-ztzhg\") pod \"opendatahub-operator-controller-manager-7d494fbff9-9ndp7\" (UID: \"729a1b01-1fa0-44fb-88e2-c23c5cad4deb\") " pod="opendatahub/opendatahub-operator-controller-manager-7d494fbff9-9ndp7" Jun 15 01:52:21.302793 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:21.302755 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/729a1b01-1fa0-44fb-88e2-c23c5cad4deb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7d494fbff9-9ndp7\" (UID: \"729a1b01-1fa0-44fb-88e2-c23c5cad4deb\") " pod="opendatahub/opendatahub-operator-controller-manager-7d494fbff9-9ndp7" Jun 15 01:52:21.302793 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:21.302777 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/729a1b01-1fa0-44fb-88e2-c23c5cad4deb-manifests\") pod \"opendatahub-operator-controller-manager-7d494fbff9-9ndp7\" (UID: \"729a1b01-1fa0-44fb-88e2-c23c5cad4deb\") " pod="opendatahub/opendatahub-operator-controller-manager-7d494fbff9-9ndp7" Jun 15 01:52:21.409136 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:21.404258 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/729a1b01-1fa0-44fb-88e2-c23c5cad4deb-manifests\") pod \"opendatahub-operator-controller-manager-7d494fbff9-9ndp7\" (UID: \"729a1b01-1fa0-44fb-88e2-c23c5cad4deb\") " pod="opendatahub/opendatahub-operator-controller-manager-7d494fbff9-9ndp7" Jun 15 01:52:21.409136 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:21.404340 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/729a1b01-1fa0-44fb-88e2-c23c5cad4deb-webhook-cert\") pod \"opendatahub-operator-controller-manager-7d494fbff9-9ndp7\" (UID: \"729a1b01-1fa0-44fb-88e2-c23c5cad4deb\") " pod="opendatahub/opendatahub-operator-controller-manager-7d494fbff9-9ndp7" Jun 15 01:52:21.409136 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:21.404395 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/729a1b01-1fa0-44fb-88e2-c23c5cad4deb-tmp\") pod \"opendatahub-operator-controller-manager-7d494fbff9-9ndp7\" (UID: \"729a1b01-1fa0-44fb-88e2-c23c5cad4deb\") " pod="opendatahub/opendatahub-operator-controller-manager-7d494fbff9-9ndp7" Jun 15 01:52:21.409136 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:21.404466 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztzhg\" (UniqueName: \"kubernetes.io/projected/729a1b01-1fa0-44fb-88e2-c23c5cad4deb-kube-api-access-ztzhg\") pod \"opendatahub-operator-controller-manager-7d494fbff9-9ndp7\" (UID: \"729a1b01-1fa0-44fb-88e2-c23c5cad4deb\") " pod="opendatahub/opendatahub-operator-controller-manager-7d494fbff9-9ndp7" Jun 15 01:52:21.409136 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:21.404515 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/729a1b01-1fa0-44fb-88e2-c23c5cad4deb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7d494fbff9-9ndp7\" (UID: \"729a1b01-1fa0-44fb-88e2-c23c5cad4deb\") " pod="opendatahub/opendatahub-operator-controller-manager-7d494fbff9-9ndp7" Jun 15 01:52:21.409136 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:21.407757 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/729a1b01-1fa0-44fb-88e2-c23c5cad4deb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7d494fbff9-9ndp7\" (UID: \"729a1b01-1fa0-44fb-88e2-c23c5cad4deb\") " pod="opendatahub/opendatahub-operator-controller-manager-7d494fbff9-9ndp7" Jun 15 01:52:21.409136 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:21.408060 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/729a1b01-1fa0-44fb-88e2-c23c5cad4deb-manifests\") pod \"opendatahub-operator-controller-manager-7d494fbff9-9ndp7\" (UID: \"729a1b01-1fa0-44fb-88e2-c23c5cad4deb\") " pod="opendatahub/opendatahub-operator-controller-manager-7d494fbff9-9ndp7" Jun 15 01:52:21.410568 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:21.410542 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/729a1b01-1fa0-44fb-88e2-c23c5cad4deb-webhook-cert\") pod \"opendatahub-operator-controller-manager-7d494fbff9-9ndp7\" (UID: \"729a1b01-1fa0-44fb-88e2-c23c5cad4deb\") " pod="opendatahub/opendatahub-operator-controller-manager-7d494fbff9-9ndp7" Jun 15 01:52:21.411139 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:21.411117 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/729a1b01-1fa0-44fb-88e2-c23c5cad4deb-tmp\") pod \"opendatahub-operator-controller-manager-7d494fbff9-9ndp7\" (UID: \"729a1b01-1fa0-44fb-88e2-c23c5cad4deb\") " pod="opendatahub/opendatahub-operator-controller-manager-7d494fbff9-9ndp7" Jun 15 01:52:21.416856 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:21.416830 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztzhg\" (UniqueName: \"kubernetes.io/projected/729a1b01-1fa0-44fb-88e2-c23c5cad4deb-kube-api-access-ztzhg\") pod \"opendatahub-operator-controller-manager-7d494fbff9-9ndp7\" (UID: \"729a1b01-1fa0-44fb-88e2-c23c5cad4deb\") " pod="opendatahub/opendatahub-operator-controller-manager-7d494fbff9-9ndp7" Jun 15 01:52:21.515970 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:21.515875 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7d494fbff9-9ndp7" Jun 15 01:52:21.656290 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:21.656256 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7d494fbff9-9ndp7"] Jun 15 01:52:21.660414 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:52:21.660377 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod729a1b01_1fa0_44fb_88e2_c23c5cad4deb.slice/crio-a352a79f8254f75e755578dd37c4ecc90c8435cc8bb0a48c9529f5a25b00d425 WatchSource:0}: Error finding container a352a79f8254f75e755578dd37c4ecc90c8435cc8bb0a48c9529f5a25b00d425: Status 404 returned error can't find the container with id a352a79f8254f75e755578dd37c4ecc90c8435cc8bb0a48c9529f5a25b00d425 Jun 15 01:52:22.555155 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:22.555110 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d494fbff9-9ndp7" event={"ID":"729a1b01-1fa0-44fb-88e2-c23c5cad4deb","Type":"ContainerStarted","Data":"a352a79f8254f75e755578dd37c4ecc90c8435cc8bb0a48c9529f5a25b00d425"} Jun 15 01:52:25.571333 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:25.571299 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d494fbff9-9ndp7" event={"ID":"729a1b01-1fa0-44fb-88e2-c23c5cad4deb","Type":"ContainerStarted","Data":"b5ac359c18b045704dd00732e76bd79fa3c5dd041aab2fee96c219917bb5751c"} Jun 15 01:52:26.577001 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:26.576970 2578 generic.go:356] "Generic (PLEG): container finished" podID="729a1b01-1fa0-44fb-88e2-c23c5cad4deb" containerID="b5ac359c18b045704dd00732e76bd79fa3c5dd041aab2fee96c219917bb5751c" exitCode=0 Jun 15 01:52:26.577458 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:26.577049 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d494fbff9-9ndp7" event={"ID":"729a1b01-1fa0-44fb-88e2-c23c5cad4deb","Type":"ContainerDied","Data":"b5ac359c18b045704dd00732e76bd79fa3c5dd041aab2fee96c219917bb5751c"} Jun 15 01:52:27.585810 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:27.585773 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d494fbff9-9ndp7" event={"ID":"729a1b01-1fa0-44fb-88e2-c23c5cad4deb","Type":"ContainerStarted","Data":"0a58b6784485d5b84035cf5c894bd7f1135c4738d952419682ad44fdf7269901"} Jun 15 01:52:27.586225 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:27.585898 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7d494fbff9-9ndp7" Jun 15 01:52:27.609459 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:27.609410 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7d494fbff9-9ndp7" podStartSLOduration=2.452988671 podStartE2EDuration="6.609394923s" totalImagesPullingTime="3.202451816s" totalInitContainerRuntime="953.954436ms" isStatefulPod=false podCreationTimestamp="2026-06-15 01:52:21 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 01:52:27.606593257 +0000 UTC m=+471.320319231" watchObservedRunningTime="2026-06-15 01:52:27.609394923 +0000 UTC m=+471.323120894" Jun 15 01:52:38.592314 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:38.592272 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7d494fbff9-9ndp7" Jun 15 01:52:40.713954 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:40.713911 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-644c7647f-gszx8"] Jun 15 01:52:40.724274 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:40.724249 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-644c7647f-gszx8" Jun 15 01:52:40.727165 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:40.727138 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jun 15 01:52:40.727340 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:40.727177 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-pfvbv\"" Jun 15 01:52:40.727478 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:40.727215 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jun 15 01:52:40.730038 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:40.730017 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-644c7647f-gszx8"] Jun 15 01:52:40.864297 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:40.864265 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0fcdf2ea-d316-456b-8f67-3454dec3558f-tls-certs\") pod \"kube-auth-proxy-644c7647f-gszx8\" (UID: \"0fcdf2ea-d316-456b-8f67-3454dec3558f\") " pod="openshift-ingress/kube-auth-proxy-644c7647f-gszx8" Jun 15 01:52:40.864469 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:40.864303 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrs46\" (UniqueName: \"kubernetes.io/projected/0fcdf2ea-d316-456b-8f67-3454dec3558f-kube-api-access-lrs46\") pod \"kube-auth-proxy-644c7647f-gszx8\" (UID: \"0fcdf2ea-d316-456b-8f67-3454dec3558f\") " pod="openshift-ingress/kube-auth-proxy-644c7647f-gszx8" Jun 15 01:52:40.864469 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:40.864335 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0fcdf2ea-d316-456b-8f67-3454dec3558f-tmp\") pod \"kube-auth-proxy-644c7647f-gszx8\" (UID: \"0fcdf2ea-d316-456b-8f67-3454dec3558f\") " pod="openshift-ingress/kube-auth-proxy-644c7647f-gszx8" Jun 15 01:52:40.920185 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:40.920148 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-6dc94f689f-jntw9"] Jun 15 01:52:40.923220 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:40.923203 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-6dc94f689f-jntw9" Jun 15 01:52:40.925919 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:40.925894 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jun 15 01:52:40.925919 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:40.925912 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-gqdcd\"" Jun 15 01:52:40.932486 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:40.932459 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-6dc94f689f-jntw9"] Jun 15 01:52:40.965063 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:40.964982 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0fcdf2ea-d316-456b-8f67-3454dec3558f-tls-certs\") pod \"kube-auth-proxy-644c7647f-gszx8\" (UID: \"0fcdf2ea-d316-456b-8f67-3454dec3558f\") " pod="openshift-ingress/kube-auth-proxy-644c7647f-gszx8" Jun 15 01:52:40.965063 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:40.965027 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrs46\" (UniqueName: \"kubernetes.io/projected/0fcdf2ea-d316-456b-8f67-3454dec3558f-kube-api-access-lrs46\") pod \"kube-auth-proxy-644c7647f-gszx8\" (UID: \"0fcdf2ea-d316-456b-8f67-3454dec3558f\") " pod="openshift-ingress/kube-auth-proxy-644c7647f-gszx8" Jun 15 01:52:40.965063 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:40.965059 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0fcdf2ea-d316-456b-8f67-3454dec3558f-tmp\") pod \"kube-auth-proxy-644c7647f-gszx8\" (UID: \"0fcdf2ea-d316-456b-8f67-3454dec3558f\") " pod="openshift-ingress/kube-auth-proxy-644c7647f-gszx8" Jun 15 01:52:40.967508 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:40.967480 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0fcdf2ea-d316-456b-8f67-3454dec3558f-tmp\") pod \"kube-auth-proxy-644c7647f-gszx8\" (UID: \"0fcdf2ea-d316-456b-8f67-3454dec3558f\") " pod="openshift-ingress/kube-auth-proxy-644c7647f-gszx8" Jun 15 01:52:40.967732 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:40.967715 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0fcdf2ea-d316-456b-8f67-3454dec3558f-tls-certs\") pod \"kube-auth-proxy-644c7647f-gszx8\" (UID: \"0fcdf2ea-d316-456b-8f67-3454dec3558f\") " pod="openshift-ingress/kube-auth-proxy-644c7647f-gszx8" Jun 15 01:52:40.976586 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:40.976562 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrs46\" (UniqueName: \"kubernetes.io/projected/0fcdf2ea-d316-456b-8f67-3454dec3558f-kube-api-access-lrs46\") pod \"kube-auth-proxy-644c7647f-gszx8\" (UID: \"0fcdf2ea-d316-456b-8f67-3454dec3558f\") " pod="openshift-ingress/kube-auth-proxy-644c7647f-gszx8" Jun 15 01:52:41.034569 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:41.034531 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-644c7647f-gszx8" Jun 15 01:52:41.065728 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:41.065694 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4cb41eb9-34d4-495b-8534-0a7b7f4b42d2-cert\") pod \"odh-model-controller-6dc94f689f-jntw9\" (UID: \"4cb41eb9-34d4-495b-8534-0a7b7f4b42d2\") " pod="opendatahub/odh-model-controller-6dc94f689f-jntw9" Jun 15 01:52:41.065728 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:41.065730 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57gnq\" (UniqueName: \"kubernetes.io/projected/4cb41eb9-34d4-495b-8534-0a7b7f4b42d2-kube-api-access-57gnq\") pod \"odh-model-controller-6dc94f689f-jntw9\" (UID: \"4cb41eb9-34d4-495b-8534-0a7b7f4b42d2\") " pod="opendatahub/odh-model-controller-6dc94f689f-jntw9" Jun 15 01:52:41.166762 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:41.166725 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4cb41eb9-34d4-495b-8534-0a7b7f4b42d2-cert\") pod \"odh-model-controller-6dc94f689f-jntw9\" (UID: \"4cb41eb9-34d4-495b-8534-0a7b7f4b42d2\") " pod="opendatahub/odh-model-controller-6dc94f689f-jntw9" Jun 15 01:52:41.166937 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:41.166770 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57gnq\" (UniqueName: \"kubernetes.io/projected/4cb41eb9-34d4-495b-8534-0a7b7f4b42d2-kube-api-access-57gnq\") pod \"odh-model-controller-6dc94f689f-jntw9\" (UID: \"4cb41eb9-34d4-495b-8534-0a7b7f4b42d2\") " pod="opendatahub/odh-model-controller-6dc94f689f-jntw9" Jun 15 01:52:41.166937 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:52:41.166887 2578 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jun 15 01:52:41.167072 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:52:41.166963 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4cb41eb9-34d4-495b-8534-0a7b7f4b42d2-cert podName:4cb41eb9-34d4-495b-8534-0a7b7f4b42d2 nodeName:}" failed. No retries permitted until 2026-06-15 01:52:41.666940842 +0000 UTC m=+485.380666802 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/4cb41eb9-34d4-495b-8534-0a7b7f4b42d2-cert") pod "odh-model-controller-6dc94f689f-jntw9" (UID: "4cb41eb9-34d4-495b-8534-0a7b7f4b42d2") : secret "odh-model-controller-webhook-cert" not found Jun 15 01:52:41.171658 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:41.171634 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-644c7647f-gszx8"] Jun 15 01:52:41.174452 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:52:41.174427 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fcdf2ea_d316_456b_8f67_3454dec3558f.slice/crio-c311f87aa716ac7740e518c66143050aebc73ee01e2a52a2556f3e217a8f7600 WatchSource:0}: Error finding container c311f87aa716ac7740e518c66143050aebc73ee01e2a52a2556f3e217a8f7600: Status 404 returned error can't find the container with id c311f87aa716ac7740e518c66143050aebc73ee01e2a52a2556f3e217a8f7600 Jun 15 01:52:41.176295 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:41.176175 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-57gnq\" (UniqueName: \"kubernetes.io/projected/4cb41eb9-34d4-495b-8534-0a7b7f4b42d2-kube-api-access-57gnq\") pod \"odh-model-controller-6dc94f689f-jntw9\" (UID: \"4cb41eb9-34d4-495b-8534-0a7b7f4b42d2\") " pod="opendatahub/odh-model-controller-6dc94f689f-jntw9" Jun 15 01:52:41.334990 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:41.334909 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-5dd488fc48-l7wlm"] Jun 15 01:52:41.339335 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:41.339316 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-5dd488fc48-l7wlm" Jun 15 01:52:41.342160 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:41.342136 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jun 15 01:52:41.362545 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:41.362502 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-5dd488fc48-l7wlm"] Jun 15 01:52:41.469406 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:41.469369 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40c9ba7d-66ce-49f9-a472-5445a57d1968-tls-certs\") pod \"model-serving-api-5dd488fc48-l7wlm\" (UID: \"40c9ba7d-66ce-49f9-a472-5445a57d1968\") " pod="opendatahub/model-serving-api-5dd488fc48-l7wlm" Jun 15 01:52:41.469600 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:41.469421 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qslc\" (UniqueName: \"kubernetes.io/projected/40c9ba7d-66ce-49f9-a472-5445a57d1968-kube-api-access-2qslc\") pod \"model-serving-api-5dd488fc48-l7wlm\" (UID: \"40c9ba7d-66ce-49f9-a472-5445a57d1968\") " pod="opendatahub/model-serving-api-5dd488fc48-l7wlm" Jun 15 01:52:41.570467 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:41.570419 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40c9ba7d-66ce-49f9-a472-5445a57d1968-tls-certs\") pod \"model-serving-api-5dd488fc48-l7wlm\" (UID: \"40c9ba7d-66ce-49f9-a472-5445a57d1968\") " pod="opendatahub/model-serving-api-5dd488fc48-l7wlm" Jun 15 01:52:41.570651 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:41.570481 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qslc\" (UniqueName: \"kubernetes.io/projected/40c9ba7d-66ce-49f9-a472-5445a57d1968-kube-api-access-2qslc\") pod \"model-serving-api-5dd488fc48-l7wlm\" (UID: \"40c9ba7d-66ce-49f9-a472-5445a57d1968\") " pod="opendatahub/model-serving-api-5dd488fc48-l7wlm" Jun 15 01:52:41.570651 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:52:41.570578 2578 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jun 15 01:52:41.570651 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:52:41.570647 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40c9ba7d-66ce-49f9-a472-5445a57d1968-tls-certs podName:40c9ba7d-66ce-49f9-a472-5445a57d1968 nodeName:}" failed. No retries permitted until 2026-06-15 01:52:42.070631984 +0000 UTC m=+485.784357934 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/40c9ba7d-66ce-49f9-a472-5445a57d1968-tls-certs") pod "model-serving-api-5dd488fc48-l7wlm" (UID: "40c9ba7d-66ce-49f9-a472-5445a57d1968") : secret "model-serving-api-tls" not found Jun 15 01:52:41.580910 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:41.580882 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qslc\" (UniqueName: \"kubernetes.io/projected/40c9ba7d-66ce-49f9-a472-5445a57d1968-kube-api-access-2qslc\") pod \"model-serving-api-5dd488fc48-l7wlm\" (UID: \"40c9ba7d-66ce-49f9-a472-5445a57d1968\") " pod="opendatahub/model-serving-api-5dd488fc48-l7wlm" Jun 15 01:52:41.649205 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:41.649170 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-644c7647f-gszx8" event={"ID":"0fcdf2ea-d316-456b-8f67-3454dec3558f","Type":"ContainerStarted","Data":"c311f87aa716ac7740e518c66143050aebc73ee01e2a52a2556f3e217a8f7600"} Jun 15 01:52:41.671677 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:41.671645 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4cb41eb9-34d4-495b-8534-0a7b7f4b42d2-cert\") pod \"odh-model-controller-6dc94f689f-jntw9\" (UID: \"4cb41eb9-34d4-495b-8534-0a7b7f4b42d2\") " pod="opendatahub/odh-model-controller-6dc94f689f-jntw9" Jun 15 01:52:41.674352 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:41.674325 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4cb41eb9-34d4-495b-8534-0a7b7f4b42d2-cert\") pod \"odh-model-controller-6dc94f689f-jntw9\" (UID: \"4cb41eb9-34d4-495b-8534-0a7b7f4b42d2\") " pod="opendatahub/odh-model-controller-6dc94f689f-jntw9" Jun 15 01:52:41.834139 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:41.834069 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-6dc94f689f-jntw9" Jun 15 01:52:42.005216 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:42.005187 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-6dc94f689f-jntw9"] Jun 15 01:52:42.007492 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:52:42.007422 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cb41eb9_34d4_495b_8534_0a7b7f4b42d2.slice/crio-2a433738bbbdebed23cb8d5823c8dfe08fb2aeaea993aced09d3d7e955131300 WatchSource:0}: Error finding container 2a433738bbbdebed23cb8d5823c8dfe08fb2aeaea993aced09d3d7e955131300: Status 404 returned error can't find the container with id 2a433738bbbdebed23cb8d5823c8dfe08fb2aeaea993aced09d3d7e955131300 Jun 15 01:52:42.076282 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:42.076235 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40c9ba7d-66ce-49f9-a472-5445a57d1968-tls-certs\") pod \"model-serving-api-5dd488fc48-l7wlm\" (UID: \"40c9ba7d-66ce-49f9-a472-5445a57d1968\") " pod="opendatahub/model-serving-api-5dd488fc48-l7wlm" Jun 15 01:52:42.080412 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:42.080354 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40c9ba7d-66ce-49f9-a472-5445a57d1968-tls-certs\") pod \"model-serving-api-5dd488fc48-l7wlm\" (UID: \"40c9ba7d-66ce-49f9-a472-5445a57d1968\") " pod="opendatahub/model-serving-api-5dd488fc48-l7wlm" Jun 15 01:52:42.251029 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:42.250946 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-5dd488fc48-l7wlm" Jun 15 01:52:42.415737 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:42.415704 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-5dd488fc48-l7wlm"] Jun 15 01:52:42.417548 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:52:42.417520 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40c9ba7d_66ce_49f9_a472_5445a57d1968.slice/crio-6294b312ad461cf351d5c468e5db575181fc594e78f9cc3da6c705717835c163 WatchSource:0}: Error finding container 6294b312ad461cf351d5c468e5db575181fc594e78f9cc3da6c705717835c163: Status 404 returned error can't find the container with id 6294b312ad461cf351d5c468e5db575181fc594e78f9cc3da6c705717835c163 Jun 15 01:52:42.451298 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:42.451263 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8rrr"] Jun 15 01:52:42.456425 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:42.456395 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8rrr" Jun 15 01:52:42.459237 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:42.459209 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ncvbf\"" Jun 15 01:52:42.459375 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:42.459270 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 15 01:52:42.459451 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:42.459219 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 15 01:52:42.463474 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:42.463412 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8rrr"] Jun 15 01:52:42.582525 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:42.582273 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ea0c4cf7-084b-4b1f-8b20-e7f2cb538da1-bundle\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8rrr\" (UID: \"ea0c4cf7-084b-4b1f-8b20-e7f2cb538da1\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8rrr" Jun 15 01:52:42.582525 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:42.582334 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ea0c4cf7-084b-4b1f-8b20-e7f2cb538da1-util\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8rrr\" (UID: \"ea0c4cf7-084b-4b1f-8b20-e7f2cb538da1\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8rrr" Jun 15 01:52:42.582525 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:42.582364 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7slrb\" (UniqueName: \"kubernetes.io/projected/ea0c4cf7-084b-4b1f-8b20-e7f2cb538da1-kube-api-access-7slrb\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8rrr\" (UID: \"ea0c4cf7-084b-4b1f-8b20-e7f2cb538da1\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8rrr" Jun 15 01:52:42.658794 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:42.658691 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-6dc94f689f-jntw9" event={"ID":"4cb41eb9-34d4-495b-8534-0a7b7f4b42d2","Type":"ContainerStarted","Data":"2a433738bbbdebed23cb8d5823c8dfe08fb2aeaea993aced09d3d7e955131300"} Jun 15 01:52:42.660741 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:42.660700 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-5dd488fc48-l7wlm" event={"ID":"40c9ba7d-66ce-49f9-a472-5445a57d1968","Type":"ContainerStarted","Data":"6294b312ad461cf351d5c468e5db575181fc594e78f9cc3da6c705717835c163"} Jun 15 01:52:42.683877 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:42.683810 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ea0c4cf7-084b-4b1f-8b20-e7f2cb538da1-bundle\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8rrr\" (UID: \"ea0c4cf7-084b-4b1f-8b20-e7f2cb538da1\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8rrr" Jun 15 01:52:42.684056 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:42.683872 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ea0c4cf7-084b-4b1f-8b20-e7f2cb538da1-util\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8rrr\" (UID: \"ea0c4cf7-084b-4b1f-8b20-e7f2cb538da1\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8rrr" Jun 15 01:52:42.684056 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:42.683914 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7slrb\" (UniqueName: \"kubernetes.io/projected/ea0c4cf7-084b-4b1f-8b20-e7f2cb538da1-kube-api-access-7slrb\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8rrr\" (UID: \"ea0c4cf7-084b-4b1f-8b20-e7f2cb538da1\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8rrr" Jun 15 01:52:42.684326 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:42.684274 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ea0c4cf7-084b-4b1f-8b20-e7f2cb538da1-bundle\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8rrr\" (UID: \"ea0c4cf7-084b-4b1f-8b20-e7f2cb538da1\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8rrr" Jun 15 01:52:42.684548 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:42.684516 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ea0c4cf7-084b-4b1f-8b20-e7f2cb538da1-util\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8rrr\" (UID: \"ea0c4cf7-084b-4b1f-8b20-e7f2cb538da1\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8rrr" Jun 15 01:52:42.695880 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:42.695848 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7slrb\" (UniqueName: \"kubernetes.io/projected/ea0c4cf7-084b-4b1f-8b20-e7f2cb538da1-kube-api-access-7slrb\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8rrr\" (UID: \"ea0c4cf7-084b-4b1f-8b20-e7f2cb538da1\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8rrr" Jun 15 01:52:42.777051 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:42.777015 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8rrr" Jun 15 01:52:44.252510 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:44.252483 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8rrr"] Jun 15 01:52:44.255292 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:52:44.255260 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea0c4cf7_084b_4b1f_8b20_e7f2cb538da1.slice/crio-c010267077a07fcf00ffadc47f6b551c0387727d30e68b99b136c3001c1d427c WatchSource:0}: Error finding container c010267077a07fcf00ffadc47f6b551c0387727d30e68b99b136c3001c1d427c: Status 404 returned error can't find the container with id c010267077a07fcf00ffadc47f6b551c0387727d30e68b99b136c3001c1d427c Jun 15 01:52:44.674520 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:44.674477 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-644c7647f-gszx8" event={"ID":"0fcdf2ea-d316-456b-8f67-3454dec3558f","Type":"ContainerStarted","Data":"373c77a48398e93c7b5d13204b30857a4a23fb095b6563d091a8208bdbc888f6"} Jun 15 01:52:44.676368 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:44.676337 2578 generic.go:356] "Generic (PLEG): container finished" podID="ea0c4cf7-084b-4b1f-8b20-e7f2cb538da1" containerID="26920d78e087e14e13143abde2b7570b3573203e51433595ec25e7dffd1ed2ab" exitCode=0 Jun 15 01:52:44.676503 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:44.676395 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8rrr" event={"ID":"ea0c4cf7-084b-4b1f-8b20-e7f2cb538da1","Type":"ContainerDied","Data":"26920d78e087e14e13143abde2b7570b3573203e51433595ec25e7dffd1ed2ab"} Jun 15 01:52:44.676503 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:44.676452 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8rrr" event={"ID":"ea0c4cf7-084b-4b1f-8b20-e7f2cb538da1","Type":"ContainerStarted","Data":"c010267077a07fcf00ffadc47f6b551c0387727d30e68b99b136c3001c1d427c"} Jun 15 01:52:44.691584 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:44.691532 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-644c7647f-gszx8" podStartSLOduration=1.695235772 podStartE2EDuration="4.691512512s" totalImagesPullingTime="2.99627674s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 01:52:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 01:52:44.689591281 +0000 UTC m=+488.403317253" watchObservedRunningTime="2026-06-15 01:52:44.691512512 +0000 UTC m=+488.405238485" Jun 15 01:52:46.689556 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:46.689516 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-5dd488fc48-l7wlm" event={"ID":"40c9ba7d-66ce-49f9-a472-5445a57d1968","Type":"ContainerStarted","Data":"cae86b5d48c6147397065712dabf75263815734428e23a7848b025c9b52ec55a"} Jun 15 01:52:46.690035 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:46.689730 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-5dd488fc48-l7wlm" Jun 15 01:52:46.691256 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:46.691221 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-6dc94f689f-jntw9" event={"ID":"4cb41eb9-34d4-495b-8534-0a7b7f4b42d2","Type":"ContainerStarted","Data":"bff0ab3ebe57260b67774faca228cdae4f89bd9fdd170ba8767de615c26b737a"} Jun 15 01:52:46.691393 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:46.691377 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-6dc94f689f-jntw9" Jun 15 01:52:46.706191 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:46.706149 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-5dd488fc48-l7wlm" podStartSLOduration=1.967249706 podStartE2EDuration="5.706134559s" totalImagesPullingTime="3.738884853s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 01:52:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 01:52:46.704855423 +0000 UTC m=+490.418581397" watchObservedRunningTime="2026-06-15 01:52:46.706134559 +0000 UTC m=+490.419860509" Jun 15 01:52:46.720072 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:46.720022 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-6dc94f689f-jntw9" podStartSLOduration=2.521319244 podStartE2EDuration="6.720009864s" totalImagesPullingTime="4.19869062s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 01:52:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 01:52:46.719391048 +0000 UTC m=+490.433117031" watchObservedRunningTime="2026-06-15 01:52:46.720009864 +0000 UTC m=+490.433735835" Jun 15 01:52:47.697720 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:47.697680 2578 generic.go:356] "Generic (PLEG): container finished" podID="ea0c4cf7-084b-4b1f-8b20-e7f2cb538da1" containerID="20199c54addd624e3d2a87422e40a9e656ee9f793b390f6c2074f6df7703a7cc" exitCode=0 Jun 15 01:52:47.698271 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:47.697768 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8rrr" event={"ID":"ea0c4cf7-084b-4b1f-8b20-e7f2cb538da1","Type":"ContainerDied","Data":"20199c54addd624e3d2a87422e40a9e656ee9f793b390f6c2074f6df7703a7cc"} Jun 15 01:52:48.704782 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:48.704753 2578 generic.go:356] "Generic (PLEG): container finished" podID="ea0c4cf7-084b-4b1f-8b20-e7f2cb538da1" containerID="dfcf98fa3286bdac216c20c0fe62923ffa65518ae2d32f430cadd5f31b70405e" exitCode=0 Jun 15 01:52:48.705206 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:48.704852 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8rrr" event={"ID":"ea0c4cf7-084b-4b1f-8b20-e7f2cb538da1","Type":"ContainerDied","Data":"dfcf98fa3286bdac216c20c0fe62923ffa65518ae2d32f430cadd5f31b70405e"} Jun 15 01:52:49.835641 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:49.835616 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8rrr" Jun 15 01:52:49.948037 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:49.947992 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ea0c4cf7-084b-4b1f-8b20-e7f2cb538da1-util\") pod \"ea0c4cf7-084b-4b1f-8b20-e7f2cb538da1\" (UID: \"ea0c4cf7-084b-4b1f-8b20-e7f2cb538da1\") " Jun 15 01:52:49.948235 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:49.948070 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7slrb\" (UniqueName: \"kubernetes.io/projected/ea0c4cf7-084b-4b1f-8b20-e7f2cb538da1-kube-api-access-7slrb\") pod \"ea0c4cf7-084b-4b1f-8b20-e7f2cb538da1\" (UID: \"ea0c4cf7-084b-4b1f-8b20-e7f2cb538da1\") " Jun 15 01:52:49.948235 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:49.948130 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ea0c4cf7-084b-4b1f-8b20-e7f2cb538da1-bundle\") pod \"ea0c4cf7-084b-4b1f-8b20-e7f2cb538da1\" (UID: \"ea0c4cf7-084b-4b1f-8b20-e7f2cb538da1\") " Jun 15 01:52:49.949045 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:49.949018 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea0c4cf7-084b-4b1f-8b20-e7f2cb538da1-bundle" (OuterVolumeSpecName: "bundle") pod "ea0c4cf7-084b-4b1f-8b20-e7f2cb538da1" (UID: "ea0c4cf7-084b-4b1f-8b20-e7f2cb538da1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 01:52:49.950480 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:49.950459 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea0c4cf7-084b-4b1f-8b20-e7f2cb538da1-kube-api-access-7slrb" (OuterVolumeSpecName: "kube-api-access-7slrb") pod "ea0c4cf7-084b-4b1f-8b20-e7f2cb538da1" (UID: "ea0c4cf7-084b-4b1f-8b20-e7f2cb538da1"). InnerVolumeSpecName "kube-api-access-7slrb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 01:52:49.952347 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:49.952310 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea0c4cf7-084b-4b1f-8b20-e7f2cb538da1-util" (OuterVolumeSpecName: "util") pod "ea0c4cf7-084b-4b1f-8b20-e7f2cb538da1" (UID: "ea0c4cf7-084b-4b1f-8b20-e7f2cb538da1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 01:52:50.049781 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:50.049669 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ea0c4cf7-084b-4b1f-8b20-e7f2cb538da1-util\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:52:50.049781 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:50.049720 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7slrb\" (UniqueName: \"kubernetes.io/projected/ea0c4cf7-084b-4b1f-8b20-e7f2cb538da1-kube-api-access-7slrb\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:52:50.049781 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:50.049735 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ea0c4cf7-084b-4b1f-8b20-e7f2cb538da1-bundle\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:52:50.716558 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:50.716530 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8rrr" Jun 15 01:52:50.716719 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:50.716571 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06d8rrr" event={"ID":"ea0c4cf7-084b-4b1f-8b20-e7f2cb538da1","Type":"ContainerDied","Data":"c010267077a07fcf00ffadc47f6b551c0387727d30e68b99b136c3001c1d427c"} Jun 15 01:52:50.716719 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:50.716626 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c010267077a07fcf00ffadc47f6b551c0387727d30e68b99b136c3001c1d427c" Jun 15 01:52:50.981455 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:50.981370 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-69fc866d76-4mzv8"] Jun 15 01:52:50.981822 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:50.981713 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea0c4cf7-084b-4b1f-8b20-e7f2cb538da1" containerName="util" Jun 15 01:52:50.981822 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:50.981724 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea0c4cf7-084b-4b1f-8b20-e7f2cb538da1" containerName="util" Jun 15 01:52:50.981822 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:50.981732 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea0c4cf7-084b-4b1f-8b20-e7f2cb538da1" containerName="pull" Jun 15 01:52:50.981822 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:50.981737 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea0c4cf7-084b-4b1f-8b20-e7f2cb538da1" containerName="pull" Jun 15 01:52:50.981822 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:50.981748 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea0c4cf7-084b-4b1f-8b20-e7f2cb538da1" containerName="extract" Jun 15 01:52:50.981822 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:50.981754 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea0c4cf7-084b-4b1f-8b20-e7f2cb538da1" containerName="extract" Jun 15 01:52:50.981822 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:50.981807 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea0c4cf7-084b-4b1f-8b20-e7f2cb538da1" containerName="extract" Jun 15 01:52:50.984708 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:50.984682 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-69fc866d76-4mzv8" Jun 15 01:52:50.988309 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:50.988287 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-qhww9\"" Jun 15 01:52:50.988671 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:50.988647 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jun 15 01:52:50.989099 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:50.989068 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-864bfd7bc9-6z22w"] Jun 15 01:52:50.992307 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:50.992282 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-6z22w" Jun 15 01:52:50.994777 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:50.994761 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-68s8p\"" Jun 15 01:52:50.994893 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:50.994875 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jun 15 01:52:51.012562 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:51.012523 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-69fc866d76-4mzv8"] Jun 15 01:52:51.015797 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:51.015766 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-864bfd7bc9-6z22w"] Jun 15 01:52:51.159590 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:51.159548 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5mll\" (UniqueName: \"kubernetes.io/projected/f694f872-296b-4205-98ef-d03f361e25ae-kube-api-access-b5mll\") pod \"llmisvc-controller-manager-864bfd7bc9-6z22w\" (UID: \"f694f872-296b-4205-98ef-d03f361e25ae\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-6z22w" Jun 15 01:52:51.159590 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:51.159602 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f694f872-296b-4205-98ef-d03f361e25ae-cert\") pod \"llmisvc-controller-manager-864bfd7bc9-6z22w\" (UID: \"f694f872-296b-4205-98ef-d03f361e25ae\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-6z22w" Jun 15 01:52:51.159814 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:51.159647 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76jwt\" (UniqueName: \"kubernetes.io/projected/019c220f-42d7-4691-b5d4-183b9f27c964-kube-api-access-76jwt\") pod \"kserve-controller-manager-69fc866d76-4mzv8\" (UID: \"019c220f-42d7-4691-b5d4-183b9f27c964\") " pod="opendatahub/kserve-controller-manager-69fc866d76-4mzv8" Jun 15 01:52:51.159814 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:51.159722 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/019c220f-42d7-4691-b5d4-183b9f27c964-cert\") pod \"kserve-controller-manager-69fc866d76-4mzv8\" (UID: \"019c220f-42d7-4691-b5d4-183b9f27c964\") " pod="opendatahub/kserve-controller-manager-69fc866d76-4mzv8" Jun 15 01:52:51.260461 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:51.260369 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5mll\" (UniqueName: \"kubernetes.io/projected/f694f872-296b-4205-98ef-d03f361e25ae-kube-api-access-b5mll\") pod \"llmisvc-controller-manager-864bfd7bc9-6z22w\" (UID: \"f694f872-296b-4205-98ef-d03f361e25ae\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-6z22w" Jun 15 01:52:51.260461 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:51.260415 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f694f872-296b-4205-98ef-d03f361e25ae-cert\") pod \"llmisvc-controller-manager-864bfd7bc9-6z22w\" (UID: \"f694f872-296b-4205-98ef-d03f361e25ae\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-6z22w" Jun 15 01:52:51.260695 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:51.260462 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76jwt\" (UniqueName: \"kubernetes.io/projected/019c220f-42d7-4691-b5d4-183b9f27c964-kube-api-access-76jwt\") pod \"kserve-controller-manager-69fc866d76-4mzv8\" (UID: \"019c220f-42d7-4691-b5d4-183b9f27c964\") " pod="opendatahub/kserve-controller-manager-69fc866d76-4mzv8" Jun 15 01:52:51.260695 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:51.260504 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/019c220f-42d7-4691-b5d4-183b9f27c964-cert\") pod \"kserve-controller-manager-69fc866d76-4mzv8\" (UID: \"019c220f-42d7-4691-b5d4-183b9f27c964\") " pod="opendatahub/kserve-controller-manager-69fc866d76-4mzv8" Jun 15 01:52:51.260695 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:52:51.260635 2578 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jun 15 01:52:51.260823 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:52:51.260701 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/019c220f-42d7-4691-b5d4-183b9f27c964-cert podName:019c220f-42d7-4691-b5d4-183b9f27c964 nodeName:}" failed. No retries permitted until 2026-06-15 01:52:51.760681485 +0000 UTC m=+495.474407434 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/019c220f-42d7-4691-b5d4-183b9f27c964-cert") pod "kserve-controller-manager-69fc866d76-4mzv8" (UID: "019c220f-42d7-4691-b5d4-183b9f27c964") : secret "kserve-webhook-server-cert" not found Jun 15 01:52:51.263118 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:51.263078 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f694f872-296b-4205-98ef-d03f361e25ae-cert\") pod \"llmisvc-controller-manager-864bfd7bc9-6z22w\" (UID: \"f694f872-296b-4205-98ef-d03f361e25ae\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-6z22w" Jun 15 01:52:51.271974 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:51.271944 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5mll\" (UniqueName: \"kubernetes.io/projected/f694f872-296b-4205-98ef-d03f361e25ae-kube-api-access-b5mll\") pod \"llmisvc-controller-manager-864bfd7bc9-6z22w\" (UID: \"f694f872-296b-4205-98ef-d03f361e25ae\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-6z22w" Jun 15 01:52:51.275510 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:51.275483 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-76jwt\" (UniqueName: \"kubernetes.io/projected/019c220f-42d7-4691-b5d4-183b9f27c964-kube-api-access-76jwt\") pod \"kserve-controller-manager-69fc866d76-4mzv8\" (UID: \"019c220f-42d7-4691-b5d4-183b9f27c964\") " pod="opendatahub/kserve-controller-manager-69fc866d76-4mzv8" Jun 15 01:52:51.304446 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:51.304412 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-6z22w" Jun 15 01:52:51.442427 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:51.442388 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-864bfd7bc9-6z22w"] Jun 15 01:52:51.444669 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:52:51.444638 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf694f872_296b_4205_98ef_d03f361e25ae.slice/crio-76cb990df4a0bfa28ecf564d2f4337d24eef3bf081b8b66cd393056757eca8da WatchSource:0}: Error finding container 76cb990df4a0bfa28ecf564d2f4337d24eef3bf081b8b66cd393056757eca8da: Status 404 returned error can't find the container with id 76cb990df4a0bfa28ecf564d2f4337d24eef3bf081b8b66cd393056757eca8da Jun 15 01:52:51.721962 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:51.721929 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-6z22w" event={"ID":"f694f872-296b-4205-98ef-d03f361e25ae","Type":"ContainerStarted","Data":"76cb990df4a0bfa28ecf564d2f4337d24eef3bf081b8b66cd393056757eca8da"} Jun 15 01:52:51.764820 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:51.764784 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/019c220f-42d7-4691-b5d4-183b9f27c964-cert\") pod \"kserve-controller-manager-69fc866d76-4mzv8\" (UID: \"019c220f-42d7-4691-b5d4-183b9f27c964\") " pod="opendatahub/kserve-controller-manager-69fc866d76-4mzv8" Jun 15 01:52:51.767313 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:51.767290 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/019c220f-42d7-4691-b5d4-183b9f27c964-cert\") pod \"kserve-controller-manager-69fc866d76-4mzv8\" (UID: \"019c220f-42d7-4691-b5d4-183b9f27c964\") " pod="opendatahub/kserve-controller-manager-69fc866d76-4mzv8" Jun 15 01:52:51.895889 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:51.895857 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-69fc866d76-4mzv8" Jun 15 01:52:52.065678 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:52.065643 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-69fc866d76-4mzv8"] Jun 15 01:52:52.129831 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:52:52.129797 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod019c220f_42d7_4691_b5d4_183b9f27c964.slice/crio-6507117c61188115331f83a43749c2a1235ac9fa54aba9b51e2cd88b045b9f42 WatchSource:0}: Error finding container 6507117c61188115331f83a43749c2a1235ac9fa54aba9b51e2cd88b045b9f42: Status 404 returned error can't find the container with id 6507117c61188115331f83a43749c2a1235ac9fa54aba9b51e2cd88b045b9f42 Jun 15 01:52:52.728502 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:52.728462 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-69fc866d76-4mzv8" event={"ID":"019c220f-42d7-4691-b5d4-183b9f27c964","Type":"ContainerStarted","Data":"6507117c61188115331f83a43749c2a1235ac9fa54aba9b51e2cd88b045b9f42"} Jun 15 01:52:55.745340 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:55.745305 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-6z22w" event={"ID":"f694f872-296b-4205-98ef-d03f361e25ae","Type":"ContainerStarted","Data":"2d9239ebe308983282dcd88866d460bea076cabec99d46c9f362c5c0edc248fa"} Jun 15 01:52:55.745825 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:55.745367 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-6z22w" Jun 15 01:52:55.746777 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:55.746750 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-69fc866d76-4mzv8" event={"ID":"019c220f-42d7-4691-b5d4-183b9f27c964","Type":"ContainerStarted","Data":"5046be210d53375c4c8ae27e5f44b92d8eb5f945f14d4b46cb72224db788582e"} Jun 15 01:52:55.746847 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:55.746803 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-69fc866d76-4mzv8" Jun 15 01:52:55.764955 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:55.764897 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-6z22w" podStartSLOduration=2.0883704339999998 podStartE2EDuration="5.764876931s" totalImagesPullingTime="3.676506497s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 01:52:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 01:52:55.763590212 +0000 UTC m=+499.477316185" watchObservedRunningTime="2026-06-15 01:52:55.764876931 +0000 UTC m=+499.478602915" Jun 15 01:52:55.782179 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:55.782129 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-69fc866d76-4mzv8" podStartSLOduration=2.788760732 podStartE2EDuration="5.782112301s" totalImagesPullingTime="2.993351569s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 01:52:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 01:52:55.780318912 +0000 UTC m=+499.494044887" watchObservedRunningTime="2026-06-15 01:52:55.782112301 +0000 UTC m=+499.495838270" Jun 15 01:52:57.700173 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:57.700137 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-6dc94f689f-jntw9" Jun 15 01:52:57.702164 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:52:57.702142 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-5dd488fc48-l7wlm" Jun 15 01:53:02.905975 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:02.905939 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aprnw9"] Jun 15 01:53:02.909563 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:02.909548 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aprnw9" Jun 15 01:53:02.913481 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:02.913461 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ncvbf\"" Jun 15 01:53:02.914380 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:02.914362 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 15 01:53:02.914466 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:02.914386 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 15 01:53:02.928925 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:02.928901 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aprnw9"] Jun 15 01:53:02.958031 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:02.958002 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a40743a2-4d26-4ce5-a5d5-58e41cc0de8a-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aprnw9\" (UID: \"a40743a2-4d26-4ce5-a5d5-58e41cc0de8a\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aprnw9" Jun 15 01:53:02.958167 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:02.958040 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxwwq\" (UniqueName: \"kubernetes.io/projected/a40743a2-4d26-4ce5-a5d5-58e41cc0de8a-kube-api-access-nxwwq\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aprnw9\" (UID: \"a40743a2-4d26-4ce5-a5d5-58e41cc0de8a\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aprnw9" Jun 15 01:53:02.958167 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:02.958069 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a40743a2-4d26-4ce5-a5d5-58e41cc0de8a-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aprnw9\" (UID: \"a40743a2-4d26-4ce5-a5d5-58e41cc0de8a\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aprnw9" Jun 15 01:53:03.059539 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:03.059498 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a40743a2-4d26-4ce5-a5d5-58e41cc0de8a-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aprnw9\" (UID: \"a40743a2-4d26-4ce5-a5d5-58e41cc0de8a\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aprnw9" Jun 15 01:53:03.059721 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:03.059581 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a40743a2-4d26-4ce5-a5d5-58e41cc0de8a-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aprnw9\" (UID: \"a40743a2-4d26-4ce5-a5d5-58e41cc0de8a\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aprnw9" Jun 15 01:53:03.059721 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:03.059614 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxwwq\" (UniqueName: \"kubernetes.io/projected/a40743a2-4d26-4ce5-a5d5-58e41cc0de8a-kube-api-access-nxwwq\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aprnw9\" (UID: \"a40743a2-4d26-4ce5-a5d5-58e41cc0de8a\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aprnw9" Jun 15 01:53:03.059947 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:03.059923 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a40743a2-4d26-4ce5-a5d5-58e41cc0de8a-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aprnw9\" (UID: \"a40743a2-4d26-4ce5-a5d5-58e41cc0de8a\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aprnw9" Jun 15 01:53:03.060005 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:03.059944 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a40743a2-4d26-4ce5-a5d5-58e41cc0de8a-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aprnw9\" (UID: \"a40743a2-4d26-4ce5-a5d5-58e41cc0de8a\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aprnw9" Jun 15 01:53:03.077170 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:03.077137 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxwwq\" (UniqueName: \"kubernetes.io/projected/a40743a2-4d26-4ce5-a5d5-58e41cc0de8a-kube-api-access-nxwwq\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aprnw9\" (UID: \"a40743a2-4d26-4ce5-a5d5-58e41cc0de8a\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aprnw9" Jun 15 01:53:03.218466 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:03.218375 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aprnw9" Jun 15 01:53:03.355195 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:03.355167 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aprnw9"] Jun 15 01:53:03.357298 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:53:03.357268 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda40743a2_4d26_4ce5_a5d5_58e41cc0de8a.slice/crio-2aa1b296f348a8acdeb057c998207fb3c919176f09ec91d1936f875fcf10ce45 WatchSource:0}: Error finding container 2aa1b296f348a8acdeb057c998207fb3c919176f09ec91d1936f875fcf10ce45: Status 404 returned error can't find the container with id 2aa1b296f348a8acdeb057c998207fb3c919176f09ec91d1936f875fcf10ce45 Jun 15 01:53:03.786906 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:03.786818 2578 generic.go:356] "Generic (PLEG): container finished" podID="a40743a2-4d26-4ce5-a5d5-58e41cc0de8a" containerID="2926c0e0935a5fe63f77bfe58a733d7c549c169feb3465e5b6c0de561227db74" exitCode=0 Jun 15 01:53:03.787047 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:03.786906 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aprnw9" event={"ID":"a40743a2-4d26-4ce5-a5d5-58e41cc0de8a","Type":"ContainerDied","Data":"2926c0e0935a5fe63f77bfe58a733d7c549c169feb3465e5b6c0de561227db74"} Jun 15 01:53:03.787047 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:03.786953 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aprnw9" event={"ID":"a40743a2-4d26-4ce5-a5d5-58e41cc0de8a","Type":"ContainerStarted","Data":"2aa1b296f348a8acdeb057c998207fb3c919176f09ec91d1936f875fcf10ce45"} Jun 15 01:53:04.793339 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:04.793308 2578 generic.go:356] "Generic (PLEG): container finished" podID="a40743a2-4d26-4ce5-a5d5-58e41cc0de8a" containerID="f13cf2e9de8f63caac754e4cbc81206fb17890a3c085575ba62e8f19087bc3fb" exitCode=0 Jun 15 01:53:04.793710 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:04.793393 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aprnw9" event={"ID":"a40743a2-4d26-4ce5-a5d5-58e41cc0de8a","Type":"ContainerDied","Data":"f13cf2e9de8f63caac754e4cbc81206fb17890a3c085575ba62e8f19087bc3fb"} Jun 15 01:53:05.204020 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:05.203978 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-operators/servicemesh-operator3-57f65f65fb-k7kj6"] Jun 15 01:53:05.207312 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:05.207289 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-57f65f65fb-k7kj6" Jun 15 01:53:05.210405 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:05.210379 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"openshift-service-ca.crt\"" Jun 15 01:53:05.210520 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:05.210470 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"kube-root-ca.crt\"" Jun 15 01:53:05.210520 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:05.210512 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-operators\"/\"servicemesh-operator3-dockercfg-ttb5v\"" Jun 15 01:53:05.221591 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:05.221570 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-57f65f65fb-k7kj6"] Jun 15 01:53:05.277948 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:05.277914 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzm9p\" (UniqueName: \"kubernetes.io/projected/6576348b-5d9e-4960-b3b7-351d766d83cf-kube-api-access-bzm9p\") pod \"servicemesh-operator3-57f65f65fb-k7kj6\" (UID: \"6576348b-5d9e-4960-b3b7-351d766d83cf\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-k7kj6" Jun 15 01:53:05.277948 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:05.277954 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/6576348b-5d9e-4960-b3b7-351d766d83cf-operator-config\") pod \"servicemesh-operator3-57f65f65fb-k7kj6\" (UID: \"6576348b-5d9e-4960-b3b7-351d766d83cf\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-k7kj6" Jun 15 01:53:05.379081 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:05.379042 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzm9p\" (UniqueName: \"kubernetes.io/projected/6576348b-5d9e-4960-b3b7-351d766d83cf-kube-api-access-bzm9p\") pod \"servicemesh-operator3-57f65f65fb-k7kj6\" (UID: \"6576348b-5d9e-4960-b3b7-351d766d83cf\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-k7kj6" Jun 15 01:53:05.379311 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:05.379111 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/6576348b-5d9e-4960-b3b7-351d766d83cf-operator-config\") pod \"servicemesh-operator3-57f65f65fb-k7kj6\" (UID: \"6576348b-5d9e-4960-b3b7-351d766d83cf\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-k7kj6" Jun 15 01:53:05.381930 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:05.381903 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/6576348b-5d9e-4960-b3b7-351d766d83cf-operator-config\") pod \"servicemesh-operator3-57f65f65fb-k7kj6\" (UID: \"6576348b-5d9e-4960-b3b7-351d766d83cf\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-k7kj6" Jun 15 01:53:05.389053 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:05.389027 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzm9p\" (UniqueName: \"kubernetes.io/projected/6576348b-5d9e-4960-b3b7-351d766d83cf-kube-api-access-bzm9p\") pod \"servicemesh-operator3-57f65f65fb-k7kj6\" (UID: \"6576348b-5d9e-4960-b3b7-351d766d83cf\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-k7kj6" Jun 15 01:53:05.517773 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:05.517677 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-57f65f65fb-k7kj6" Jun 15 01:53:05.666672 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:05.666639 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-57f65f65fb-k7kj6"] Jun 15 01:53:05.670052 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:53:05.670019 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6576348b_5d9e_4960_b3b7_351d766d83cf.slice/crio-cb6e288d29a97f140d4690bd9cb8fbad3e1a152bf00a22efad73f5dadfae4a97 WatchSource:0}: Error finding container cb6e288d29a97f140d4690bd9cb8fbad3e1a152bf00a22efad73f5dadfae4a97: Status 404 returned error can't find the container with id cb6e288d29a97f140d4690bd9cb8fbad3e1a152bf00a22efad73f5dadfae4a97 Jun 15 01:53:05.805526 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:05.805437 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-57f65f65fb-k7kj6" event={"ID":"6576348b-5d9e-4960-b3b7-351d766d83cf","Type":"ContainerStarted","Data":"cb6e288d29a97f140d4690bd9cb8fbad3e1a152bf00a22efad73f5dadfae4a97"} Jun 15 01:53:05.807572 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:05.807545 2578 generic.go:356] "Generic (PLEG): container finished" podID="a40743a2-4d26-4ce5-a5d5-58e41cc0de8a" containerID="b5200b58d156b98b7acd2ae6d866d9bb556facbaa364273cf7b447732e5e5896" exitCode=0 Jun 15 01:53:05.807697 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:05.807588 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aprnw9" event={"ID":"a40743a2-4d26-4ce5-a5d5-58e41cc0de8a","Type":"ContainerDied","Data":"b5200b58d156b98b7acd2ae6d866d9bb556facbaa364273cf7b447732e5e5896"} Jun 15 01:53:07.163377 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:07.163348 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aprnw9" Jun 15 01:53:07.196202 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:07.196170 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a40743a2-4d26-4ce5-a5d5-58e41cc0de8a-util\") pod \"a40743a2-4d26-4ce5-a5d5-58e41cc0de8a\" (UID: \"a40743a2-4d26-4ce5-a5d5-58e41cc0de8a\") " Jun 15 01:53:07.196391 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:07.196237 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nxwwq\" (UniqueName: \"kubernetes.io/projected/a40743a2-4d26-4ce5-a5d5-58e41cc0de8a-kube-api-access-nxwwq\") pod \"a40743a2-4d26-4ce5-a5d5-58e41cc0de8a\" (UID: \"a40743a2-4d26-4ce5-a5d5-58e41cc0de8a\") " Jun 15 01:53:07.196391 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:07.196329 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a40743a2-4d26-4ce5-a5d5-58e41cc0de8a-bundle\") pod \"a40743a2-4d26-4ce5-a5d5-58e41cc0de8a\" (UID: \"a40743a2-4d26-4ce5-a5d5-58e41cc0de8a\") " Jun 15 01:53:07.197569 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:07.197522 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a40743a2-4d26-4ce5-a5d5-58e41cc0de8a-bundle" (OuterVolumeSpecName: "bundle") pod "a40743a2-4d26-4ce5-a5d5-58e41cc0de8a" (UID: "a40743a2-4d26-4ce5-a5d5-58e41cc0de8a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 01:53:07.198955 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:07.198930 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a40743a2-4d26-4ce5-a5d5-58e41cc0de8a-kube-api-access-nxwwq" (OuterVolumeSpecName: "kube-api-access-nxwwq") pod "a40743a2-4d26-4ce5-a5d5-58e41cc0de8a" (UID: "a40743a2-4d26-4ce5-a5d5-58e41cc0de8a"). InnerVolumeSpecName "kube-api-access-nxwwq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 01:53:07.205130 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:07.204929 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a40743a2-4d26-4ce5-a5d5-58e41cc0de8a-util" (OuterVolumeSpecName: "util") pod "a40743a2-4d26-4ce5-a5d5-58e41cc0de8a" (UID: "a40743a2-4d26-4ce5-a5d5-58e41cc0de8a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 01:53:07.297994 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:07.297956 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a40743a2-4d26-4ce5-a5d5-58e41cc0de8a-util\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:53:07.297994 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:07.297991 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nxwwq\" (UniqueName: \"kubernetes.io/projected/a40743a2-4d26-4ce5-a5d5-58e41cc0de8a-kube-api-access-nxwwq\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:53:07.298222 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:07.298004 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a40743a2-4d26-4ce5-a5d5-58e41cc0de8a-bundle\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:53:07.823260 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:07.823227 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aprnw9" Jun 15 01:53:07.823260 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:07.823242 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aprnw9" event={"ID":"a40743a2-4d26-4ce5-a5d5-58e41cc0de8a","Type":"ContainerDied","Data":"2aa1b296f348a8acdeb057c998207fb3c919176f09ec91d1936f875fcf10ce45"} Jun 15 01:53:07.823509 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:07.823298 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2aa1b296f348a8acdeb057c998207fb3c919176f09ec91d1936f875fcf10ce45" Jun 15 01:53:08.830568 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:08.830529 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-57f65f65fb-k7kj6" event={"ID":"6576348b-5d9e-4960-b3b7-351d766d83cf","Type":"ContainerStarted","Data":"d600d5f564d2acf6fd46b176e9c2bea72769f0664ca4655511dddf016c2a175c"} Jun 15 01:53:08.831051 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:08.830655 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-operators/servicemesh-operator3-57f65f65fb-k7kj6" Jun 15 01:53:08.854982 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:08.854933 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-operators/servicemesh-operator3-57f65f65fb-k7kj6" podStartSLOduration=1.172259038 podStartE2EDuration="3.854917247s" totalImagesPullingTime="2.682658209s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 01:53:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 01:53:08.852183654 +0000 UTC m=+512.565909626" watchObservedRunningTime="2026-06-15 01:53:08.854917247 +0000 UTC m=+512.568643218" Jun 15 01:53:16.559537 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:16.559498 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/dashboard-redirect-79b56865f7-btnp2"] Jun 15 01:53:16.559901 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:16.559871 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a40743a2-4d26-4ce5-a5d5-58e41cc0de8a" containerName="extract" Jun 15 01:53:16.559901 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:16.559882 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a40743a2-4d26-4ce5-a5d5-58e41cc0de8a" containerName="extract" Jun 15 01:53:16.559901 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:16.559890 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a40743a2-4d26-4ce5-a5d5-58e41cc0de8a" containerName="util" Jun 15 01:53:16.559901 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:16.559899 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a40743a2-4d26-4ce5-a5d5-58e41cc0de8a" containerName="util" Jun 15 01:53:16.560035 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:16.559916 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a40743a2-4d26-4ce5-a5d5-58e41cc0de8a" containerName="pull" Jun 15 01:53:16.560035 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:16.559922 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a40743a2-4d26-4ce5-a5d5-58e41cc0de8a" containerName="pull" Jun 15 01:53:16.560035 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:16.560004 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a40743a2-4d26-4ce5-a5d5-58e41cc0de8a" containerName="extract" Jun 15 01:53:16.563491 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:16.563471 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-79b56865f7-btnp2" Jun 15 01:53:16.566295 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:16.566273 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-9px6r\"" Jun 15 01:53:16.566468 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:16.566451 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"dashboard-redirect-config\"" Jun 15 01:53:16.580195 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:16.580160 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-79b56865f7-btnp2"] Jun 15 01:53:16.676921 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:16.676880 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xd65p\" (UniqueName: \"kubernetes.io/projected/f890d1a2-5c4a-4eac-bd8f-8f7c11c07a57-kube-api-access-xd65p\") pod \"dashboard-redirect-79b56865f7-btnp2\" (UID: \"f890d1a2-5c4a-4eac-bd8f-8f7c11c07a57\") " pod="opendatahub/dashboard-redirect-79b56865f7-btnp2" Jun 15 01:53:16.677129 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:16.677001 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/f890d1a2-5c4a-4eac-bd8f-8f7c11c07a57-redirect-config\") pod \"dashboard-redirect-79b56865f7-btnp2\" (UID: \"f890d1a2-5c4a-4eac-bd8f-8f7c11c07a57\") " pod="opendatahub/dashboard-redirect-79b56865f7-btnp2" Jun 15 01:53:16.778097 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:16.778061 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xd65p\" (UniqueName: \"kubernetes.io/projected/f890d1a2-5c4a-4eac-bd8f-8f7c11c07a57-kube-api-access-xd65p\") pod \"dashboard-redirect-79b56865f7-btnp2\" (UID: \"f890d1a2-5c4a-4eac-bd8f-8f7c11c07a57\") " pod="opendatahub/dashboard-redirect-79b56865f7-btnp2" Jun 15 01:53:16.778261 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:16.778181 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/f890d1a2-5c4a-4eac-bd8f-8f7c11c07a57-redirect-config\") pod \"dashboard-redirect-79b56865f7-btnp2\" (UID: \"f890d1a2-5c4a-4eac-bd8f-8f7c11c07a57\") " pod="opendatahub/dashboard-redirect-79b56865f7-btnp2" Jun 15 01:53:16.778785 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:16.778768 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/f890d1a2-5c4a-4eac-bd8f-8f7c11c07a57-redirect-config\") pod \"dashboard-redirect-79b56865f7-btnp2\" (UID: \"f890d1a2-5c4a-4eac-bd8f-8f7c11c07a57\") " pod="opendatahub/dashboard-redirect-79b56865f7-btnp2" Jun 15 01:53:16.786883 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:16.786853 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xd65p\" (UniqueName: \"kubernetes.io/projected/f890d1a2-5c4a-4eac-bd8f-8f7c11c07a57-kube-api-access-xd65p\") pod \"dashboard-redirect-79b56865f7-btnp2\" (UID: \"f890d1a2-5c4a-4eac-bd8f-8f7c11c07a57\") " pod="opendatahub/dashboard-redirect-79b56865f7-btnp2" Jun 15 01:53:16.876584 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:16.876543 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-79b56865f7-btnp2" Jun 15 01:53:17.010448 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:17.010411 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-79b56865f7-btnp2"] Jun 15 01:53:17.012437 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:53:17.012409 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf890d1a2_5c4a_4eac_bd8f_8f7c11c07a57.slice/crio-b2005b9c68f06c843db8bebaa859c591e83af679b2525a9a307af96f0354db6b WatchSource:0}: Error finding container b2005b9c68f06c843db8bebaa859c591e83af679b2525a9a307af96f0354db6b: Status 404 returned error can't find the container with id b2005b9c68f06c843db8bebaa859c591e83af679b2525a9a307af96f0354db6b Jun 15 01:53:17.874547 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:17.874506 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-79b56865f7-btnp2" event={"ID":"f890d1a2-5c4a-4eac-bd8f-8f7c11c07a57","Type":"ContainerStarted","Data":"b2005b9c68f06c843db8bebaa859c591e83af679b2525a9a307af96f0354db6b"} Jun 15 01:53:19.837886 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:19.837799 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-operators/servicemesh-operator3-57f65f65fb-k7kj6" Jun 15 01:53:20.897047 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:20.897006 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-79b56865f7-btnp2" event={"ID":"f890d1a2-5c4a-4eac-bd8f-8f7c11c07a57","Type":"ContainerStarted","Data":"33a699d741a2a219291437dfafab88035fffb9114324d7979b369028975379dd"} Jun 15 01:53:20.921504 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:20.921453 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/dashboard-redirect-79b56865f7-btnp2" podStartSLOduration=1.6425824709999999 podStartE2EDuration="4.921437041s" totalImagesPullingTime="3.27885457s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 01:53:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 01:53:20.91970159 +0000 UTC m=+524.633427563" watchObservedRunningTime="2026-06-15 01:53:20.921437041 +0000 UTC m=+524.635163054" Jun 15 01:53:26.753676 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:26.753643 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-6z22w" Jun 15 01:53:26.756650 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:26.756627 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-69fc866d76-4mzv8" Jun 15 01:53:29.603152 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:29.603111 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-75c67f8887-4nlft"] Jun 15 01:53:29.608176 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:29.608153 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-4nlft" Jun 15 01:53:29.610819 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:29.610789 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jun 15 01:53:29.610819 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:29.610797 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jun 15 01:53:29.611000 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:29.610810 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jun 15 01:53:29.611158 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:29.611141 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jun 15 01:53:29.611267 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:29.611143 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-h868n\"" Jun 15 01:53:29.618783 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:29.618757 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-75c67f8887-4nlft"] Jun 15 01:53:29.691526 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:29.691489 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwjwt\" (UniqueName: \"kubernetes.io/projected/b50e4740-5402-4abc-8737-a2aebb7e69c7-kube-api-access-xwjwt\") pod \"istiod-openshift-gateway-75c67f8887-4nlft\" (UID: \"b50e4740-5402-4abc-8737-a2aebb7e69c7\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-4nlft" Jun 15 01:53:29.691526 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:29.691528 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b50e4740-5402-4abc-8737-a2aebb7e69c7-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-75c67f8887-4nlft\" (UID: \"b50e4740-5402-4abc-8737-a2aebb7e69c7\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-4nlft" Jun 15 01:53:29.691734 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:29.691550 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b50e4740-5402-4abc-8737-a2aebb7e69c7-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-75c67f8887-4nlft\" (UID: \"b50e4740-5402-4abc-8737-a2aebb7e69c7\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-4nlft" Jun 15 01:53:29.691734 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:29.691673 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b50e4740-5402-4abc-8737-a2aebb7e69c7-istio-token\") pod \"istiod-openshift-gateway-75c67f8887-4nlft\" (UID: \"b50e4740-5402-4abc-8737-a2aebb7e69c7\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-4nlft" Jun 15 01:53:29.691734 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:29.691709 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b50e4740-5402-4abc-8737-a2aebb7e69c7-cacerts\") pod \"istiod-openshift-gateway-75c67f8887-4nlft\" (UID: \"b50e4740-5402-4abc-8737-a2aebb7e69c7\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-4nlft" Jun 15 01:53:29.691828 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:29.691740 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b50e4740-5402-4abc-8737-a2aebb7e69c7-istio-kubeconfig\") pod \"istiod-openshift-gateway-75c67f8887-4nlft\" (UID: \"b50e4740-5402-4abc-8737-a2aebb7e69c7\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-4nlft" Jun 15 01:53:29.691828 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:29.691767 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b50e4740-5402-4abc-8737-a2aebb7e69c7-local-certs\") pod \"istiod-openshift-gateway-75c67f8887-4nlft\" (UID: \"b50e4740-5402-4abc-8737-a2aebb7e69c7\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-4nlft" Jun 15 01:53:29.793298 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:29.793255 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b50e4740-5402-4abc-8737-a2aebb7e69c7-istio-token\") pod \"istiod-openshift-gateway-75c67f8887-4nlft\" (UID: \"b50e4740-5402-4abc-8737-a2aebb7e69c7\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-4nlft" Jun 15 01:53:29.793298 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:29.793303 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b50e4740-5402-4abc-8737-a2aebb7e69c7-cacerts\") pod \"istiod-openshift-gateway-75c67f8887-4nlft\" (UID: \"b50e4740-5402-4abc-8737-a2aebb7e69c7\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-4nlft" Jun 15 01:53:29.793563 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:29.793323 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b50e4740-5402-4abc-8737-a2aebb7e69c7-istio-kubeconfig\") pod \"istiod-openshift-gateway-75c67f8887-4nlft\" (UID: \"b50e4740-5402-4abc-8737-a2aebb7e69c7\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-4nlft" Jun 15 01:53:29.793563 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:29.793347 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b50e4740-5402-4abc-8737-a2aebb7e69c7-local-certs\") pod \"istiod-openshift-gateway-75c67f8887-4nlft\" (UID: \"b50e4740-5402-4abc-8737-a2aebb7e69c7\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-4nlft" Jun 15 01:53:29.793563 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:29.793391 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwjwt\" (UniqueName: \"kubernetes.io/projected/b50e4740-5402-4abc-8737-a2aebb7e69c7-kube-api-access-xwjwt\") pod \"istiod-openshift-gateway-75c67f8887-4nlft\" (UID: \"b50e4740-5402-4abc-8737-a2aebb7e69c7\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-4nlft" Jun 15 01:53:29.793563 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:29.793428 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b50e4740-5402-4abc-8737-a2aebb7e69c7-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-75c67f8887-4nlft\" (UID: \"b50e4740-5402-4abc-8737-a2aebb7e69c7\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-4nlft" Jun 15 01:53:29.793765 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:29.793636 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b50e4740-5402-4abc-8737-a2aebb7e69c7-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-75c67f8887-4nlft\" (UID: \"b50e4740-5402-4abc-8737-a2aebb7e69c7\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-4nlft" Jun 15 01:53:29.794376 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:29.794245 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b50e4740-5402-4abc-8737-a2aebb7e69c7-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-75c67f8887-4nlft\" (UID: \"b50e4740-5402-4abc-8737-a2aebb7e69c7\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-4nlft" Jun 15 01:53:29.796042 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:29.796021 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b50e4740-5402-4abc-8737-a2aebb7e69c7-istio-kubeconfig\") pod \"istiod-openshift-gateway-75c67f8887-4nlft\" (UID: \"b50e4740-5402-4abc-8737-a2aebb7e69c7\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-4nlft" Jun 15 01:53:29.796392 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:29.796373 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b50e4740-5402-4abc-8737-a2aebb7e69c7-cacerts\") pod \"istiod-openshift-gateway-75c67f8887-4nlft\" (UID: \"b50e4740-5402-4abc-8737-a2aebb7e69c7\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-4nlft" Jun 15 01:53:29.796475 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:29.796434 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b50e4740-5402-4abc-8737-a2aebb7e69c7-local-certs\") pod \"istiod-openshift-gateway-75c67f8887-4nlft\" (UID: \"b50e4740-5402-4abc-8737-a2aebb7e69c7\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-4nlft" Jun 15 01:53:29.796513 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:29.796489 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b50e4740-5402-4abc-8737-a2aebb7e69c7-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-75c67f8887-4nlft\" (UID: \"b50e4740-5402-4abc-8737-a2aebb7e69c7\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-4nlft" Jun 15 01:53:29.801282 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:29.801262 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b50e4740-5402-4abc-8737-a2aebb7e69c7-istio-token\") pod \"istiod-openshift-gateway-75c67f8887-4nlft\" (UID: \"b50e4740-5402-4abc-8737-a2aebb7e69c7\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-4nlft" Jun 15 01:53:29.801737 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:29.801714 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwjwt\" (UniqueName: \"kubernetes.io/projected/b50e4740-5402-4abc-8737-a2aebb7e69c7-kube-api-access-xwjwt\") pod \"istiod-openshift-gateway-75c67f8887-4nlft\" (UID: \"b50e4740-5402-4abc-8737-a2aebb7e69c7\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-4nlft" Jun 15 01:53:29.918329 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:29.918295 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-4nlft" Jun 15 01:53:30.060390 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:30.060360 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-75c67f8887-4nlft"] Jun 15 01:53:30.063253 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:53:30.063225 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb50e4740_5402_4abc_8737_a2aebb7e69c7.slice/crio-3495c00bee0eea4fb92ba98e7ea30aa76a615b0cf3bdac42f1b0acf58cf89b15 WatchSource:0}: Error finding container 3495c00bee0eea4fb92ba98e7ea30aa76a615b0cf3bdac42f1b0acf58cf89b15: Status 404 returned error can't find the container with id 3495c00bee0eea4fb92ba98e7ea30aa76a615b0cf3bdac42f1b0acf58cf89b15 Jun 15 01:53:30.948859 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:30.948818 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-4nlft" event={"ID":"b50e4740-5402-4abc-8737-a2aebb7e69c7","Type":"ContainerStarted","Data":"3495c00bee0eea4fb92ba98e7ea30aa76a615b0cf3bdac42f1b0acf58cf89b15"} Jun 15 01:53:32.635905 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:32.635859 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jun 15 01:53:32.636219 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:32.635943 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jun 15 01:53:32.960843 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:32.960749 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-4nlft" event={"ID":"b50e4740-5402-4abc-8737-a2aebb7e69c7","Type":"ContainerStarted","Data":"1ac842e07652b8e23b45218c4399edac8c62c861cc6ce1104d3e8c8c983cd6ca"} Jun 15 01:53:32.960843 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:32.960812 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-4nlft" Jun 15 01:53:32.981729 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:32.981677 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-4nlft" podStartSLOduration=1.411605884 podStartE2EDuration="3.981663581s" totalImagesPullingTime="2.570057697s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 01:53:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 01:53:32.979903185 +0000 UTC m=+536.693629159" watchObservedRunningTime="2026-06-15 01:53:32.981663581 +0000 UTC m=+536.695389552" Jun 15 01:53:33.967992 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:33.967953 2578 patch_prober.go:28] interesting pod/istiod-openshift-gateway-75c67f8887-4nlft container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jun 15 01:53:33.968393 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:33.968017 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-4nlft" podUID="b50e4740-5402-4abc-8737-a2aebb7e69c7" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 15 01:53:36.967398 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:36.967364 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-4nlft" Jun 15 01:53:57.779889 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:57.779850 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p7crd"] Jun 15 01:53:57.791402 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:57.791375 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p7crd"] Jun 15 01:53:57.791533 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:57.791492 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p7crd" Jun 15 01:53:57.794308 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:57.794270 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 15 01:53:57.795523 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:57.795504 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-kcgnv\"" Jun 15 01:53:57.795642 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:57.795511 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 15 01:53:57.820955 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:57.820924 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/43f8bbf1-6346-42b0-9581-547004591362-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p7crd\" (UID: \"43f8bbf1-6346-42b0-9581-547004591362\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p7crd" Jun 15 01:53:57.821145 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:57.821110 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/43f8bbf1-6346-42b0-9581-547004591362-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p7crd\" (UID: \"43f8bbf1-6346-42b0-9581-547004591362\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p7crd" Jun 15 01:53:57.821312 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:57.821292 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vkct\" (UniqueName: \"kubernetes.io/projected/43f8bbf1-6346-42b0-9581-547004591362-kube-api-access-4vkct\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p7crd\" (UID: \"43f8bbf1-6346-42b0-9581-547004591362\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p7crd" Jun 15 01:53:57.922570 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:57.922526 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/43f8bbf1-6346-42b0-9581-547004591362-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p7crd\" (UID: \"43f8bbf1-6346-42b0-9581-547004591362\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p7crd" Jun 15 01:53:57.922760 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:57.922575 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vkct\" (UniqueName: \"kubernetes.io/projected/43f8bbf1-6346-42b0-9581-547004591362-kube-api-access-4vkct\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p7crd\" (UID: \"43f8bbf1-6346-42b0-9581-547004591362\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p7crd" Jun 15 01:53:57.922760 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:57.922622 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/43f8bbf1-6346-42b0-9581-547004591362-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p7crd\" (UID: \"43f8bbf1-6346-42b0-9581-547004591362\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p7crd" Jun 15 01:53:57.922902 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:57.922882 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/43f8bbf1-6346-42b0-9581-547004591362-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p7crd\" (UID: \"43f8bbf1-6346-42b0-9581-547004591362\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p7crd" Jun 15 01:53:57.922996 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:57.922977 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/43f8bbf1-6346-42b0-9581-547004591362-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p7crd\" (UID: \"43f8bbf1-6346-42b0-9581-547004591362\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p7crd" Jun 15 01:53:57.930683 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:57.930663 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vkct\" (UniqueName: \"kubernetes.io/projected/43f8bbf1-6346-42b0-9581-547004591362-kube-api-access-4vkct\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p7crd\" (UID: \"43f8bbf1-6346-42b0-9581-547004591362\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p7crd" Jun 15 01:53:58.101663 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:58.101570 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p7crd" Jun 15 01:53:58.245224 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:58.245196 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p7crd"] Jun 15 01:53:58.247184 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:53:58.247148 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43f8bbf1_6346_42b0_9581_547004591362.slice/crio-e05b660e1a8619acdf5598cc1df72083487c7f84931f9fe0fd496c6e0f45c393 WatchSource:0}: Error finding container e05b660e1a8619acdf5598cc1df72083487c7f84931f9fe0fd496c6e0f45c393: Status 404 returned error can't find the container with id e05b660e1a8619acdf5598cc1df72083487c7f84931f9fe0fd496c6e0f45c393 Jun 15 01:53:58.573666 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:58.573626 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12tpbp"] Jun 15 01:53:58.577280 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:58.577265 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12tpbp" Jun 15 01:53:58.587230 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:58.587207 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12tpbp"] Jun 15 01:53:58.627442 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:58.627405 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqkgf\" (UniqueName: \"kubernetes.io/projected/9ccbd467-a8f2-43a8-ad76-b08698c29c54-kube-api-access-lqkgf\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12tpbp\" (UID: \"9ccbd467-a8f2-43a8-ad76-b08698c29c54\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12tpbp" Jun 15 01:53:58.627585 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:58.627467 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9ccbd467-a8f2-43a8-ad76-b08698c29c54-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12tpbp\" (UID: \"9ccbd467-a8f2-43a8-ad76-b08698c29c54\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12tpbp" Jun 15 01:53:58.627585 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:58.627539 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9ccbd467-a8f2-43a8-ad76-b08698c29c54-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12tpbp\" (UID: \"9ccbd467-a8f2-43a8-ad76-b08698c29c54\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12tpbp" Jun 15 01:53:58.728472 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:58.728435 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9ccbd467-a8f2-43a8-ad76-b08698c29c54-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12tpbp\" (UID: \"9ccbd467-a8f2-43a8-ad76-b08698c29c54\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12tpbp" Jun 15 01:53:58.728644 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:58.728483 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9ccbd467-a8f2-43a8-ad76-b08698c29c54-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12tpbp\" (UID: \"9ccbd467-a8f2-43a8-ad76-b08698c29c54\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12tpbp" Jun 15 01:53:58.728644 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:58.728527 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqkgf\" (UniqueName: \"kubernetes.io/projected/9ccbd467-a8f2-43a8-ad76-b08698c29c54-kube-api-access-lqkgf\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12tpbp\" (UID: \"9ccbd467-a8f2-43a8-ad76-b08698c29c54\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12tpbp" Jun 15 01:53:58.728924 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:58.728901 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9ccbd467-a8f2-43a8-ad76-b08698c29c54-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12tpbp\" (UID: \"9ccbd467-a8f2-43a8-ad76-b08698c29c54\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12tpbp" Jun 15 01:53:58.728960 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:58.728908 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9ccbd467-a8f2-43a8-ad76-b08698c29c54-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12tpbp\" (UID: \"9ccbd467-a8f2-43a8-ad76-b08698c29c54\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12tpbp" Jun 15 01:53:58.736499 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:58.736475 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqkgf\" (UniqueName: \"kubernetes.io/projected/9ccbd467-a8f2-43a8-ad76-b08698c29c54-kube-api-access-lqkgf\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12tpbp\" (UID: \"9ccbd467-a8f2-43a8-ad76-b08698c29c54\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12tpbp" Jun 15 01:53:58.886620 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:58.886587 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12tpbp" Jun 15 01:53:59.023912 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:59.023882 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12tpbp"] Jun 15 01:53:59.025529 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:53:59.025483 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ccbd467_a8f2_43a8_ad76_b08698c29c54.slice/crio-45bb4aff09399c36331597edad05407eb343b2c546013424f7a8f5cac0ff4381 WatchSource:0}: Error finding container 45bb4aff09399c36331597edad05407eb343b2c546013424f7a8f5cac0ff4381: Status 404 returned error can't find the container with id 45bb4aff09399c36331597edad05407eb343b2c546013424f7a8f5cac0ff4381 Jun 15 01:53:59.096680 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:59.096650 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12tpbp" event={"ID":"9ccbd467-a8f2-43a8-ad76-b08698c29c54","Type":"ContainerStarted","Data":"45bb4aff09399c36331597edad05407eb343b2c546013424f7a8f5cac0ff4381"} Jun 15 01:53:59.098316 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:59.098286 2578 generic.go:356] "Generic (PLEG): container finished" podID="43f8bbf1-6346-42b0-9581-547004591362" containerID="9920c61412bbb5765c63cf2507fa6b09268754cf75585c6f4e80bfd2e0811e41" exitCode=0 Jun 15 01:53:59.098415 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:59.098367 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p7crd" event={"ID":"43f8bbf1-6346-42b0-9581-547004591362","Type":"ContainerDied","Data":"9920c61412bbb5765c63cf2507fa6b09268754cf75585c6f4e80bfd2e0811e41"} Jun 15 01:53:59.098455 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:59.098409 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p7crd" event={"ID":"43f8bbf1-6346-42b0-9581-547004591362","Type":"ContainerStarted","Data":"e05b660e1a8619acdf5598cc1df72083487c7f84931f9fe0fd496c6e0f45c393"} Jun 15 01:53:59.168824 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:59.168792 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v2wpk"] Jun 15 01:53:59.172291 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:59.172275 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v2wpk" Jun 15 01:53:59.179877 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:59.179855 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v2wpk"] Jun 15 01:53:59.231870 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:59.231838 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f483827b-baf7-4a44-8bbb-8027562a973e-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v2wpk\" (UID: \"f483827b-baf7-4a44-8bbb-8027562a973e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v2wpk" Jun 15 01:53:59.232022 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:59.231883 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxhbl\" (UniqueName: \"kubernetes.io/projected/f483827b-baf7-4a44-8bbb-8027562a973e-kube-api-access-pxhbl\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v2wpk\" (UID: \"f483827b-baf7-4a44-8bbb-8027562a973e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v2wpk" Jun 15 01:53:59.232022 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:59.231920 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f483827b-baf7-4a44-8bbb-8027562a973e-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v2wpk\" (UID: \"f483827b-baf7-4a44-8bbb-8027562a973e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v2wpk" Jun 15 01:53:59.332322 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:59.332285 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f483827b-baf7-4a44-8bbb-8027562a973e-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v2wpk\" (UID: \"f483827b-baf7-4a44-8bbb-8027562a973e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v2wpk" Jun 15 01:53:59.332477 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:59.332345 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxhbl\" (UniqueName: \"kubernetes.io/projected/f483827b-baf7-4a44-8bbb-8027562a973e-kube-api-access-pxhbl\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v2wpk\" (UID: \"f483827b-baf7-4a44-8bbb-8027562a973e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v2wpk" Jun 15 01:53:59.332477 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:59.332386 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f483827b-baf7-4a44-8bbb-8027562a973e-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v2wpk\" (UID: \"f483827b-baf7-4a44-8bbb-8027562a973e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v2wpk" Jun 15 01:53:59.332664 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:59.332645 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f483827b-baf7-4a44-8bbb-8027562a973e-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v2wpk\" (UID: \"f483827b-baf7-4a44-8bbb-8027562a973e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v2wpk" Jun 15 01:53:59.332713 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:59.332671 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f483827b-baf7-4a44-8bbb-8027562a973e-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v2wpk\" (UID: \"f483827b-baf7-4a44-8bbb-8027562a973e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v2wpk" Jun 15 01:53:59.341560 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:59.341530 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxhbl\" (UniqueName: \"kubernetes.io/projected/f483827b-baf7-4a44-8bbb-8027562a973e-kube-api-access-pxhbl\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v2wpk\" (UID: \"f483827b-baf7-4a44-8bbb-8027562a973e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v2wpk" Jun 15 01:53:59.508999 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:59.508916 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v2wpk" Jun 15 01:53:59.645179 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:59.645154 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v2wpk"] Jun 15 01:53:59.646778 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:53:59.646750 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf483827b_baf7_4a44_8bbb_8027562a973e.slice/crio-72993ac866800c8b735b4b96c7a9ea6487b79fd78bc6a19f4b406a3e2f1c2757 WatchSource:0}: Error finding container 72993ac866800c8b735b4b96c7a9ea6487b79fd78bc6a19f4b406a3e2f1c2757: Status 404 returned error can't find the container with id 72993ac866800c8b735b4b96c7a9ea6487b79fd78bc6a19f4b406a3e2f1c2757 Jun 15 01:53:59.811317 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:59.811288 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wmpwk"] Jun 15 01:53:59.815217 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:59.815197 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wmpwk" Jun 15 01:53:59.824268 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:59.824239 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wmpwk"] Jun 15 01:53:59.836612 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:59.836581 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e4f239ad-a463-4488-95af-aa5739b6a334-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wmpwk\" (UID: \"e4f239ad-a463-4488-95af-aa5739b6a334\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wmpwk" Jun 15 01:53:59.836714 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:59.836660 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6z7ws\" (UniqueName: \"kubernetes.io/projected/e4f239ad-a463-4488-95af-aa5739b6a334-kube-api-access-6z7ws\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wmpwk\" (UID: \"e4f239ad-a463-4488-95af-aa5739b6a334\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wmpwk" Jun 15 01:53:59.836773 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:59.836748 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e4f239ad-a463-4488-95af-aa5739b6a334-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wmpwk\" (UID: \"e4f239ad-a463-4488-95af-aa5739b6a334\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wmpwk" Jun 15 01:53:59.937777 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:59.937737 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e4f239ad-a463-4488-95af-aa5739b6a334-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wmpwk\" (UID: \"e4f239ad-a463-4488-95af-aa5739b6a334\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wmpwk" Jun 15 01:53:59.938159 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:59.937818 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e4f239ad-a463-4488-95af-aa5739b6a334-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wmpwk\" (UID: \"e4f239ad-a463-4488-95af-aa5739b6a334\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wmpwk" Jun 15 01:53:59.938159 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:59.937853 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6z7ws\" (UniqueName: \"kubernetes.io/projected/e4f239ad-a463-4488-95af-aa5739b6a334-kube-api-access-6z7ws\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wmpwk\" (UID: \"e4f239ad-a463-4488-95af-aa5739b6a334\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wmpwk" Jun 15 01:53:59.938159 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:59.938077 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e4f239ad-a463-4488-95af-aa5739b6a334-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wmpwk\" (UID: \"e4f239ad-a463-4488-95af-aa5739b6a334\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wmpwk" Jun 15 01:53:59.938159 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:59.938138 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e4f239ad-a463-4488-95af-aa5739b6a334-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wmpwk\" (UID: \"e4f239ad-a463-4488-95af-aa5739b6a334\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wmpwk" Jun 15 01:53:59.950050 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:53:59.950030 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6z7ws\" (UniqueName: \"kubernetes.io/projected/e4f239ad-a463-4488-95af-aa5739b6a334-kube-api-access-6z7ws\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wmpwk\" (UID: \"e4f239ad-a463-4488-95af-aa5739b6a334\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wmpwk" Jun 15 01:54:00.105551 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:00.105469 2578 generic.go:356] "Generic (PLEG): container finished" podID="9ccbd467-a8f2-43a8-ad76-b08698c29c54" containerID="b58c42fd706e527a3d66ebe628f9370dda7f52da9bfe44919506040dfc37be1a" exitCode=0 Jun 15 01:54:00.105711 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:00.105555 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12tpbp" event={"ID":"9ccbd467-a8f2-43a8-ad76-b08698c29c54","Type":"ContainerDied","Data":"b58c42fd706e527a3d66ebe628f9370dda7f52da9bfe44919506040dfc37be1a"} Jun 15 01:54:00.107170 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:00.107146 2578 generic.go:356] "Generic (PLEG): container finished" podID="f483827b-baf7-4a44-8bbb-8027562a973e" containerID="f1eca9d53ff3accbfb9a4d19d81ba3ddfe9b1420504dc398e2b99a79385707d9" exitCode=0 Jun 15 01:54:00.107275 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:00.107243 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v2wpk" event={"ID":"f483827b-baf7-4a44-8bbb-8027562a973e","Type":"ContainerDied","Data":"f1eca9d53ff3accbfb9a4d19d81ba3ddfe9b1420504dc398e2b99a79385707d9"} Jun 15 01:54:00.107327 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:00.107286 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v2wpk" event={"ID":"f483827b-baf7-4a44-8bbb-8027562a973e","Type":"ContainerStarted","Data":"72993ac866800c8b735b4b96c7a9ea6487b79fd78bc6a19f4b406a3e2f1c2757"} Jun 15 01:54:00.109215 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:00.109187 2578 generic.go:356] "Generic (PLEG): container finished" podID="43f8bbf1-6346-42b0-9581-547004591362" containerID="cf22995471398e54b89d95f6939c9667547c050172112e3f0edaba185d6621d8" exitCode=0 Jun 15 01:54:00.109330 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:00.109224 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p7crd" event={"ID":"43f8bbf1-6346-42b0-9581-547004591362","Type":"ContainerDied","Data":"cf22995471398e54b89d95f6939c9667547c050172112e3f0edaba185d6621d8"} Jun 15 01:54:00.129209 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:00.129187 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wmpwk" Jun 15 01:54:00.275122 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:00.275078 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wmpwk"] Jun 15 01:54:00.300911 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:54:00.300882 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4f239ad_a463_4488_95af_aa5739b6a334.slice/crio-608fdcb99b152873a7b9c89ff3cea83b00c72bc81e3dde8a283b1d45369a3ab5 WatchSource:0}: Error finding container 608fdcb99b152873a7b9c89ff3cea83b00c72bc81e3dde8a283b1d45369a3ab5: Status 404 returned error can't find the container with id 608fdcb99b152873a7b9c89ff3cea83b00c72bc81e3dde8a283b1d45369a3ab5 Jun 15 01:54:01.117964 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:01.117933 2578 generic.go:356] "Generic (PLEG): container finished" podID="43f8bbf1-6346-42b0-9581-547004591362" containerID="bd3f2c159fbec0889e84eb41993b76e5a1e1599277fb2af82c8975f9b5750aed" exitCode=0 Jun 15 01:54:01.118459 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:01.118017 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p7crd" event={"ID":"43f8bbf1-6346-42b0-9581-547004591362","Type":"ContainerDied","Data":"bd3f2c159fbec0889e84eb41993b76e5a1e1599277fb2af82c8975f9b5750aed"} Jun 15 01:54:01.119809 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:01.119785 2578 generic.go:356] "Generic (PLEG): container finished" podID="9ccbd467-a8f2-43a8-ad76-b08698c29c54" containerID="f2fc58a34acb3fe3fdcf710c37275b4e70f959b8cd7a335d2da0a2ff49c834f0" exitCode=0 Jun 15 01:54:01.119876 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:01.119850 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12tpbp" event={"ID":"9ccbd467-a8f2-43a8-ad76-b08698c29c54","Type":"ContainerDied","Data":"f2fc58a34acb3fe3fdcf710c37275b4e70f959b8cd7a335d2da0a2ff49c834f0"} Jun 15 01:54:01.121790 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:01.121769 2578 generic.go:356] "Generic (PLEG): container finished" podID="f483827b-baf7-4a44-8bbb-8027562a973e" containerID="1f60894b85a6f5747fcb071efc926b1b389fc5b7a58e6ab4eda88c3fad992c73" exitCode=0 Jun 15 01:54:01.121897 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:01.121834 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v2wpk" event={"ID":"f483827b-baf7-4a44-8bbb-8027562a973e","Type":"ContainerDied","Data":"1f60894b85a6f5747fcb071efc926b1b389fc5b7a58e6ab4eda88c3fad992c73"} Jun 15 01:54:01.123462 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:01.123440 2578 generic.go:356] "Generic (PLEG): container finished" podID="e4f239ad-a463-4488-95af-aa5739b6a334" containerID="a9730705adc7e60a06e9d045b127635075b71bb6914b3b6b1dd7665c4df5ab4f" exitCode=0 Jun 15 01:54:01.123547 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:01.123501 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wmpwk" event={"ID":"e4f239ad-a463-4488-95af-aa5739b6a334","Type":"ContainerDied","Data":"a9730705adc7e60a06e9d045b127635075b71bb6914b3b6b1dd7665c4df5ab4f"} Jun 15 01:54:01.123547 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:01.123537 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wmpwk" event={"ID":"e4f239ad-a463-4488-95af-aa5739b6a334","Type":"ContainerStarted","Data":"608fdcb99b152873a7b9c89ff3cea83b00c72bc81e3dde8a283b1d45369a3ab5"} Jun 15 01:54:02.131379 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:02.131346 2578 generic.go:356] "Generic (PLEG): container finished" podID="9ccbd467-a8f2-43a8-ad76-b08698c29c54" containerID="9e76bf4c407aa1112167a7cadc20a2ba90756d697b2451141931807c6e485942" exitCode=0 Jun 15 01:54:02.131821 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:02.131427 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12tpbp" event={"ID":"9ccbd467-a8f2-43a8-ad76-b08698c29c54","Type":"ContainerDied","Data":"9e76bf4c407aa1112167a7cadc20a2ba90756d697b2451141931807c6e485942"} Jun 15 01:54:02.133419 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:02.133392 2578 generic.go:356] "Generic (PLEG): container finished" podID="f483827b-baf7-4a44-8bbb-8027562a973e" containerID="691bad6cdce6cd6bd874290b79f00c2b8c030b3ee1658cc508e5ce863273578f" exitCode=0 Jun 15 01:54:02.133531 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:02.133470 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v2wpk" event={"ID":"f483827b-baf7-4a44-8bbb-8027562a973e","Type":"ContainerDied","Data":"691bad6cdce6cd6bd874290b79f00c2b8c030b3ee1658cc508e5ce863273578f"} Jun 15 01:54:02.135134 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:02.135109 2578 generic.go:356] "Generic (PLEG): container finished" podID="e4f239ad-a463-4488-95af-aa5739b6a334" containerID="7f752430d0f17b8161ea67d525e3abb39863e8c7572c18f9734c2192449f80c2" exitCode=0 Jun 15 01:54:02.135230 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:02.135199 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wmpwk" event={"ID":"e4f239ad-a463-4488-95af-aa5739b6a334","Type":"ContainerDied","Data":"7f752430d0f17b8161ea67d525e3abb39863e8c7572c18f9734c2192449f80c2"} Jun 15 01:54:02.293023 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:02.292999 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p7crd" Jun 15 01:54:02.356532 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:02.356504 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/43f8bbf1-6346-42b0-9581-547004591362-util\") pod \"43f8bbf1-6346-42b0-9581-547004591362\" (UID: \"43f8bbf1-6346-42b0-9581-547004591362\") " Jun 15 01:54:02.356651 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:02.356592 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4vkct\" (UniqueName: \"kubernetes.io/projected/43f8bbf1-6346-42b0-9581-547004591362-kube-api-access-4vkct\") pod \"43f8bbf1-6346-42b0-9581-547004591362\" (UID: \"43f8bbf1-6346-42b0-9581-547004591362\") " Jun 15 01:54:02.356651 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:02.356620 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/43f8bbf1-6346-42b0-9581-547004591362-bundle\") pod \"43f8bbf1-6346-42b0-9581-547004591362\" (UID: \"43f8bbf1-6346-42b0-9581-547004591362\") " Jun 15 01:54:02.357149 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:02.357120 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43f8bbf1-6346-42b0-9581-547004591362-bundle" (OuterVolumeSpecName: "bundle") pod "43f8bbf1-6346-42b0-9581-547004591362" (UID: "43f8bbf1-6346-42b0-9581-547004591362"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 01:54:02.358860 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:02.358837 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43f8bbf1-6346-42b0-9581-547004591362-kube-api-access-4vkct" (OuterVolumeSpecName: "kube-api-access-4vkct") pod "43f8bbf1-6346-42b0-9581-547004591362" (UID: "43f8bbf1-6346-42b0-9581-547004591362"). InnerVolumeSpecName "kube-api-access-4vkct". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 01:54:02.361932 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:02.361912 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43f8bbf1-6346-42b0-9581-547004591362-util" (OuterVolumeSpecName: "util") pod "43f8bbf1-6346-42b0-9581-547004591362" (UID: "43f8bbf1-6346-42b0-9581-547004591362"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 01:54:02.458176 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:02.458050 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4vkct\" (UniqueName: \"kubernetes.io/projected/43f8bbf1-6346-42b0-9581-547004591362-kube-api-access-4vkct\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:54:02.458176 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:02.458116 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/43f8bbf1-6346-42b0-9581-547004591362-bundle\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:54:02.458176 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:02.458129 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/43f8bbf1-6346-42b0-9581-547004591362-util\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:54:03.142612 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:03.142588 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p7crd" Jun 15 01:54:03.143070 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:03.142619 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p7crd" event={"ID":"43f8bbf1-6346-42b0-9581-547004591362","Type":"ContainerDied","Data":"e05b660e1a8619acdf5598cc1df72083487c7f84931f9fe0fd496c6e0f45c393"} Jun 15 01:54:03.143070 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:03.142670 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e05b660e1a8619acdf5598cc1df72083487c7f84931f9fe0fd496c6e0f45c393" Jun 15 01:54:03.144903 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:03.144880 2578 generic.go:356] "Generic (PLEG): container finished" podID="e4f239ad-a463-4488-95af-aa5739b6a334" containerID="c064a7dbaf6cb6cfba0f702b13d24f5c82f7d45c2daba6ce24bb4acd7feef0cc" exitCode=0 Jun 15 01:54:03.145024 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:03.144965 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wmpwk" event={"ID":"e4f239ad-a463-4488-95af-aa5739b6a334","Type":"ContainerDied","Data":"c064a7dbaf6cb6cfba0f702b13d24f5c82f7d45c2daba6ce24bb4acd7feef0cc"} Jun 15 01:54:03.277302 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:03.277276 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12tpbp" Jun 15 01:54:03.308294 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:03.308267 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v2wpk" Jun 15 01:54:03.366193 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:03.366148 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9ccbd467-a8f2-43a8-ad76-b08698c29c54-bundle\") pod \"9ccbd467-a8f2-43a8-ad76-b08698c29c54\" (UID: \"9ccbd467-a8f2-43a8-ad76-b08698c29c54\") " Jun 15 01:54:03.366386 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:03.366218 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f483827b-baf7-4a44-8bbb-8027562a973e-bundle\") pod \"f483827b-baf7-4a44-8bbb-8027562a973e\" (UID: \"f483827b-baf7-4a44-8bbb-8027562a973e\") " Jun 15 01:54:03.366386 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:03.366252 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f483827b-baf7-4a44-8bbb-8027562a973e-util\") pod \"f483827b-baf7-4a44-8bbb-8027562a973e\" (UID: \"f483827b-baf7-4a44-8bbb-8027562a973e\") " Jun 15 01:54:03.366386 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:03.366307 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lqkgf\" (UniqueName: \"kubernetes.io/projected/9ccbd467-a8f2-43a8-ad76-b08698c29c54-kube-api-access-lqkgf\") pod \"9ccbd467-a8f2-43a8-ad76-b08698c29c54\" (UID: \"9ccbd467-a8f2-43a8-ad76-b08698c29c54\") " Jun 15 01:54:03.366386 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:03.366332 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pxhbl\" (UniqueName: \"kubernetes.io/projected/f483827b-baf7-4a44-8bbb-8027562a973e-kube-api-access-pxhbl\") pod \"f483827b-baf7-4a44-8bbb-8027562a973e\" (UID: \"f483827b-baf7-4a44-8bbb-8027562a973e\") " Jun 15 01:54:03.366583 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:03.366402 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9ccbd467-a8f2-43a8-ad76-b08698c29c54-util\") pod \"9ccbd467-a8f2-43a8-ad76-b08698c29c54\" (UID: \"9ccbd467-a8f2-43a8-ad76-b08698c29c54\") " Jun 15 01:54:03.366865 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:03.366820 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f483827b-baf7-4a44-8bbb-8027562a973e-bundle" (OuterVolumeSpecName: "bundle") pod "f483827b-baf7-4a44-8bbb-8027562a973e" (UID: "f483827b-baf7-4a44-8bbb-8027562a973e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 01:54:03.367013 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:03.366869 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ccbd467-a8f2-43a8-ad76-b08698c29c54-bundle" (OuterVolumeSpecName: "bundle") pod "9ccbd467-a8f2-43a8-ad76-b08698c29c54" (UID: "9ccbd467-a8f2-43a8-ad76-b08698c29c54"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 01:54:03.368496 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:03.368477 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ccbd467-a8f2-43a8-ad76-b08698c29c54-kube-api-access-lqkgf" (OuterVolumeSpecName: "kube-api-access-lqkgf") pod "9ccbd467-a8f2-43a8-ad76-b08698c29c54" (UID: "9ccbd467-a8f2-43a8-ad76-b08698c29c54"). InnerVolumeSpecName "kube-api-access-lqkgf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 01:54:03.368924 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:03.368907 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f483827b-baf7-4a44-8bbb-8027562a973e-kube-api-access-pxhbl" (OuterVolumeSpecName: "kube-api-access-pxhbl") pod "f483827b-baf7-4a44-8bbb-8027562a973e" (UID: "f483827b-baf7-4a44-8bbb-8027562a973e"). InnerVolumeSpecName "kube-api-access-pxhbl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 01:54:03.371991 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:03.371951 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ccbd467-a8f2-43a8-ad76-b08698c29c54-util" (OuterVolumeSpecName: "util") pod "9ccbd467-a8f2-43a8-ad76-b08698c29c54" (UID: "9ccbd467-a8f2-43a8-ad76-b08698c29c54"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 01:54:03.372625 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:03.372591 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f483827b-baf7-4a44-8bbb-8027562a973e-util" (OuterVolumeSpecName: "util") pod "f483827b-baf7-4a44-8bbb-8027562a973e" (UID: "f483827b-baf7-4a44-8bbb-8027562a973e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 01:54:03.467819 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:03.467729 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lqkgf\" (UniqueName: \"kubernetes.io/projected/9ccbd467-a8f2-43a8-ad76-b08698c29c54-kube-api-access-lqkgf\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:54:03.467819 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:03.467767 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pxhbl\" (UniqueName: \"kubernetes.io/projected/f483827b-baf7-4a44-8bbb-8027562a973e-kube-api-access-pxhbl\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:54:03.467819 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:03.467777 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9ccbd467-a8f2-43a8-ad76-b08698c29c54-util\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:54:03.467819 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:03.467786 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9ccbd467-a8f2-43a8-ad76-b08698c29c54-bundle\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:54:03.467819 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:03.467796 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f483827b-baf7-4a44-8bbb-8027562a973e-bundle\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:54:03.467819 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:03.467803 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f483827b-baf7-4a44-8bbb-8027562a973e-util\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:54:04.152659 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:04.152625 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12tpbp" Jun 15 01:54:04.153055 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:04.152616 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12tpbp" event={"ID":"9ccbd467-a8f2-43a8-ad76-b08698c29c54","Type":"ContainerDied","Data":"45bb4aff09399c36331597edad05407eb343b2c546013424f7a8f5cac0ff4381"} Jun 15 01:54:04.153055 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:04.152751 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="45bb4aff09399c36331597edad05407eb343b2c546013424f7a8f5cac0ff4381" Jun 15 01:54:04.154624 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:04.154607 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v2wpk" Jun 15 01:54:04.154690 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:04.154607 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v2wpk" event={"ID":"f483827b-baf7-4a44-8bbb-8027562a973e","Type":"ContainerDied","Data":"72993ac866800c8b735b4b96c7a9ea6487b79fd78bc6a19f4b406a3e2f1c2757"} Jun 15 01:54:04.154731 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:04.154709 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="72993ac866800c8b735b4b96c7a9ea6487b79fd78bc6a19f4b406a3e2f1c2757" Jun 15 01:54:04.304074 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:04.304048 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wmpwk" Jun 15 01:54:04.376724 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:04.376691 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e4f239ad-a463-4488-95af-aa5739b6a334-bundle\") pod \"e4f239ad-a463-4488-95af-aa5739b6a334\" (UID: \"e4f239ad-a463-4488-95af-aa5739b6a334\") " Jun 15 01:54:04.376896 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:04.376755 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6z7ws\" (UniqueName: \"kubernetes.io/projected/e4f239ad-a463-4488-95af-aa5739b6a334-kube-api-access-6z7ws\") pod \"e4f239ad-a463-4488-95af-aa5739b6a334\" (UID: \"e4f239ad-a463-4488-95af-aa5739b6a334\") " Jun 15 01:54:04.376896 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:04.376795 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e4f239ad-a463-4488-95af-aa5739b6a334-util\") pod \"e4f239ad-a463-4488-95af-aa5739b6a334\" (UID: \"e4f239ad-a463-4488-95af-aa5739b6a334\") " Jun 15 01:54:04.377345 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:04.377318 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4f239ad-a463-4488-95af-aa5739b6a334-bundle" (OuterVolumeSpecName: "bundle") pod "e4f239ad-a463-4488-95af-aa5739b6a334" (UID: "e4f239ad-a463-4488-95af-aa5739b6a334"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 01:54:04.379103 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:04.379057 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4f239ad-a463-4488-95af-aa5739b6a334-kube-api-access-6z7ws" (OuterVolumeSpecName: "kube-api-access-6z7ws") pod "e4f239ad-a463-4488-95af-aa5739b6a334" (UID: "e4f239ad-a463-4488-95af-aa5739b6a334"). InnerVolumeSpecName "kube-api-access-6z7ws". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 01:54:04.382074 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:04.382039 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4f239ad-a463-4488-95af-aa5739b6a334-util" (OuterVolumeSpecName: "util") pod "e4f239ad-a463-4488-95af-aa5739b6a334" (UID: "e4f239ad-a463-4488-95af-aa5739b6a334"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 01:54:04.477813 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:04.477776 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6z7ws\" (UniqueName: \"kubernetes.io/projected/e4f239ad-a463-4488-95af-aa5739b6a334-kube-api-access-6z7ws\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:54:04.477813 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:04.477807 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e4f239ad-a463-4488-95af-aa5739b6a334-util\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:54:04.477813 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:04.477819 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e4f239ad-a463-4488-95af-aa5739b6a334-bundle\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:54:05.162204 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:05.162173 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wmpwk" event={"ID":"e4f239ad-a463-4488-95af-aa5739b6a334","Type":"ContainerDied","Data":"608fdcb99b152873a7b9c89ff3cea83b00c72bc81e3dde8a283b1d45369a3ab5"} Jun 15 01:54:05.162204 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:05.162194 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wmpwk" Jun 15 01:54:05.162720 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:05.162217 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="608fdcb99b152873a7b9c89ff3cea83b00c72bc81e3dde8a283b1d45369a3ab5" Jun 15 01:54:07.941673 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:07.941637 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6798ff8d9c-bx88t"] Jun 15 01:54:07.942168 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:07.941993 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43f8bbf1-6346-42b0-9581-547004591362" containerName="pull" Jun 15 01:54:07.942168 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:07.942004 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="43f8bbf1-6346-42b0-9581-547004591362" containerName="pull" Jun 15 01:54:07.942168 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:07.942012 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4f239ad-a463-4488-95af-aa5739b6a334" containerName="extract" Jun 15 01:54:07.942168 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:07.942018 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4f239ad-a463-4488-95af-aa5739b6a334" containerName="extract" Jun 15 01:54:07.942168 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:07.942061 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f483827b-baf7-4a44-8bbb-8027562a973e" containerName="util" Jun 15 01:54:07.942168 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:07.942067 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f483827b-baf7-4a44-8bbb-8027562a973e" containerName="util" Jun 15 01:54:07.942168 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:07.942075 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f483827b-baf7-4a44-8bbb-8027562a973e" containerName="pull" Jun 15 01:54:07.942168 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:07.942080 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f483827b-baf7-4a44-8bbb-8027562a973e" containerName="pull" Jun 15 01:54:07.942168 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:07.942107 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f483827b-baf7-4a44-8bbb-8027562a973e" containerName="extract" Jun 15 01:54:07.942168 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:07.942116 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f483827b-baf7-4a44-8bbb-8027562a973e" containerName="extract" Jun 15 01:54:07.942168 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:07.942123 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43f8bbf1-6346-42b0-9581-547004591362" containerName="extract" Jun 15 01:54:07.942168 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:07.942128 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="43f8bbf1-6346-42b0-9581-547004591362" containerName="extract" Jun 15 01:54:07.942168 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:07.942141 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ccbd467-a8f2-43a8-ad76-b08698c29c54" containerName="util" Jun 15 01:54:07.942168 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:07.942147 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ccbd467-a8f2-43a8-ad76-b08698c29c54" containerName="util" Jun 15 01:54:07.942168 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:07.942153 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4f239ad-a463-4488-95af-aa5739b6a334" containerName="util" Jun 15 01:54:07.942168 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:07.942158 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4f239ad-a463-4488-95af-aa5739b6a334" containerName="util" Jun 15 01:54:07.942168 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:07.942164 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4f239ad-a463-4488-95af-aa5739b6a334" containerName="pull" Jun 15 01:54:07.942168 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:07.942168 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4f239ad-a463-4488-95af-aa5739b6a334" containerName="pull" Jun 15 01:54:07.942168 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:07.942179 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43f8bbf1-6346-42b0-9581-547004591362" containerName="util" Jun 15 01:54:07.942836 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:07.942184 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="43f8bbf1-6346-42b0-9581-547004591362" containerName="util" Jun 15 01:54:07.942836 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:07.942189 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ccbd467-a8f2-43a8-ad76-b08698c29c54" containerName="pull" Jun 15 01:54:07.942836 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:07.942194 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ccbd467-a8f2-43a8-ad76-b08698c29c54" containerName="pull" Jun 15 01:54:07.942836 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:07.942199 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ccbd467-a8f2-43a8-ad76-b08698c29c54" containerName="extract" Jun 15 01:54:07.942836 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:07.942205 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ccbd467-a8f2-43a8-ad76-b08698c29c54" containerName="extract" Jun 15 01:54:07.942836 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:07.942263 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4f239ad-a463-4488-95af-aa5739b6a334" containerName="extract" Jun 15 01:54:07.942836 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:07.942270 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="43f8bbf1-6346-42b0-9581-547004591362" containerName="extract" Jun 15 01:54:07.942836 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:07.942277 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ccbd467-a8f2-43a8-ad76-b08698c29c54" containerName="extract" Jun 15 01:54:07.942836 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:07.942283 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="f483827b-baf7-4a44-8bbb-8027562a973e" containerName="extract" Jun 15 01:54:07.945342 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:07.945325 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6798ff8d9c-bx88t" Jun 15 01:54:07.948003 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:07.947982 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jun 15 01:54:07.949305 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:07.949140 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jun 15 01:54:07.949305 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:07.949159 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jun 15 01:54:07.949305 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:07.949229 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jun 15 01:54:07.949305 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:07.949292 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jun 15 01:54:07.949305 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:07.949299 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-s92rf\"" Jun 15 01:54:07.949564 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:07.949370 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jun 15 01:54:07.949681 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:07.949665 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jun 15 01:54:07.954724 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:07.954705 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jun 15 01:54:07.956709 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:07.956686 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6798ff8d9c-bx88t"] Jun 15 01:54:08.009158 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:08.009116 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8bec6edd-4845-4d89-8532-e95b73315323-console-serving-cert\") pod \"console-6798ff8d9c-bx88t\" (UID: \"8bec6edd-4845-4d89-8532-e95b73315323\") " pod="openshift-console/console-6798ff8d9c-bx88t" Jun 15 01:54:08.009158 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:08.009162 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bec6edd-4845-4d89-8532-e95b73315323-trusted-ca-bundle\") pod \"console-6798ff8d9c-bx88t\" (UID: \"8bec6edd-4845-4d89-8532-e95b73315323\") " pod="openshift-console/console-6798ff8d9c-bx88t" Jun 15 01:54:08.009393 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:08.009188 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlwhl\" (UniqueName: \"kubernetes.io/projected/8bec6edd-4845-4d89-8532-e95b73315323-kube-api-access-zlwhl\") pod \"console-6798ff8d9c-bx88t\" (UID: \"8bec6edd-4845-4d89-8532-e95b73315323\") " pod="openshift-console/console-6798ff8d9c-bx88t" Jun 15 01:54:08.009393 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:08.009245 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8bec6edd-4845-4d89-8532-e95b73315323-console-config\") pod \"console-6798ff8d9c-bx88t\" (UID: \"8bec6edd-4845-4d89-8532-e95b73315323\") " pod="openshift-console/console-6798ff8d9c-bx88t" Jun 15 01:54:08.009393 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:08.009276 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8bec6edd-4845-4d89-8532-e95b73315323-service-ca\") pod \"console-6798ff8d9c-bx88t\" (UID: \"8bec6edd-4845-4d89-8532-e95b73315323\") " pod="openshift-console/console-6798ff8d9c-bx88t" Jun 15 01:54:08.009393 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:08.009334 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8bec6edd-4845-4d89-8532-e95b73315323-console-oauth-config\") pod \"console-6798ff8d9c-bx88t\" (UID: \"8bec6edd-4845-4d89-8532-e95b73315323\") " pod="openshift-console/console-6798ff8d9c-bx88t" Jun 15 01:54:08.009393 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:08.009381 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8bec6edd-4845-4d89-8532-e95b73315323-oauth-serving-cert\") pod \"console-6798ff8d9c-bx88t\" (UID: \"8bec6edd-4845-4d89-8532-e95b73315323\") " pod="openshift-console/console-6798ff8d9c-bx88t" Jun 15 01:54:08.109997 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:08.109961 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8bec6edd-4845-4d89-8532-e95b73315323-console-serving-cert\") pod \"console-6798ff8d9c-bx88t\" (UID: \"8bec6edd-4845-4d89-8532-e95b73315323\") " pod="openshift-console/console-6798ff8d9c-bx88t" Jun 15 01:54:08.109997 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:08.110001 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bec6edd-4845-4d89-8532-e95b73315323-trusted-ca-bundle\") pod \"console-6798ff8d9c-bx88t\" (UID: \"8bec6edd-4845-4d89-8532-e95b73315323\") " pod="openshift-console/console-6798ff8d9c-bx88t" Jun 15 01:54:08.110308 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:08.110022 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlwhl\" (UniqueName: \"kubernetes.io/projected/8bec6edd-4845-4d89-8532-e95b73315323-kube-api-access-zlwhl\") pod \"console-6798ff8d9c-bx88t\" (UID: \"8bec6edd-4845-4d89-8532-e95b73315323\") " pod="openshift-console/console-6798ff8d9c-bx88t" Jun 15 01:54:08.110308 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:08.110039 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8bec6edd-4845-4d89-8532-e95b73315323-console-config\") pod \"console-6798ff8d9c-bx88t\" (UID: \"8bec6edd-4845-4d89-8532-e95b73315323\") " pod="openshift-console/console-6798ff8d9c-bx88t" Jun 15 01:54:08.110308 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:08.110151 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8bec6edd-4845-4d89-8532-e95b73315323-service-ca\") pod \"console-6798ff8d9c-bx88t\" (UID: \"8bec6edd-4845-4d89-8532-e95b73315323\") " pod="openshift-console/console-6798ff8d9c-bx88t" Jun 15 01:54:08.110308 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:08.110215 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8bec6edd-4845-4d89-8532-e95b73315323-console-oauth-config\") pod \"console-6798ff8d9c-bx88t\" (UID: \"8bec6edd-4845-4d89-8532-e95b73315323\") " pod="openshift-console/console-6798ff8d9c-bx88t" Jun 15 01:54:08.110308 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:08.110269 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8bec6edd-4845-4d89-8532-e95b73315323-oauth-serving-cert\") pod \"console-6798ff8d9c-bx88t\" (UID: \"8bec6edd-4845-4d89-8532-e95b73315323\") " pod="openshift-console/console-6798ff8d9c-bx88t" Jun 15 01:54:08.110899 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:08.110878 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8bec6edd-4845-4d89-8532-e95b73315323-service-ca\") pod \"console-6798ff8d9c-bx88t\" (UID: \"8bec6edd-4845-4d89-8532-e95b73315323\") " pod="openshift-console/console-6798ff8d9c-bx88t" Jun 15 01:54:08.110987 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:08.110967 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8bec6edd-4845-4d89-8532-e95b73315323-oauth-serving-cert\") pod \"console-6798ff8d9c-bx88t\" (UID: \"8bec6edd-4845-4d89-8532-e95b73315323\") " pod="openshift-console/console-6798ff8d9c-bx88t" Jun 15 01:54:08.110987 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:08.110977 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8bec6edd-4845-4d89-8532-e95b73315323-console-config\") pod \"console-6798ff8d9c-bx88t\" (UID: \"8bec6edd-4845-4d89-8532-e95b73315323\") " pod="openshift-console/console-6798ff8d9c-bx88t" Jun 15 01:54:08.111272 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:08.111249 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bec6edd-4845-4d89-8532-e95b73315323-trusted-ca-bundle\") pod \"console-6798ff8d9c-bx88t\" (UID: \"8bec6edd-4845-4d89-8532-e95b73315323\") " pod="openshift-console/console-6798ff8d9c-bx88t" Jun 15 01:54:08.112756 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:08.112734 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8bec6edd-4845-4d89-8532-e95b73315323-console-oauth-config\") pod \"console-6798ff8d9c-bx88t\" (UID: \"8bec6edd-4845-4d89-8532-e95b73315323\") " pod="openshift-console/console-6798ff8d9c-bx88t" Jun 15 01:54:08.112826 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:08.112810 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8bec6edd-4845-4d89-8532-e95b73315323-console-serving-cert\") pod \"console-6798ff8d9c-bx88t\" (UID: \"8bec6edd-4845-4d89-8532-e95b73315323\") " pod="openshift-console/console-6798ff8d9c-bx88t" Jun 15 01:54:08.118373 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:08.118346 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlwhl\" (UniqueName: \"kubernetes.io/projected/8bec6edd-4845-4d89-8532-e95b73315323-kube-api-access-zlwhl\") pod \"console-6798ff8d9c-bx88t\" (UID: \"8bec6edd-4845-4d89-8532-e95b73315323\") " pod="openshift-console/console-6798ff8d9c-bx88t" Jun 15 01:54:08.255895 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:08.255812 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6798ff8d9c-bx88t" Jun 15 01:54:08.390010 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:08.389977 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6798ff8d9c-bx88t"] Jun 15 01:54:08.391953 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:54:08.391923 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bec6edd_4845_4d89_8532_e95b73315323.slice/crio-d4566feebc8711d5c50fd1413474ccc072faf6b6d8ef2a994edf6448e68b4a6c WatchSource:0}: Error finding container d4566feebc8711d5c50fd1413474ccc072faf6b6d8ef2a994edf6448e68b4a6c: Status 404 returned error can't find the container with id d4566feebc8711d5c50fd1413474ccc072faf6b6d8ef2a994edf6448e68b4a6c Jun 15 01:54:09.189733 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:09.189699 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6798ff8d9c-bx88t" event={"ID":"8bec6edd-4845-4d89-8532-e95b73315323","Type":"ContainerStarted","Data":"5f931ed2849c9e29c34fb892ecd036651a95770a4d88595cbd3cd706d2f99c24"} Jun 15 01:54:09.190136 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:09.189774 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6798ff8d9c-bx88t" event={"ID":"8bec6edd-4845-4d89-8532-e95b73315323","Type":"ContainerStarted","Data":"d4566feebc8711d5c50fd1413474ccc072faf6b6d8ef2a994edf6448e68b4a6c"} Jun 15 01:54:09.208434 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:09.208389 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6798ff8d9c-bx88t" podStartSLOduration=2.208375923 podStartE2EDuration="2.208375923s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 01:54:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 01:54:09.206419471 +0000 UTC m=+572.920145444" watchObservedRunningTime="2026-06-15 01:54:09.208375923 +0000 UTC m=+572.922101894" Jun 15 01:54:18.256050 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:18.256012 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6798ff8d9c-bx88t" Jun 15 01:54:18.256451 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:18.256151 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6798ff8d9c-bx88t" Jun 15 01:54:18.262028 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:18.261999 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6798ff8d9c-bx88t" Jun 15 01:54:19.248303 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:19.248273 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6798ff8d9c-bx88t" Jun 15 01:54:25.343827 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:25.343786 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qrsx2"] Jun 15 01:54:25.347299 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:25.347282 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qrsx2" Jun 15 01:54:25.350057 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:25.350030 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 15 01:54:25.350214 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:25.350078 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-bfzfh\"" Jun 15 01:54:25.350214 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:25.350136 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 15 01:54:25.370280 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:25.370250 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qrsx2"] Jun 15 01:54:25.463278 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:25.463239 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fxnl\" (UniqueName: \"kubernetes.io/projected/0f0b4be3-e5e0-4eb8-803f-51f4a9820023-kube-api-access-5fxnl\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qrsx2\" (UID: \"0f0b4be3-e5e0-4eb8-803f-51f4a9820023\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qrsx2" Jun 15 01:54:25.463452 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:25.463310 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0f0b4be3-e5e0-4eb8-803f-51f4a9820023-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qrsx2\" (UID: \"0f0b4be3-e5e0-4eb8-803f-51f4a9820023\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qrsx2" Jun 15 01:54:25.564613 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:25.564574 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fxnl\" (UniqueName: \"kubernetes.io/projected/0f0b4be3-e5e0-4eb8-803f-51f4a9820023-kube-api-access-5fxnl\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qrsx2\" (UID: \"0f0b4be3-e5e0-4eb8-803f-51f4a9820023\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qrsx2" Jun 15 01:54:25.564810 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:25.564639 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0f0b4be3-e5e0-4eb8-803f-51f4a9820023-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qrsx2\" (UID: \"0f0b4be3-e5e0-4eb8-803f-51f4a9820023\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qrsx2" Jun 15 01:54:25.564981 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:25.564963 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0f0b4be3-e5e0-4eb8-803f-51f4a9820023-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qrsx2\" (UID: \"0f0b4be3-e5e0-4eb8-803f-51f4a9820023\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qrsx2" Jun 15 01:54:25.574509 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:25.574484 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fxnl\" (UniqueName: \"kubernetes.io/projected/0f0b4be3-e5e0-4eb8-803f-51f4a9820023-kube-api-access-5fxnl\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qrsx2\" (UID: \"0f0b4be3-e5e0-4eb8-803f-51f4a9820023\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qrsx2" Jun 15 01:54:25.658260 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:25.658226 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qrsx2" Jun 15 01:54:25.802738 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:25.802706 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qrsx2"] Jun 15 01:54:25.803845 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:54:25.803823 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f0b4be3_e5e0_4eb8_803f_51f4a9820023.slice/crio-6b071c831ed8563c8b5728b43338f515292d78a082eba7f0e3641223b2095500 WatchSource:0}: Error finding container 6b071c831ed8563c8b5728b43338f515292d78a082eba7f0e3641223b2095500: Status 404 returned error can't find the container with id 6b071c831ed8563c8b5728b43338f515292d78a082eba7f0e3641223b2095500 Jun 15 01:54:26.282682 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:26.282647 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qrsx2" event={"ID":"0f0b4be3-e5e0-4eb8-803f-51f4a9820023","Type":"ContainerStarted","Data":"6b071c831ed8563c8b5728b43338f515292d78a082eba7f0e3641223b2095500"} Jun 15 01:54:34.335931 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:34.335828 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qrsx2" event={"ID":"0f0b4be3-e5e0-4eb8-803f-51f4a9820023","Type":"ContainerStarted","Data":"f0cb27369012c041cd15e30f07c9fae3f54e364a9202e809d059405f2372f8ee"} Jun 15 01:54:34.336302 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:34.336009 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qrsx2" Jun 15 01:54:34.356232 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:34.356176 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qrsx2" podStartSLOduration=1.145101494 podStartE2EDuration="9.356158244s" totalImagesPullingTime="8.21105675s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 01:54:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 01:54:34.353465599 +0000 UTC m=+598.067191571" watchObservedRunningTime="2026-06-15 01:54:34.356158244 +0000 UTC m=+598.069884216" Jun 15 01:54:36.765008 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:36.764980 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cf286_64f557b0-78d8-45a6-9f54-756417231d6c/ovn-acl-logging/0.log" Jun 15 01:54:36.767408 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:36.767386 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cf286_64f557b0-78d8-45a6-9f54-756417231d6c/ovn-acl-logging/0.log" Jun 15 01:54:45.343888 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:45.343856 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qrsx2" Jun 15 01:54:46.376259 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:46.376221 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7x7tv"] Jun 15 01:54:46.379583 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:46.379563 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7x7tv" Jun 15 01:54:46.403979 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:46.403953 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7x7tv"] Jun 15 01:54:46.447719 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:46.447673 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c9571510-e31d-41f7-b74f-de47ff9fd969-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-7x7tv\" (UID: \"c9571510-e31d-41f7-b74f-de47ff9fd969\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7x7tv" Jun 15 01:54:46.447900 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:46.447762 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdx9n\" (UniqueName: \"kubernetes.io/projected/c9571510-e31d-41f7-b74f-de47ff9fd969-kube-api-access-hdx9n\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-7x7tv\" (UID: \"c9571510-e31d-41f7-b74f-de47ff9fd969\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7x7tv" Jun 15 01:54:46.548531 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:46.548487 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c9571510-e31d-41f7-b74f-de47ff9fd969-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-7x7tv\" (UID: \"c9571510-e31d-41f7-b74f-de47ff9fd969\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7x7tv" Jun 15 01:54:46.548720 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:46.548578 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdx9n\" (UniqueName: \"kubernetes.io/projected/c9571510-e31d-41f7-b74f-de47ff9fd969-kube-api-access-hdx9n\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-7x7tv\" (UID: \"c9571510-e31d-41f7-b74f-de47ff9fd969\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7x7tv" Jun 15 01:54:46.548891 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:46.548869 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c9571510-e31d-41f7-b74f-de47ff9fd969-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-7x7tv\" (UID: \"c9571510-e31d-41f7-b74f-de47ff9fd969\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7x7tv" Jun 15 01:54:46.557371 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:46.557344 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdx9n\" (UniqueName: \"kubernetes.io/projected/c9571510-e31d-41f7-b74f-de47ff9fd969-kube-api-access-hdx9n\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-7x7tv\" (UID: \"c9571510-e31d-41f7-b74f-de47ff9fd969\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7x7tv" Jun 15 01:54:46.689468 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:46.689375 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7x7tv" Jun 15 01:54:46.834215 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:46.834183 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7x7tv"] Jun 15 01:54:46.836342 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:54:46.836317 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9571510_e31d_41f7_b74f_de47ff9fd969.slice/crio-a06b5bdef5638d8aa1d359951ab2f330f0210733d4a8ffc42750a5f49411d8a5 WatchSource:0}: Error finding container a06b5bdef5638d8aa1d359951ab2f330f0210733d4a8ffc42750a5f49411d8a5: Status 404 returned error can't find the container with id a06b5bdef5638d8aa1d359951ab2f330f0210733d4a8ffc42750a5f49411d8a5 Jun 15 01:54:46.997235 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:46.997152 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7x7tv"] Jun 15 01:54:47.003947 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:47.003917 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7x7tv"] Jun 15 01:54:47.016712 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:47.016685 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qrsx2"] Jun 15 01:54:47.016941 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:47.016917 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qrsx2" podUID="0f0b4be3-e5e0-4eb8-803f-51f4a9820023" containerName="manager" containerID="cri-o://f0cb27369012c041cd15e30f07c9fae3f54e364a9202e809d059405f2372f8ee" gracePeriod=2 Jun 15 01:54:47.027413 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:47.027381 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qrsx2"] Jun 15 01:54:47.030054 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:47.029990 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qsxqx"] Jun 15 01:54:47.030512 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:47.030490 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9571510-e31d-41f7-b74f-de47ff9fd969" containerName="manager" Jun 15 01:54:47.030512 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:47.030514 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9571510-e31d-41f7-b74f-de47ff9fd969" containerName="manager" Jun 15 01:54:47.030655 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:47.030537 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f0b4be3-e5e0-4eb8-803f-51f4a9820023" containerName="manager" Jun 15 01:54:47.030655 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:47.030546 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f0b4be3-e5e0-4eb8-803f-51f4a9820023" containerName="manager" Jun 15 01:54:47.030655 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:47.030629 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9571510-e31d-41f7-b74f-de47ff9fd969" containerName="manager" Jun 15 01:54:47.030655 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:47.030640 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f0b4be3-e5e0-4eb8-803f-51f4a9820023" containerName="manager" Jun 15 01:54:47.033595 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:47.033576 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qsxqx" Jun 15 01:54:47.052663 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:47.052630 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qsxqx"] Jun 15 01:54:47.054056 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:47.054031 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqdcn\" (UniqueName: \"kubernetes.io/projected/86a7cf17-a6d2-4416-97fc-b55d693f2a32-kube-api-access-zqdcn\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-qsxqx\" (UID: \"86a7cf17-a6d2-4416-97fc-b55d693f2a32\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qsxqx" Jun 15 01:54:47.054200 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:47.054078 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/86a7cf17-a6d2-4416-97fc-b55d693f2a32-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-qsxqx\" (UID: \"86a7cf17-a6d2-4416-97fc-b55d693f2a32\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qsxqx" Jun 15 01:54:47.070351 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:54:47.070310 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qrsx2\" is forbidden: User \"system:node:ip-10-0-143-196.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-196.ec2.internal' and this object" podUID="0f0b4be3-e5e0-4eb8-803f-51f4a9820023" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qrsx2" Jun 15 01:54:47.071893 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:47.071868 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9j5lx"] Jun 15 01:54:47.076146 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:47.076121 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9j5lx" Jun 15 01:54:47.095101 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:47.095054 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9j5lx"] Jun 15 01:54:47.129998 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:54:47.129934 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qrsx2\" is forbidden: User \"system:node:ip-10-0-143-196.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-196.ec2.internal' and this object" podUID="0f0b4be3-e5e0-4eb8-803f-51f4a9820023" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qrsx2" Jun 15 01:54:47.155216 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:47.155183 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqdcn\" (UniqueName: \"kubernetes.io/projected/86a7cf17-a6d2-4416-97fc-b55d693f2a32-kube-api-access-zqdcn\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-qsxqx\" (UID: \"86a7cf17-a6d2-4416-97fc-b55d693f2a32\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qsxqx" Jun 15 01:54:47.155216 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:47.155223 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvpnh\" (UniqueName: \"kubernetes.io/projected/d1e69d47-badd-48e7-9554-27e5a81201ca-kube-api-access-xvpnh\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9j5lx\" (UID: \"d1e69d47-badd-48e7-9554-27e5a81201ca\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9j5lx" Jun 15 01:54:47.155450 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:47.155273 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/86a7cf17-a6d2-4416-97fc-b55d693f2a32-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-qsxqx\" (UID: \"86a7cf17-a6d2-4416-97fc-b55d693f2a32\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qsxqx" Jun 15 01:54:47.155450 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:47.155291 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d1e69d47-badd-48e7-9554-27e5a81201ca-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9j5lx\" (UID: \"d1e69d47-badd-48e7-9554-27e5a81201ca\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9j5lx" Jun 15 01:54:47.155652 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:47.155631 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/86a7cf17-a6d2-4416-97fc-b55d693f2a32-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-qsxqx\" (UID: \"86a7cf17-a6d2-4416-97fc-b55d693f2a32\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qsxqx" Jun 15 01:54:47.167028 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:47.166985 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqdcn\" (UniqueName: \"kubernetes.io/projected/86a7cf17-a6d2-4416-97fc-b55d693f2a32-kube-api-access-zqdcn\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-qsxqx\" (UID: \"86a7cf17-a6d2-4416-97fc-b55d693f2a32\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qsxqx" Jun 15 01:54:47.247380 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:47.247323 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qrsx2" Jun 15 01:54:47.250223 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:54:47.250184 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qrsx2\" is forbidden: User \"system:node:ip-10-0-143-196.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-196.ec2.internal' and this object" podUID="0f0b4be3-e5e0-4eb8-803f-51f4a9820023" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qrsx2" Jun 15 01:54:47.256580 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:47.256563 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5fxnl\" (UniqueName: \"kubernetes.io/projected/0f0b4be3-e5e0-4eb8-803f-51f4a9820023-kube-api-access-5fxnl\") pod \"0f0b4be3-e5e0-4eb8-803f-51f4a9820023\" (UID: \"0f0b4be3-e5e0-4eb8-803f-51f4a9820023\") " Jun 15 01:54:47.256656 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:47.256631 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0f0b4be3-e5e0-4eb8-803f-51f4a9820023-extensions-socket-volume\") pod \"0f0b4be3-e5e0-4eb8-803f-51f4a9820023\" (UID: \"0f0b4be3-e5e0-4eb8-803f-51f4a9820023\") " Jun 15 01:54:47.256746 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:47.256733 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvpnh\" (UniqueName: \"kubernetes.io/projected/d1e69d47-badd-48e7-9554-27e5a81201ca-kube-api-access-xvpnh\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9j5lx\" (UID: \"d1e69d47-badd-48e7-9554-27e5a81201ca\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9j5lx" Jun 15 01:54:47.256797 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:47.256774 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d1e69d47-badd-48e7-9554-27e5a81201ca-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9j5lx\" (UID: \"d1e69d47-badd-48e7-9554-27e5a81201ca\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9j5lx" Jun 15 01:54:47.257022 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:47.256997 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f0b4be3-e5e0-4eb8-803f-51f4a9820023-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "0f0b4be3-e5e0-4eb8-803f-51f4a9820023" (UID: "0f0b4be3-e5e0-4eb8-803f-51f4a9820023"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 01:54:47.257130 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:47.257116 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d1e69d47-badd-48e7-9554-27e5a81201ca-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9j5lx\" (UID: \"d1e69d47-badd-48e7-9554-27e5a81201ca\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9j5lx" Jun 15 01:54:47.258854 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:47.258833 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f0b4be3-e5e0-4eb8-803f-51f4a9820023-kube-api-access-5fxnl" (OuterVolumeSpecName: "kube-api-access-5fxnl") pod "0f0b4be3-e5e0-4eb8-803f-51f4a9820023" (UID: "0f0b4be3-e5e0-4eb8-803f-51f4a9820023"). InnerVolumeSpecName "kube-api-access-5fxnl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 01:54:47.268759 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:47.268732 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvpnh\" (UniqueName: \"kubernetes.io/projected/d1e69d47-badd-48e7-9554-27e5a81201ca-kube-api-access-xvpnh\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9j5lx\" (UID: \"d1e69d47-badd-48e7-9554-27e5a81201ca\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9j5lx" Jun 15 01:54:47.357520 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:47.357485 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5fxnl\" (UniqueName: \"kubernetes.io/projected/0f0b4be3-e5e0-4eb8-803f-51f4a9820023-kube-api-access-5fxnl\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:54:47.357520 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:47.357514 2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0f0b4be3-e5e0-4eb8-803f-51f4a9820023-extensions-socket-volume\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:54:47.389843 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:47.389805 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qsxqx" Jun 15 01:54:47.398902 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:47.398878 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9j5lx" Jun 15 01:54:47.413302 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:54:47.413265 2578 kuberuntime_manager.go:649] "Missing actuated resource record" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7x7tv" container="manager" Jun 15 01:54:47.415010 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:47.414975 2578 generic.go:356] "Generic (PLEG): container finished" podID="0f0b4be3-e5e0-4eb8-803f-51f4a9820023" containerID="f0cb27369012c041cd15e30f07c9fae3f54e364a9202e809d059405f2372f8ee" exitCode=0 Jun 15 01:54:47.415175 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:47.415039 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qrsx2" Jun 15 01:54:47.415175 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:47.415059 2578 scope.go:117] "RemoveContainer" containerID="f0cb27369012c041cd15e30f07c9fae3f54e364a9202e809d059405f2372f8ee" Jun 15 01:54:47.416074 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:54:47.416039 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-7x7tv\" is forbidden: User \"system:node:ip-10-0-143-196.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-196.ec2.internal' and this object" podUID="c9571510-e31d-41f7-b74f-de47ff9fd969" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7x7tv" Jun 15 01:54:47.418557 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:54:47.418521 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qrsx2\" is forbidden: User \"system:node:ip-10-0-143-196.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-196.ec2.internal' and this object" podUID="0f0b4be3-e5e0-4eb8-803f-51f4a9820023" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qrsx2" Jun 15 01:54:47.421226 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:54:47.421166 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-7x7tv\" is forbidden: User \"system:node:ip-10-0-143-196.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-196.ec2.internal' and this object" podUID="c9571510-e31d-41f7-b74f-de47ff9fd969" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7x7tv" Jun 15 01:54:47.423545 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:54:47.423522 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qrsx2\" is forbidden: User \"system:node:ip-10-0-143-196.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-196.ec2.internal' and this object" podUID="0f0b4be3-e5e0-4eb8-803f-51f4a9820023" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qrsx2" Jun 15 01:54:47.427892 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:54:47.427865 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qrsx2\" is forbidden: User \"system:node:ip-10-0-143-196.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-196.ec2.internal' and this object" podUID="0f0b4be3-e5e0-4eb8-803f-51f4a9820023" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qrsx2" Jun 15 01:54:47.428276 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:47.428126 2578 scope.go:117] "RemoveContainer" containerID="f0cb27369012c041cd15e30f07c9fae3f54e364a9202e809d059405f2372f8ee" Jun 15 01:54:47.430970 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:54:47.430487 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-7x7tv\" is forbidden: User \"system:node:ip-10-0-143-196.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-196.ec2.internal' and this object" podUID="c9571510-e31d-41f7-b74f-de47ff9fd969" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7x7tv" Jun 15 01:54:47.430970 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:54:47.430579 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0cb27369012c041cd15e30f07c9fae3f54e364a9202e809d059405f2372f8ee\": container with ID starting with f0cb27369012c041cd15e30f07c9fae3f54e364a9202e809d059405f2372f8ee not found: ID does not exist" containerID="f0cb27369012c041cd15e30f07c9fae3f54e364a9202e809d059405f2372f8ee" Jun 15 01:54:47.430970 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:47.430609 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0cb27369012c041cd15e30f07c9fae3f54e364a9202e809d059405f2372f8ee"} err="failed to get container status \"f0cb27369012c041cd15e30f07c9fae3f54e364a9202e809d059405f2372f8ee\": rpc error: code = NotFound desc = could not find container \"f0cb27369012c041cd15e30f07c9fae3f54e364a9202e809d059405f2372f8ee\": container with ID starting with f0cb27369012c041cd15e30f07c9fae3f54e364a9202e809d059405f2372f8ee not found: ID does not exist" Jun 15 01:54:47.544254 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:47.544225 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qsxqx"] Jun 15 01:54:47.546607 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:54:47.546578 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86a7cf17_a6d2_4416_97fc_b55d693f2a32.slice/crio-f14c4d6babe1e12d502cdf7382af9c344ac8541c0a13d8fa4c386a7aee640bde WatchSource:0}: Error finding container f14c4d6babe1e12d502cdf7382af9c344ac8541c0a13d8fa4c386a7aee640bde: Status 404 returned error can't find the container with id f14c4d6babe1e12d502cdf7382af9c344ac8541c0a13d8fa4c386a7aee640bde Jun 15 01:54:47.573754 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:47.573727 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9j5lx"] Jun 15 01:54:47.575532 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:54:47.575492 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1e69d47_badd_48e7_9554_27e5a81201ca.slice/crio-bbcbcf099456f57321aca479f000a38a99bea36ba5685ee4d013ae8ff799fdd6 WatchSource:0}: Error finding container bbcbcf099456f57321aca479f000a38a99bea36ba5685ee4d013ae8ff799fdd6: Status 404 returned error can't find the container with id bbcbcf099456f57321aca479f000a38a99bea36ba5685ee4d013ae8ff799fdd6 Jun 15 01:54:48.422854 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:48.422812 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qsxqx" event={"ID":"86a7cf17-a6d2-4416-97fc-b55d693f2a32","Type":"ContainerStarted","Data":"35820a6c303995904a4df853f852194be79005a45f71db12b36630717937152c"} Jun 15 01:54:48.423333 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:48.422872 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qsxqx" event={"ID":"86a7cf17-a6d2-4416-97fc-b55d693f2a32","Type":"ContainerStarted","Data":"f14c4d6babe1e12d502cdf7382af9c344ac8541c0a13d8fa4c386a7aee640bde"} Jun 15 01:54:48.423333 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:48.422901 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qsxqx" Jun 15 01:54:48.424719 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:48.424682 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7x7tv" podUID="c9571510-e31d-41f7-b74f-de47ff9fd969" containerName="manager" containerID="cri-o://3aaffbfc256f77ce1a91487870320b679b757b99bb9b15d6b0e5c12e4725b2ff" gracePeriod=2 Jun 15 01:54:48.426331 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:48.426293 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9j5lx" event={"ID":"d1e69d47-badd-48e7-9554-27e5a81201ca","Type":"ContainerStarted","Data":"c9ac537ea9e53be11eb8127df9eb4a1162d03353bd1dcd9a74d4fef9696cd4a2"} Jun 15 01:54:48.426449 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:48.426346 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9j5lx" event={"ID":"d1e69d47-badd-48e7-9554-27e5a81201ca","Type":"ContainerStarted","Data":"bbcbcf099456f57321aca479f000a38a99bea36ba5685ee4d013ae8ff799fdd6"} Jun 15 01:54:48.426449 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:48.426369 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9j5lx" Jun 15 01:54:48.449456 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:54:48.449424 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-7x7tv\" is forbidden: User \"system:node:ip-10-0-143-196.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-196.ec2.internal' and this object" podUID="c9571510-e31d-41f7-b74f-de47ff9fd969" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7x7tv" Jun 15 01:54:48.450673 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:48.450635 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qsxqx" podStartSLOduration=1.450621607 podStartE2EDuration="1.450621607s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 01:54:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 01:54:48.447356501 +0000 UTC m=+612.161082474" watchObservedRunningTime="2026-06-15 01:54:48.450621607 +0000 UTC m=+612.164347579" Jun 15 01:54:48.451406 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:54:48.451383 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qrsx2\" is forbidden: User \"system:node:ip-10-0-143-196.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-196.ec2.internal' and this object" podUID="0f0b4be3-e5e0-4eb8-803f-51f4a9820023" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qrsx2" Jun 15 01:54:48.453345 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:54:48.453307 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-7x7tv\" is forbidden: User \"system:node:ip-10-0-143-196.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-196.ec2.internal' and this object" podUID="c9571510-e31d-41f7-b74f-de47ff9fd969" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7x7tv" Jun 15 01:54:48.476152 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:54:48.476080 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qrsx2\" is forbidden: User \"system:node:ip-10-0-143-196.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-196.ec2.internal' and this object" podUID="0f0b4be3-e5e0-4eb8-803f-51f4a9820023" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qrsx2" Jun 15 01:54:48.476599 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:48.476545 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9j5lx" podStartSLOduration=1.476526548 podStartE2EDuration="1.476526548s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 01:54:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 01:54:48.473177755 +0000 UTC m=+612.186903730" watchObservedRunningTime="2026-06-15 01:54:48.476526548 +0000 UTC m=+612.190252522" Jun 15 01:54:48.678564 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:48.678497 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7x7tv" Jun 15 01:54:48.680916 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:54:48.680889 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-7x7tv\" is forbidden: User \"system:node:ip-10-0-143-196.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-196.ec2.internal' and this object" podUID="c9571510-e31d-41f7-b74f-de47ff9fd969" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7x7tv" Jun 15 01:54:48.682984 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:54:48.682956 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qrsx2\" is forbidden: User \"system:node:ip-10-0-143-196.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-196.ec2.internal' and this object" podUID="0f0b4be3-e5e0-4eb8-803f-51f4a9820023" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qrsx2" Jun 15 01:54:48.766600 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:48.766564 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f0b4be3-e5e0-4eb8-803f-51f4a9820023" path="/var/lib/kubelet/pods/0f0b4be3-e5e0-4eb8-803f-51f4a9820023/volumes" Jun 15 01:54:48.768299 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:48.768268 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c9571510-e31d-41f7-b74f-de47ff9fd969-extensions-socket-volume\") pod \"c9571510-e31d-41f7-b74f-de47ff9fd969\" (UID: \"c9571510-e31d-41f7-b74f-de47ff9fd969\") " Jun 15 01:54:48.768449 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:48.768378 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hdx9n\" (UniqueName: \"kubernetes.io/projected/c9571510-e31d-41f7-b74f-de47ff9fd969-kube-api-access-hdx9n\") pod \"c9571510-e31d-41f7-b74f-de47ff9fd969\" (UID: \"c9571510-e31d-41f7-b74f-de47ff9fd969\") " Jun 15 01:54:48.768548 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:48.768523 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9571510-e31d-41f7-b74f-de47ff9fd969-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "c9571510-e31d-41f7-b74f-de47ff9fd969" (UID: "c9571510-e31d-41f7-b74f-de47ff9fd969"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 01:54:48.770613 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:48.770586 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9571510-e31d-41f7-b74f-de47ff9fd969-kube-api-access-hdx9n" (OuterVolumeSpecName: "kube-api-access-hdx9n") pod "c9571510-e31d-41f7-b74f-de47ff9fd969" (UID: "c9571510-e31d-41f7-b74f-de47ff9fd969"). InnerVolumeSpecName "kube-api-access-hdx9n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 01:54:48.869353 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:48.869317 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hdx9n\" (UniqueName: \"kubernetes.io/projected/c9571510-e31d-41f7-b74f-de47ff9fd969-kube-api-access-hdx9n\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:54:48.869353 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:48.869349 2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c9571510-e31d-41f7-b74f-de47ff9fd969-extensions-socket-volume\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:54:49.434347 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:49.434313 2578 generic.go:356] "Generic (PLEG): container finished" podID="c9571510-e31d-41f7-b74f-de47ff9fd969" containerID="3aaffbfc256f77ce1a91487870320b679b757b99bb9b15d6b0e5c12e4725b2ff" exitCode=2 Jun 15 01:54:49.434772 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:49.434362 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7x7tv" Jun 15 01:54:49.434772 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:49.434421 2578 scope.go:117] "RemoveContainer" containerID="3aaffbfc256f77ce1a91487870320b679b757b99bb9b15d6b0e5c12e4725b2ff" Jun 15 01:54:49.444270 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:49.444254 2578 scope.go:117] "RemoveContainer" containerID="3aaffbfc256f77ce1a91487870320b679b757b99bb9b15d6b0e5c12e4725b2ff" Jun 15 01:54:49.444543 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:54:49.444524 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3aaffbfc256f77ce1a91487870320b679b757b99bb9b15d6b0e5c12e4725b2ff\": container with ID starting with 3aaffbfc256f77ce1a91487870320b679b757b99bb9b15d6b0e5c12e4725b2ff not found: ID does not exist" containerID="3aaffbfc256f77ce1a91487870320b679b757b99bb9b15d6b0e5c12e4725b2ff" Jun 15 01:54:49.444623 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:49.444550 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3aaffbfc256f77ce1a91487870320b679b757b99bb9b15d6b0e5c12e4725b2ff"} err="failed to get container status \"3aaffbfc256f77ce1a91487870320b679b757b99bb9b15d6b0e5c12e4725b2ff\": rpc error: code = NotFound desc = could not find container \"3aaffbfc256f77ce1a91487870320b679b757b99bb9b15d6b0e5c12e4725b2ff\": container with ID starting with 3aaffbfc256f77ce1a91487870320b679b757b99bb9b15d6b0e5c12e4725b2ff not found: ID does not exist" Jun 15 01:54:49.444993 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:54:49.444966 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-7x7tv\" is forbidden: User \"system:node:ip-10-0-143-196.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-196.ec2.internal' and this object" podUID="c9571510-e31d-41f7-b74f-de47ff9fd969" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7x7tv" Jun 15 01:54:50.766164 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:50.766131 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9571510-e31d-41f7-b74f-de47ff9fd969" path="/var/lib/kubelet/pods/c9571510-e31d-41f7-b74f-de47ff9fd969/volumes" Jun 15 01:54:59.437489 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:59.437456 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9j5lx" Jun 15 01:54:59.437998 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:59.437541 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qsxqx" Jun 15 01:54:59.499483 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:59.499446 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9j5lx"] Jun 15 01:54:59.499764 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:59.499722 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9j5lx" podUID="d1e69d47-badd-48e7-9554-27e5a81201ca" containerName="manager" containerID="cri-o://c9ac537ea9e53be11eb8127df9eb4a1162d03353bd1dcd9a74d4fef9696cd4a2" gracePeriod=10 Jun 15 01:54:59.756981 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:59.756944 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-64fsh"] Jun 15 01:54:59.760889 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:59.760868 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-64fsh" Jun 15 01:54:59.764878 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:59.764856 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9j5lx" Jun 15 01:54:59.773039 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:59.773011 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-64fsh"] Jun 15 01:54:59.870732 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:59.870694 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d1e69d47-badd-48e7-9554-27e5a81201ca-extensions-socket-volume\") pod \"d1e69d47-badd-48e7-9554-27e5a81201ca\" (UID: \"d1e69d47-badd-48e7-9554-27e5a81201ca\") " Jun 15 01:54:59.870929 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:59.870830 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xvpnh\" (UniqueName: \"kubernetes.io/projected/d1e69d47-badd-48e7-9554-27e5a81201ca-kube-api-access-xvpnh\") pod \"d1e69d47-badd-48e7-9554-27e5a81201ca\" (UID: \"d1e69d47-badd-48e7-9554-27e5a81201ca\") " Jun 15 01:54:59.870994 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:59.870953 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6fa80792-4f41-4d2e-9fab-722a4fc58910-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-64fsh\" (UID: \"6fa80792-4f41-4d2e-9fab-722a4fc58910\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-64fsh" Jun 15 01:54:59.871115 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:59.871061 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1e69d47-badd-48e7-9554-27e5a81201ca-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d1e69d47-badd-48e7-9554-27e5a81201ca" (UID: "d1e69d47-badd-48e7-9554-27e5a81201ca"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 01:54:59.871217 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:59.871191 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9794b\" (UniqueName: \"kubernetes.io/projected/6fa80792-4f41-4d2e-9fab-722a4fc58910-kube-api-access-9794b\") pod \"kuadrant-operator-controller-manager-86878c769f-64fsh\" (UID: \"6fa80792-4f41-4d2e-9fab-722a4fc58910\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-64fsh" Jun 15 01:54:59.871269 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:59.871258 2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d1e69d47-badd-48e7-9554-27e5a81201ca-extensions-socket-volume\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:54:59.873132 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:59.873104 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1e69d47-badd-48e7-9554-27e5a81201ca-kube-api-access-xvpnh" (OuterVolumeSpecName: "kube-api-access-xvpnh") pod "d1e69d47-badd-48e7-9554-27e5a81201ca" (UID: "d1e69d47-badd-48e7-9554-27e5a81201ca"). InnerVolumeSpecName "kube-api-access-xvpnh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 01:54:59.971864 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:59.971776 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6fa80792-4f41-4d2e-9fab-722a4fc58910-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-64fsh\" (UID: \"6fa80792-4f41-4d2e-9fab-722a4fc58910\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-64fsh" Jun 15 01:54:59.972019 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:59.971868 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9794b\" (UniqueName: \"kubernetes.io/projected/6fa80792-4f41-4d2e-9fab-722a4fc58910-kube-api-access-9794b\") pod \"kuadrant-operator-controller-manager-86878c769f-64fsh\" (UID: \"6fa80792-4f41-4d2e-9fab-722a4fc58910\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-64fsh" Jun 15 01:54:59.972019 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:59.971900 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xvpnh\" (UniqueName: \"kubernetes.io/projected/d1e69d47-badd-48e7-9554-27e5a81201ca-kube-api-access-xvpnh\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:54:59.972212 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:59.972190 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6fa80792-4f41-4d2e-9fab-722a4fc58910-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-64fsh\" (UID: \"6fa80792-4f41-4d2e-9fab-722a4fc58910\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-64fsh" Jun 15 01:54:59.983664 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:54:59.983637 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9794b\" (UniqueName: \"kubernetes.io/projected/6fa80792-4f41-4d2e-9fab-722a4fc58910-kube-api-access-9794b\") pod \"kuadrant-operator-controller-manager-86878c769f-64fsh\" (UID: \"6fa80792-4f41-4d2e-9fab-722a4fc58910\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-64fsh" Jun 15 01:55:00.071902 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:00.071858 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-64fsh" Jun 15 01:55:00.237155 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:00.237129 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-64fsh"] Jun 15 01:55:00.238617 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:55:00.238590 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fa80792_4f41_4d2e_9fab_722a4fc58910.slice/crio-7a7c28050949f9377017d986e9d10cb69c7af1ed2f081bc97a9899f78ed3a4e8 WatchSource:0}: Error finding container 7a7c28050949f9377017d986e9d10cb69c7af1ed2f081bc97a9899f78ed3a4e8: Status 404 returned error can't find the container with id 7a7c28050949f9377017d986e9d10cb69c7af1ed2f081bc97a9899f78ed3a4e8 Jun 15 01:55:00.501241 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:00.501148 2578 generic.go:356] "Generic (PLEG): container finished" podID="d1e69d47-badd-48e7-9554-27e5a81201ca" containerID="c9ac537ea9e53be11eb8127df9eb4a1162d03353bd1dcd9a74d4fef9696cd4a2" exitCode=0 Jun 15 01:55:00.501241 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:00.501201 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9j5lx" Jun 15 01:55:00.501716 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:00.501232 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9j5lx" event={"ID":"d1e69d47-badd-48e7-9554-27e5a81201ca","Type":"ContainerDied","Data":"c9ac537ea9e53be11eb8127df9eb4a1162d03353bd1dcd9a74d4fef9696cd4a2"} Jun 15 01:55:00.501716 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:00.501284 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9j5lx" event={"ID":"d1e69d47-badd-48e7-9554-27e5a81201ca","Type":"ContainerDied","Data":"bbcbcf099456f57321aca479f000a38a99bea36ba5685ee4d013ae8ff799fdd6"} Jun 15 01:55:00.501716 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:00.501318 2578 scope.go:117] "RemoveContainer" containerID="c9ac537ea9e53be11eb8127df9eb4a1162d03353bd1dcd9a74d4fef9696cd4a2" Jun 15 01:55:00.503050 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:00.503024 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-64fsh" event={"ID":"6fa80792-4f41-4d2e-9fab-722a4fc58910","Type":"ContainerStarted","Data":"5cf52accb774f0ba564e7dbe4b5c51f4b52bb11e23f3ee2665155f0f9eb4666d"} Jun 15 01:55:00.503166 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:00.503072 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-64fsh" event={"ID":"6fa80792-4f41-4d2e-9fab-722a4fc58910","Type":"ContainerStarted","Data":"7a7c28050949f9377017d986e9d10cb69c7af1ed2f081bc97a9899f78ed3a4e8"} Jun 15 01:55:00.503274 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:00.503189 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-64fsh" Jun 15 01:55:00.511068 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:00.511051 2578 scope.go:117] "RemoveContainer" containerID="c9ac537ea9e53be11eb8127df9eb4a1162d03353bd1dcd9a74d4fef9696cd4a2" Jun 15 01:55:00.511356 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:55:00.511337 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9ac537ea9e53be11eb8127df9eb4a1162d03353bd1dcd9a74d4fef9696cd4a2\": container with ID starting with c9ac537ea9e53be11eb8127df9eb4a1162d03353bd1dcd9a74d4fef9696cd4a2 not found: ID does not exist" containerID="c9ac537ea9e53be11eb8127df9eb4a1162d03353bd1dcd9a74d4fef9696cd4a2" Jun 15 01:55:00.511412 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:00.511366 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9ac537ea9e53be11eb8127df9eb4a1162d03353bd1dcd9a74d4fef9696cd4a2"} err="failed to get container status \"c9ac537ea9e53be11eb8127df9eb4a1162d03353bd1dcd9a74d4fef9696cd4a2\": rpc error: code = NotFound desc = could not find container \"c9ac537ea9e53be11eb8127df9eb4a1162d03353bd1dcd9a74d4fef9696cd4a2\": container with ID starting with c9ac537ea9e53be11eb8127df9eb4a1162d03353bd1dcd9a74d4fef9696cd4a2 not found: ID does not exist" Jun 15 01:55:00.526358 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:00.526295 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-64fsh" podStartSLOduration=1.5262789730000001 podStartE2EDuration="1.526278973s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 01:54:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 01:55:00.524473901 +0000 UTC m=+624.238199872" watchObservedRunningTime="2026-06-15 01:55:00.526278973 +0000 UTC m=+624.240004944" Jun 15 01:55:00.543694 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:00.543652 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9j5lx"] Jun 15 01:55:00.550172 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:00.550139 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9j5lx"] Jun 15 01:55:00.766787 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:00.766712 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1e69d47-badd-48e7-9554-27e5a81201ca" path="/var/lib/kubelet/pods/d1e69d47-badd-48e7-9554-27e5a81201ca/volumes" Jun 15 01:55:11.512440 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:11.512406 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-64fsh" Jun 15 01:55:11.565963 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:11.565933 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qsxqx"] Jun 15 01:55:11.566220 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:11.566196 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qsxqx" podUID="86a7cf17-a6d2-4416-97fc-b55d693f2a32" containerName="manager" containerID="cri-o://35820a6c303995904a4df853f852194be79005a45f71db12b36630717937152c" gracePeriod=10 Jun 15 01:55:11.835172 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:11.835146 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qsxqx" Jun 15 01:55:11.991210 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:11.991178 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/86a7cf17-a6d2-4416-97fc-b55d693f2a32-extensions-socket-volume\") pod \"86a7cf17-a6d2-4416-97fc-b55d693f2a32\" (UID: \"86a7cf17-a6d2-4416-97fc-b55d693f2a32\") " Jun 15 01:55:11.991405 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:11.991268 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zqdcn\" (UniqueName: \"kubernetes.io/projected/86a7cf17-a6d2-4416-97fc-b55d693f2a32-kube-api-access-zqdcn\") pod \"86a7cf17-a6d2-4416-97fc-b55d693f2a32\" (UID: \"86a7cf17-a6d2-4416-97fc-b55d693f2a32\") " Jun 15 01:55:11.991609 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:11.991584 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86a7cf17-a6d2-4416-97fc-b55d693f2a32-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "86a7cf17-a6d2-4416-97fc-b55d693f2a32" (UID: "86a7cf17-a6d2-4416-97fc-b55d693f2a32"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 01:55:11.993464 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:11.993438 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86a7cf17-a6d2-4416-97fc-b55d693f2a32-kube-api-access-zqdcn" (OuterVolumeSpecName: "kube-api-access-zqdcn") pod "86a7cf17-a6d2-4416-97fc-b55d693f2a32" (UID: "86a7cf17-a6d2-4416-97fc-b55d693f2a32"). InnerVolumeSpecName "kube-api-access-zqdcn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 01:55:12.092269 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:12.092181 2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/86a7cf17-a6d2-4416-97fc-b55d693f2a32-extensions-socket-volume\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:55:12.092269 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:12.092211 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zqdcn\" (UniqueName: \"kubernetes.io/projected/86a7cf17-a6d2-4416-97fc-b55d693f2a32-kube-api-access-zqdcn\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:55:12.576071 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:12.576035 2578 generic.go:356] "Generic (PLEG): container finished" podID="86a7cf17-a6d2-4416-97fc-b55d693f2a32" containerID="35820a6c303995904a4df853f852194be79005a45f71db12b36630717937152c" exitCode=0 Jun 15 01:55:12.576489 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:12.576097 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qsxqx" Jun 15 01:55:12.576489 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:12.576120 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qsxqx" event={"ID":"86a7cf17-a6d2-4416-97fc-b55d693f2a32","Type":"ContainerDied","Data":"35820a6c303995904a4df853f852194be79005a45f71db12b36630717937152c"} Jun 15 01:55:12.576489 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:12.576178 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qsxqx" event={"ID":"86a7cf17-a6d2-4416-97fc-b55d693f2a32","Type":"ContainerDied","Data":"f14c4d6babe1e12d502cdf7382af9c344ac8541c0a13d8fa4c386a7aee640bde"} Jun 15 01:55:12.576489 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:12.576202 2578 scope.go:117] "RemoveContainer" containerID="35820a6c303995904a4df853f852194be79005a45f71db12b36630717937152c" Jun 15 01:55:12.589460 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:12.589440 2578 scope.go:117] "RemoveContainer" containerID="35820a6c303995904a4df853f852194be79005a45f71db12b36630717937152c" Jun 15 01:55:12.589774 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:55:12.589741 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35820a6c303995904a4df853f852194be79005a45f71db12b36630717937152c\": container with ID starting with 35820a6c303995904a4df853f852194be79005a45f71db12b36630717937152c not found: ID does not exist" containerID="35820a6c303995904a4df853f852194be79005a45f71db12b36630717937152c" Jun 15 01:55:12.589870 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:12.589780 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35820a6c303995904a4df853f852194be79005a45f71db12b36630717937152c"} err="failed to get container status \"35820a6c303995904a4df853f852194be79005a45f71db12b36630717937152c\": rpc error: code = NotFound desc = could not find container \"35820a6c303995904a4df853f852194be79005a45f71db12b36630717937152c\": container with ID starting with 35820a6c303995904a4df853f852194be79005a45f71db12b36630717937152c not found: ID does not exist" Jun 15 01:55:12.600320 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:12.600293 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qsxqx"] Jun 15 01:55:12.608729 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:12.608700 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qsxqx"] Jun 15 01:55:12.767356 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:12.767321 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86a7cf17-a6d2-4416-97fc-b55d693f2a32" path="/var/lib/kubelet/pods/86a7cf17-a6d2-4416-97fc-b55d693f2a32/volumes" Jun 15 01:55:31.323480 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:31.323444 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-nmm7m"] Jun 15 01:55:31.324011 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:31.323987 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1e69d47-badd-48e7-9554-27e5a81201ca" containerName="manager" Jun 15 01:55:31.324110 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:31.324015 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1e69d47-badd-48e7-9554-27e5a81201ca" containerName="manager" Jun 15 01:55:31.324110 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:31.324030 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86a7cf17-a6d2-4416-97fc-b55d693f2a32" containerName="manager" Jun 15 01:55:31.324110 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:31.324038 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="86a7cf17-a6d2-4416-97fc-b55d693f2a32" containerName="manager" Jun 15 01:55:31.324273 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:31.324174 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1e69d47-badd-48e7-9554-27e5a81201ca" containerName="manager" Jun 15 01:55:31.324273 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:31.324191 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="86a7cf17-a6d2-4416-97fc-b55d693f2a32" containerName="manager" Jun 15 01:55:31.327457 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:31.327436 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-nmm7m" Jun 15 01:55:31.330447 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:31.330425 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-kcgnv\"" Jun 15 01:55:31.330447 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:31.330438 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jun 15 01:55:31.335946 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:31.335925 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-nmm7m"] Jun 15 01:55:31.354062 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:31.354032 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/95a64326-0625-4f8a-a576-d5a60f05797e-config-file\") pod \"limitador-limitador-64c94fd64f-nmm7m\" (UID: \"95a64326-0625-4f8a-a576-d5a60f05797e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-nmm7m" Jun 15 01:55:31.354210 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:31.354072 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cx6pn\" (UniqueName: \"kubernetes.io/projected/95a64326-0625-4f8a-a576-d5a60f05797e-kube-api-access-cx6pn\") pod \"limitador-limitador-64c94fd64f-nmm7m\" (UID: \"95a64326-0625-4f8a-a576-d5a60f05797e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-nmm7m" Jun 15 01:55:31.419478 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:31.419446 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-nmm7m"] Jun 15 01:55:31.455241 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:31.455204 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/95a64326-0625-4f8a-a576-d5a60f05797e-config-file\") pod \"limitador-limitador-64c94fd64f-nmm7m\" (UID: \"95a64326-0625-4f8a-a576-d5a60f05797e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-nmm7m" Jun 15 01:55:31.455413 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:31.455252 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cx6pn\" (UniqueName: \"kubernetes.io/projected/95a64326-0625-4f8a-a576-d5a60f05797e-kube-api-access-cx6pn\") pod \"limitador-limitador-64c94fd64f-nmm7m\" (UID: \"95a64326-0625-4f8a-a576-d5a60f05797e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-nmm7m" Jun 15 01:55:31.455884 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:31.455864 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/95a64326-0625-4f8a-a576-d5a60f05797e-config-file\") pod \"limitador-limitador-64c94fd64f-nmm7m\" (UID: \"95a64326-0625-4f8a-a576-d5a60f05797e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-nmm7m" Jun 15 01:55:31.463507 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:31.463482 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cx6pn\" (UniqueName: \"kubernetes.io/projected/95a64326-0625-4f8a-a576-d5a60f05797e-kube-api-access-cx6pn\") pod \"limitador-limitador-64c94fd64f-nmm7m\" (UID: \"95a64326-0625-4f8a-a576-d5a60f05797e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-nmm7m" Jun 15 01:55:31.639006 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:31.638962 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-nmm7m" Jun 15 01:55:31.780969 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:31.780937 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-nmm7m"] Jun 15 01:55:31.782530 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:55:31.782505 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95a64326_0625_4f8a_a576_d5a60f05797e.slice/crio-5666fb1a4dfdb2f798f197e113f707ce27f41e69ea4773d33c5e9f621b99c965 WatchSource:0}: Error finding container 5666fb1a4dfdb2f798f197e113f707ce27f41e69ea4773d33c5e9f621b99c965: Status 404 returned error can't find the container with id 5666fb1a4dfdb2f798f197e113f707ce27f41e69ea4773d33c5e9f621b99c965 Jun 15 01:55:32.698385 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:32.698333 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-nmm7m" event={"ID":"95a64326-0625-4f8a-a576-d5a60f05797e","Type":"ContainerStarted","Data":"5666fb1a4dfdb2f798f197e113f707ce27f41e69ea4773d33c5e9f621b99c965"} Jun 15 01:55:34.714230 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:34.714131 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-nmm7m" event={"ID":"95a64326-0625-4f8a-a576-d5a60f05797e","Type":"ContainerStarted","Data":"42ba5d271c0307881b0d4138ae91c73319a0d9e67e4896b2a1dec2b4b74e11e3"} Jun 15 01:55:34.714230 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:34.714205 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-nmm7m" Jun 15 01:55:34.732462 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:34.732407 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-nmm7m" podStartSLOduration=1.120656639 podStartE2EDuration="3.732391413s" totalImagesPullingTime="2.611734774s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 01:55:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 01:55:34.730329691 +0000 UTC m=+658.444055664" watchObservedRunningTime="2026-06-15 01:55:34.732391413 +0000 UTC m=+658.446117388" Jun 15 01:55:45.720817 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:45.720779 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-nmm7m" Jun 15 01:55:48.036609 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:48.036569 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-nmm7m"] Jun 15 01:55:48.037152 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:48.037011 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-nmm7m" podUID="95a64326-0625-4f8a-a576-d5a60f05797e" containerName="limitador" containerID="cri-o://42ba5d271c0307881b0d4138ae91c73319a0d9e67e4896b2a1dec2b4b74e11e3" gracePeriod=30 Jun 15 01:55:48.594152 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:48.594128 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-nmm7m" Jun 15 01:55:48.709372 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:48.709325 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cx6pn\" (UniqueName: \"kubernetes.io/projected/95a64326-0625-4f8a-a576-d5a60f05797e-kube-api-access-cx6pn\") pod \"95a64326-0625-4f8a-a576-d5a60f05797e\" (UID: \"95a64326-0625-4f8a-a576-d5a60f05797e\") " Jun 15 01:55:48.709545 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:48.709422 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/95a64326-0625-4f8a-a576-d5a60f05797e-config-file\") pod \"95a64326-0625-4f8a-a576-d5a60f05797e\" (UID: \"95a64326-0625-4f8a-a576-d5a60f05797e\") " Jun 15 01:55:48.709792 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:48.709768 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95a64326-0625-4f8a-a576-d5a60f05797e-config-file" (OuterVolumeSpecName: "config-file") pod "95a64326-0625-4f8a-a576-d5a60f05797e" (UID: "95a64326-0625-4f8a-a576-d5a60f05797e"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 01:55:48.711620 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:48.711591 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95a64326-0625-4f8a-a576-d5a60f05797e-kube-api-access-cx6pn" (OuterVolumeSpecName: "kube-api-access-cx6pn") pod "95a64326-0625-4f8a-a576-d5a60f05797e" (UID: "95a64326-0625-4f8a-a576-d5a60f05797e"). InnerVolumeSpecName "kube-api-access-cx6pn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 01:55:48.800354 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:48.800323 2578 generic.go:356] "Generic (PLEG): container finished" podID="95a64326-0625-4f8a-a576-d5a60f05797e" containerID="42ba5d271c0307881b0d4138ae91c73319a0d9e67e4896b2a1dec2b4b74e11e3" exitCode=0 Jun 15 01:55:48.800526 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:48.800372 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-nmm7m" Jun 15 01:55:48.800526 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:48.800414 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-nmm7m" event={"ID":"95a64326-0625-4f8a-a576-d5a60f05797e","Type":"ContainerDied","Data":"42ba5d271c0307881b0d4138ae91c73319a0d9e67e4896b2a1dec2b4b74e11e3"} Jun 15 01:55:48.800526 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:48.800466 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-nmm7m" event={"ID":"95a64326-0625-4f8a-a576-d5a60f05797e","Type":"ContainerDied","Data":"5666fb1a4dfdb2f798f197e113f707ce27f41e69ea4773d33c5e9f621b99c965"} Jun 15 01:55:48.800526 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:48.800488 2578 scope.go:117] "RemoveContainer" containerID="42ba5d271c0307881b0d4138ae91c73319a0d9e67e4896b2a1dec2b4b74e11e3" Jun 15 01:55:48.809548 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:48.809530 2578 scope.go:117] "RemoveContainer" containerID="42ba5d271c0307881b0d4138ae91c73319a0d9e67e4896b2a1dec2b4b74e11e3" Jun 15 01:55:48.809815 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:55:48.809794 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42ba5d271c0307881b0d4138ae91c73319a0d9e67e4896b2a1dec2b4b74e11e3\": container with ID starting with 42ba5d271c0307881b0d4138ae91c73319a0d9e67e4896b2a1dec2b4b74e11e3 not found: ID does not exist" containerID="42ba5d271c0307881b0d4138ae91c73319a0d9e67e4896b2a1dec2b4b74e11e3" Jun 15 01:55:48.809885 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:48.809822 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42ba5d271c0307881b0d4138ae91c73319a0d9e67e4896b2a1dec2b4b74e11e3"} err="failed to get container status \"42ba5d271c0307881b0d4138ae91c73319a0d9e67e4896b2a1dec2b4b74e11e3\": rpc error: code = NotFound desc = could not find container \"42ba5d271c0307881b0d4138ae91c73319a0d9e67e4896b2a1dec2b4b74e11e3\": container with ID starting with 42ba5d271c0307881b0d4138ae91c73319a0d9e67e4896b2a1dec2b4b74e11e3 not found: ID does not exist" Jun 15 01:55:48.810031 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:48.810014 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cx6pn\" (UniqueName: \"kubernetes.io/projected/95a64326-0625-4f8a-a576-d5a60f05797e-kube-api-access-cx6pn\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:55:48.810068 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:48.810037 2578 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/95a64326-0625-4f8a-a576-d5a60f05797e-config-file\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:55:48.821432 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:48.821388 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-nmm7m"] Jun 15 01:55:48.827776 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:48.827749 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-nmm7m"] Jun 15 01:55:49.946838 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:49.946807 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-mg9dq"] Jun 15 01:55:49.947328 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:49.947207 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95a64326-0625-4f8a-a576-d5a60f05797e" containerName="limitador" Jun 15 01:55:49.947328 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:49.947220 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="95a64326-0625-4f8a-a576-d5a60f05797e" containerName="limitador" Jun 15 01:55:49.947328 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:49.947306 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="95a64326-0625-4f8a-a576-d5a60f05797e" containerName="limitador" Jun 15 01:55:49.951749 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:49.951727 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-mg9dq" Jun 15 01:55:49.954412 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:49.954385 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jun 15 01:55:49.956737 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:49.956710 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-mg9dq"] Jun 15 01:55:50.020476 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:50.020429 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7rxm\" (UniqueName: \"kubernetes.io/projected/e8bb261a-c997-4d6a-a00a-9a009562f23c-kube-api-access-m7rxm\") pod \"postgres-69fb4594cf-mg9dq\" (UID: \"e8bb261a-c997-4d6a-a00a-9a009562f23c\") " pod="opendatahub/postgres-69fb4594cf-mg9dq" Jun 15 01:55:50.020666 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:50.020570 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e8bb261a-c997-4d6a-a00a-9a009562f23c-data\") pod \"postgres-69fb4594cf-mg9dq\" (UID: \"e8bb261a-c997-4d6a-a00a-9a009562f23c\") " pod="opendatahub/postgres-69fb4594cf-mg9dq" Jun 15 01:55:50.121578 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:50.121536 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7rxm\" (UniqueName: \"kubernetes.io/projected/e8bb261a-c997-4d6a-a00a-9a009562f23c-kube-api-access-m7rxm\") pod \"postgres-69fb4594cf-mg9dq\" (UID: \"e8bb261a-c997-4d6a-a00a-9a009562f23c\") " pod="opendatahub/postgres-69fb4594cf-mg9dq" Jun 15 01:55:50.121761 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:50.121651 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e8bb261a-c997-4d6a-a00a-9a009562f23c-data\") pod \"postgres-69fb4594cf-mg9dq\" (UID: \"e8bb261a-c997-4d6a-a00a-9a009562f23c\") " pod="opendatahub/postgres-69fb4594cf-mg9dq" Jun 15 01:55:50.122046 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:50.122027 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e8bb261a-c997-4d6a-a00a-9a009562f23c-data\") pod \"postgres-69fb4594cf-mg9dq\" (UID: \"e8bb261a-c997-4d6a-a00a-9a009562f23c\") " pod="opendatahub/postgres-69fb4594cf-mg9dq" Jun 15 01:55:50.129657 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:50.129631 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7rxm\" (UniqueName: \"kubernetes.io/projected/e8bb261a-c997-4d6a-a00a-9a009562f23c-kube-api-access-m7rxm\") pod \"postgres-69fb4594cf-mg9dq\" (UID: \"e8bb261a-c997-4d6a-a00a-9a009562f23c\") " pod="opendatahub/postgres-69fb4594cf-mg9dq" Jun 15 01:55:50.265869 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:50.265775 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-mg9dq" Jun 15 01:55:50.399307 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:50.399274 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-mg9dq"] Jun 15 01:55:50.401033 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:55:50.400987 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8bb261a_c997_4d6a_a00a_9a009562f23c.slice/crio-e96d2d6a1166bf8a6c6717e890737f82ca26eec8a3fb9961153949eaa0e06818 WatchSource:0}: Error finding container e96d2d6a1166bf8a6c6717e890737f82ca26eec8a3fb9961153949eaa0e06818: Status 404 returned error can't find the container with id e96d2d6a1166bf8a6c6717e890737f82ca26eec8a3fb9961153949eaa0e06818 Jun 15 01:55:50.766798 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:50.766768 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95a64326-0625-4f8a-a576-d5a60f05797e" path="/var/lib/kubelet/pods/95a64326-0625-4f8a-a576-d5a60f05797e/volumes" Jun 15 01:55:50.813095 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:50.813059 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-mg9dq" event={"ID":"e8bb261a-c997-4d6a-a00a-9a009562f23c","Type":"ContainerStarted","Data":"e96d2d6a1166bf8a6c6717e890737f82ca26eec8a3fb9961153949eaa0e06818"} Jun 15 01:55:53.832278 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:53.832247 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-mg9dq" event={"ID":"e8bb261a-c997-4d6a-a00a-9a009562f23c","Type":"ContainerStarted","Data":"1b82c03f4a5ae235c37c24f01921a55c18dc46ce1738242ae95b3774d20c4877"} Jun 15 01:55:53.832659 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:53.832391 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-mg9dq" Jun 15 01:55:53.849258 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:53.849209 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-mg9dq" podStartSLOduration=2.381841649 podStartE2EDuration="4.84919308s" totalImagesPullingTime="2.467351431s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 01:55:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 01:55:53.848235316 +0000 UTC m=+677.561961287" watchObservedRunningTime="2026-06-15 01:55:53.84919308 +0000 UTC m=+677.562919051" Jun 15 01:55:59.870454 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:55:59.870424 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-mg9dq" Jun 15 01:56:01.369531 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:01.369490 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7489kppv"] Jun 15 01:56:01.375722 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:01.375697 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7489kppv" Jun 15 01:56:01.378998 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:01.378967 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 15 01:56:01.380318 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:01.380284 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 15 01:56:01.380487 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:01.380350 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ncvbf\"" Jun 15 01:56:01.382160 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:01.382134 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7489kppv"] Jun 15 01:56:01.528445 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:01.528400 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pt7l\" (UniqueName: \"kubernetes.io/projected/f7e4f85e-a0ed-4ce6-8a16-585d69e7a1d4-kube-api-access-5pt7l\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7489kppv\" (UID: \"f7e4f85e-a0ed-4ce6-8a16-585d69e7a1d4\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7489kppv" Jun 15 01:56:01.528653 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:01.528463 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7e4f85e-a0ed-4ce6-8a16-585d69e7a1d4-util\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7489kppv\" (UID: \"f7e4f85e-a0ed-4ce6-8a16-585d69e7a1d4\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7489kppv" Jun 15 01:56:01.528653 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:01.528528 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7e4f85e-a0ed-4ce6-8a16-585d69e7a1d4-bundle\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7489kppv\" (UID: \"f7e4f85e-a0ed-4ce6-8a16-585d69e7a1d4\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7489kppv" Jun 15 01:56:01.629223 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:01.629179 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7e4f85e-a0ed-4ce6-8a16-585d69e7a1d4-bundle\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7489kppv\" (UID: \"f7e4f85e-a0ed-4ce6-8a16-585d69e7a1d4\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7489kppv" Jun 15 01:56:01.629433 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:01.629280 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pt7l\" (UniqueName: \"kubernetes.io/projected/f7e4f85e-a0ed-4ce6-8a16-585d69e7a1d4-kube-api-access-5pt7l\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7489kppv\" (UID: \"f7e4f85e-a0ed-4ce6-8a16-585d69e7a1d4\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7489kppv" Jun 15 01:56:01.629433 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:01.629328 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7e4f85e-a0ed-4ce6-8a16-585d69e7a1d4-util\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7489kppv\" (UID: \"f7e4f85e-a0ed-4ce6-8a16-585d69e7a1d4\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7489kppv" Jun 15 01:56:01.629586 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:01.629564 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7e4f85e-a0ed-4ce6-8a16-585d69e7a1d4-bundle\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7489kppv\" (UID: \"f7e4f85e-a0ed-4ce6-8a16-585d69e7a1d4\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7489kppv" Jun 15 01:56:01.629688 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:01.629668 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7e4f85e-a0ed-4ce6-8a16-585d69e7a1d4-util\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7489kppv\" (UID: \"f7e4f85e-a0ed-4ce6-8a16-585d69e7a1d4\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7489kppv" Jun 15 01:56:01.638970 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:01.638933 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pt7l\" (UniqueName: \"kubernetes.io/projected/f7e4f85e-a0ed-4ce6-8a16-585d69e7a1d4-kube-api-access-5pt7l\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7489kppv\" (UID: \"f7e4f85e-a0ed-4ce6-8a16-585d69e7a1d4\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7489kppv" Jun 15 01:56:01.686846 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:01.686805 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7489kppv" Jun 15 01:56:01.827804 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:01.827771 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7489kppv"] Jun 15 01:56:01.828428 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:56:01.828398 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7e4f85e_a0ed_4ce6_8a16_585d69e7a1d4.slice/crio-1ce86e0cfc3d8daa1928e2657dc2a1e62e27a9c92a86c5eef2a3a8d63eef3bc7 WatchSource:0}: Error finding container 1ce86e0cfc3d8daa1928e2657dc2a1e62e27a9c92a86c5eef2a3a8d63eef3bc7: Status 404 returned error can't find the container with id 1ce86e0cfc3d8daa1928e2657dc2a1e62e27a9c92a86c5eef2a3a8d63eef3bc7 Jun 15 01:56:01.878643 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:01.878611 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7489kppv" event={"ID":"f7e4f85e-a0ed-4ce6-8a16-585d69e7a1d4","Type":"ContainerStarted","Data":"1ce86e0cfc3d8daa1928e2657dc2a1e62e27a9c92a86c5eef2a3a8d63eef3bc7"} Jun 15 01:56:02.885341 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:02.885309 2578 generic.go:356] "Generic (PLEG): container finished" podID="f7e4f85e-a0ed-4ce6-8a16-585d69e7a1d4" containerID="f21bb66f6166882a71231db63ecc58e2e27ab2de436e622733c90fbb9bc4f0e6" exitCode=0 Jun 15 01:56:02.885723 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:02.885379 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7489kppv" event={"ID":"f7e4f85e-a0ed-4ce6-8a16-585d69e7a1d4","Type":"ContainerDied","Data":"f21bb66f6166882a71231db63ecc58e2e27ab2de436e622733c90fbb9bc4f0e6"} Jun 15 01:56:03.893438 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:03.893394 2578 generic.go:356] "Generic (PLEG): container finished" podID="f7e4f85e-a0ed-4ce6-8a16-585d69e7a1d4" containerID="acbbb02655016ed04d9156a7f113ad41cc280392f074e96651602128490c6e74" exitCode=0 Jun 15 01:56:03.893850 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:03.893480 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7489kppv" event={"ID":"f7e4f85e-a0ed-4ce6-8a16-585d69e7a1d4","Type":"ContainerDied","Data":"acbbb02655016ed04d9156a7f113ad41cc280392f074e96651602128490c6e74"} Jun 15 01:56:04.901330 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:04.901295 2578 generic.go:356] "Generic (PLEG): container finished" podID="f7e4f85e-a0ed-4ce6-8a16-585d69e7a1d4" containerID="df8df1d56d03b82927bd62eeb3a9b9700d1a24eb81ff6e9add671659f59c844e" exitCode=0 Jun 15 01:56:04.901790 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:04.901354 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7489kppv" event={"ID":"f7e4f85e-a0ed-4ce6-8a16-585d69e7a1d4","Type":"ContainerDied","Data":"df8df1d56d03b82927bd62eeb3a9b9700d1a24eb81ff6e9add671659f59c844e"} Jun 15 01:56:06.043160 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:06.043136 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7489kppv" Jun 15 01:56:06.169720 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:06.169626 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5pt7l\" (UniqueName: \"kubernetes.io/projected/f7e4f85e-a0ed-4ce6-8a16-585d69e7a1d4-kube-api-access-5pt7l\") pod \"f7e4f85e-a0ed-4ce6-8a16-585d69e7a1d4\" (UID: \"f7e4f85e-a0ed-4ce6-8a16-585d69e7a1d4\") " Jun 15 01:56:06.169720 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:06.169664 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7e4f85e-a0ed-4ce6-8a16-585d69e7a1d4-bundle\") pod \"f7e4f85e-a0ed-4ce6-8a16-585d69e7a1d4\" (UID: \"f7e4f85e-a0ed-4ce6-8a16-585d69e7a1d4\") " Jun 15 01:56:06.169720 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:06.169716 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7e4f85e-a0ed-4ce6-8a16-585d69e7a1d4-util\") pod \"f7e4f85e-a0ed-4ce6-8a16-585d69e7a1d4\" (UID: \"f7e4f85e-a0ed-4ce6-8a16-585d69e7a1d4\") " Jun 15 01:56:06.170221 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:06.170192 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7e4f85e-a0ed-4ce6-8a16-585d69e7a1d4-bundle" (OuterVolumeSpecName: "bundle") pod "f7e4f85e-a0ed-4ce6-8a16-585d69e7a1d4" (UID: "f7e4f85e-a0ed-4ce6-8a16-585d69e7a1d4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 01:56:06.171954 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:06.171930 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7e4f85e-a0ed-4ce6-8a16-585d69e7a1d4-kube-api-access-5pt7l" (OuterVolumeSpecName: "kube-api-access-5pt7l") pod "f7e4f85e-a0ed-4ce6-8a16-585d69e7a1d4" (UID: "f7e4f85e-a0ed-4ce6-8a16-585d69e7a1d4"). InnerVolumeSpecName "kube-api-access-5pt7l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 01:56:06.175761 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:06.175736 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7e4f85e-a0ed-4ce6-8a16-585d69e7a1d4-util" (OuterVolumeSpecName: "util") pod "f7e4f85e-a0ed-4ce6-8a16-585d69e7a1d4" (UID: "f7e4f85e-a0ed-4ce6-8a16-585d69e7a1d4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 01:56:06.270917 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:06.270874 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5pt7l\" (UniqueName: \"kubernetes.io/projected/f7e4f85e-a0ed-4ce6-8a16-585d69e7a1d4-kube-api-access-5pt7l\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:56:06.270917 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:06.270909 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7e4f85e-a0ed-4ce6-8a16-585d69e7a1d4-bundle\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:56:06.270917 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:06.270919 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7e4f85e-a0ed-4ce6-8a16-585d69e7a1d4-util\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:56:06.915059 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:06.915027 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7489kppv" Jun 15 01:56:06.915258 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:06.915026 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7489kppv" event={"ID":"f7e4f85e-a0ed-4ce6-8a16-585d69e7a1d4","Type":"ContainerDied","Data":"1ce86e0cfc3d8daa1928e2657dc2a1e62e27a9c92a86c5eef2a3a8d63eef3bc7"} Jun 15 01:56:06.915258 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:06.915144 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1ce86e0cfc3d8daa1928e2657dc2a1e62e27a9c92a86c5eef2a3a8d63eef3bc7" Jun 15 01:56:29.493576 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:29.493495 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 15 01:56:29.493981 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:29.493897 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7e4f85e-a0ed-4ce6-8a16-585d69e7a1d4" containerName="util" Jun 15 01:56:29.493981 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:29.493908 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7e4f85e-a0ed-4ce6-8a16-585d69e7a1d4" containerName="util" Jun 15 01:56:29.493981 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:29.493924 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7e4f85e-a0ed-4ce6-8a16-585d69e7a1d4" containerName="extract" Jun 15 01:56:29.493981 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:29.493930 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7e4f85e-a0ed-4ce6-8a16-585d69e7a1d4" containerName="extract" Jun 15 01:56:29.493981 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:29.493938 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7e4f85e-a0ed-4ce6-8a16-585d69e7a1d4" containerName="pull" Jun 15 01:56:29.493981 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:29.493944 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7e4f85e-a0ed-4ce6-8a16-585d69e7a1d4" containerName="pull" Jun 15 01:56:29.494198 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:29.493999 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7e4f85e-a0ed-4ce6-8a16-585d69e7a1d4" containerName="extract" Jun 15 01:56:29.497005 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:29.496987 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 15 01:56:29.499905 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:29.499879 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 15 01:56:29.500009 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:29.499892 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 15 01:56:29.500009 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:29.499942 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 15 01:56:29.500997 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:29.500982 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-cxqpg\"" Jun 15 01:56:29.505262 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:29.505240 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 15 01:56:29.559863 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:29.559821 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7fxc\" (UniqueName: \"kubernetes.io/projected/d95cf931-e057-4768-b92f-6b11cf85694d-kube-api-access-h7fxc\") pod \"maas-keycloak-0\" (UID: \"d95cf931-e057-4768-b92f-6b11cf85694d\") " pod="keycloak-system/maas-keycloak-0" Jun 15 01:56:29.660954 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:29.660912 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7fxc\" (UniqueName: \"kubernetes.io/projected/d95cf931-e057-4768-b92f-6b11cf85694d-kube-api-access-h7fxc\") pod \"maas-keycloak-0\" (UID: \"d95cf931-e057-4768-b92f-6b11cf85694d\") " pod="keycloak-system/maas-keycloak-0" Jun 15 01:56:29.669342 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:29.669310 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7fxc\" (UniqueName: \"kubernetes.io/projected/d95cf931-e057-4768-b92f-6b11cf85694d-kube-api-access-h7fxc\") pod \"maas-keycloak-0\" (UID: \"d95cf931-e057-4768-b92f-6b11cf85694d\") " pod="keycloak-system/maas-keycloak-0" Jun 15 01:56:29.807743 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:29.807634 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 15 01:56:29.948029 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:29.948004 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 15 01:56:29.948726 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:56:29.948701 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd95cf931_e057_4768_b92f_6b11cf85694d.slice/crio-64ee3eaa44e714a2a1a37c7b333b3cea2796addf4acae987fe8dc50f3090d605 WatchSource:0}: Error finding container 64ee3eaa44e714a2a1a37c7b333b3cea2796addf4acae987fe8dc50f3090d605: Status 404 returned error can't find the container with id 64ee3eaa44e714a2a1a37c7b333b3cea2796addf4acae987fe8dc50f3090d605 Jun 15 01:56:29.950139 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:29.950123 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 15 01:56:30.044962 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:30.044917 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"d95cf931-e057-4768-b92f-6b11cf85694d","Type":"ContainerStarted","Data":"64ee3eaa44e714a2a1a37c7b333b3cea2796addf4acae987fe8dc50f3090d605"} Jun 15 01:56:35.082559 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:35.082523 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"d95cf931-e057-4768-b92f-6b11cf85694d","Type":"ContainerStarted","Data":"6d560d2f494e7ff8b07555e300bee12746544446b072e63d79a3bf99e1f6c233"} Jun 15 01:56:35.100891 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:35.100828 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=1.632041735 podStartE2EDuration="6.100808207s" totalImagesPullingTime="4.468766472s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 01:56:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 01:56:35.099421445 +0000 UTC m=+718.813147418" watchObservedRunningTime="2026-06-15 01:56:35.100808207 +0000 UTC m=+718.814534180" Jun 15 01:56:35.807898 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:35.807857 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jun 15 01:56:35.809688 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:35.809645 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="d95cf931-e057-4768-b92f-6b11cf85694d" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.52:9000/health/started\": dial tcp 10.132.0.52:9000: connect: connection refused" Jun 15 01:56:36.808309 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:36.808276 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="d95cf931-e057-4768-b92f-6b11cf85694d" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.52:9000/health/started\": dial tcp 10.132.0.52:9000: connect: connection refused" Jun 15 01:56:37.809072 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:37.809023 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="d95cf931-e057-4768-b92f-6b11cf85694d" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.52:9000/health/started\": dial tcp 10.132.0.52:9000: connect: connection refused" Jun 15 01:56:38.808178 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:38.808126 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="d95cf931-e057-4768-b92f-6b11cf85694d" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.52:9000/health/started\": dial tcp 10.132.0.52:9000: connect: connection refused" Jun 15 01:56:39.808003 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:39.807945 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jun 15 01:56:39.809076 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:39.808484 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="d95cf931-e057-4768-b92f-6b11cf85694d" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.52:9000/health/started\": dial tcp 10.132.0.52:9000: connect: connection refused" Jun 15 01:56:40.808429 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:40.808379 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="d95cf931-e057-4768-b92f-6b11cf85694d" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.52:9000/health/started\": dial tcp 10.132.0.52:9000: connect: connection refused" Jun 15 01:56:41.808821 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:41.808769 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="d95cf931-e057-4768-b92f-6b11cf85694d" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.52:9000/health/started\": dial tcp 10.132.0.52:9000: connect: connection refused" Jun 15 01:56:42.808263 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:42.808219 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="d95cf931-e057-4768-b92f-6b11cf85694d" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.52:9000/health/started\": dial tcp 10.132.0.52:9000: connect: connection refused" Jun 15 01:56:43.808193 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:43.808139 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="d95cf931-e057-4768-b92f-6b11cf85694d" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.52:9000/health/started\": dial tcp 10.132.0.52:9000: connect: connection refused" Jun 15 01:56:44.808500 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:44.808446 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="d95cf931-e057-4768-b92f-6b11cf85694d" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.52:9000/health/started\": dial tcp 10.132.0.52:9000: connect: connection refused" Jun 15 01:56:45.808493 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:45.808444 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="d95cf931-e057-4768-b92f-6b11cf85694d" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.52:9000/health/started\": dial tcp 10.132.0.52:9000: connect: connection refused" Jun 15 01:56:46.808257 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:46.808215 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="d95cf931-e057-4768-b92f-6b11cf85694d" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.52:9000/health/started\": dial tcp 10.132.0.52:9000: connect: connection refused" Jun 15 01:56:47.934996 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:47.934958 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jun 15 01:56:47.950168 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:47.950112 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="d95cf931-e057-4768-b92f-6b11cf85694d" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 15 01:56:57.942507 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:56:57.942475 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jun 15 01:57:37.663108 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:37.663054 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-bb7bb5956-bh2nv"] Jun 15 01:57:37.666576 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:37.666554 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-bb7bb5956-bh2nv" Jun 15 01:57:37.669396 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:37.669352 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-r266q\"" Jun 15 01:57:37.669571 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:37.669553 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jun 15 01:57:37.676433 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:37.676409 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-bb7bb5956-bh2nv"] Jun 15 01:57:37.773846 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:37.773801 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0bdec198-1402-4d1a-91c8-5be0b4383a40-openshift-service-ca\") pod \"maas-api-bb7bb5956-bh2nv\" (UID: \"0bdec198-1402-4d1a-91c8-5be0b4383a40\") " pod="opendatahub/maas-api-bb7bb5956-bh2nv" Jun 15 01:57:37.774041 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:37.773855 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0bdec198-1402-4d1a-91c8-5be0b4383a40-maas-api-tls\") pod \"maas-api-bb7bb5956-bh2nv\" (UID: \"0bdec198-1402-4d1a-91c8-5be0b4383a40\") " pod="opendatahub/maas-api-bb7bb5956-bh2nv" Jun 15 01:57:37.774041 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:37.773885 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7mnb\" (UniqueName: \"kubernetes.io/projected/0bdec198-1402-4d1a-91c8-5be0b4383a40-kube-api-access-h7mnb\") pod \"maas-api-bb7bb5956-bh2nv\" (UID: \"0bdec198-1402-4d1a-91c8-5be0b4383a40\") " pod="opendatahub/maas-api-bb7bb5956-bh2nv" Jun 15 01:57:37.875364 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:37.875325 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0bdec198-1402-4d1a-91c8-5be0b4383a40-openshift-service-ca\") pod \"maas-api-bb7bb5956-bh2nv\" (UID: \"0bdec198-1402-4d1a-91c8-5be0b4383a40\") " pod="opendatahub/maas-api-bb7bb5956-bh2nv" Jun 15 01:57:37.875568 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:37.875371 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0bdec198-1402-4d1a-91c8-5be0b4383a40-maas-api-tls\") pod \"maas-api-bb7bb5956-bh2nv\" (UID: \"0bdec198-1402-4d1a-91c8-5be0b4383a40\") " pod="opendatahub/maas-api-bb7bb5956-bh2nv" Jun 15 01:57:37.875568 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:37.875401 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7mnb\" (UniqueName: \"kubernetes.io/projected/0bdec198-1402-4d1a-91c8-5be0b4383a40-kube-api-access-h7mnb\") pod \"maas-api-bb7bb5956-bh2nv\" (UID: \"0bdec198-1402-4d1a-91c8-5be0b4383a40\") " pod="opendatahub/maas-api-bb7bb5956-bh2nv" Jun 15 01:57:37.875568 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:57:37.875510 2578 secret.go:189] Couldn't get secret opendatahub/maas-api-serving-cert: secret "maas-api-serving-cert" not found Jun 15 01:57:37.875779 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:57:37.875579 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bdec198-1402-4d1a-91c8-5be0b4383a40-maas-api-tls podName:0bdec198-1402-4d1a-91c8-5be0b4383a40 nodeName:}" failed. No retries permitted until 2026-06-15 01:57:38.375559504 +0000 UTC m=+782.089285473 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0bdec198-1402-4d1a-91c8-5be0b4383a40-maas-api-tls") pod "maas-api-bb7bb5956-bh2nv" (UID: "0bdec198-1402-4d1a-91c8-5be0b4383a40") : secret "maas-api-serving-cert" not found Jun 15 01:57:37.876408 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:37.876377 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0bdec198-1402-4d1a-91c8-5be0b4383a40-openshift-service-ca\") pod \"maas-api-bb7bb5956-bh2nv\" (UID: \"0bdec198-1402-4d1a-91c8-5be0b4383a40\") " pod="opendatahub/maas-api-bb7bb5956-bh2nv" Jun 15 01:57:37.884370 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:37.884349 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7mnb\" (UniqueName: \"kubernetes.io/projected/0bdec198-1402-4d1a-91c8-5be0b4383a40-kube-api-access-h7mnb\") pod \"maas-api-bb7bb5956-bh2nv\" (UID: \"0bdec198-1402-4d1a-91c8-5be0b4383a40\") " pod="opendatahub/maas-api-bb7bb5956-bh2nv" Jun 15 01:57:38.116027 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:38.115949 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-579785bc64-clpvv"] Jun 15 01:57:38.119651 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:38.119629 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-579785bc64-clpvv" Jun 15 01:57:38.122349 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:38.122327 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jun 15 01:57:38.122349 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:38.122351 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-fwpc9\"" Jun 15 01:57:38.127443 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:38.127415 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-579785bc64-clpvv"] Jun 15 01:57:38.164030 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:38.163992 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-699b6cd6c-pl44j"] Jun 15 01:57:38.167624 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:38.167606 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-699b6cd6c-pl44j" Jun 15 01:57:38.177115 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:38.177071 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-699b6cd6c-pl44j"] Jun 15 01:57:38.178524 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:38.178493 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwl7x\" (UniqueName: \"kubernetes.io/projected/f843f2ab-3bc0-42e0-88dc-a17d7d1443a7-kube-api-access-dwl7x\") pod \"payload-pre-processing-579785bc64-clpvv\" (UID: \"f843f2ab-3bc0-42e0-88dc-a17d7d1443a7\") " pod="openshift-ingress/payload-pre-processing-579785bc64-clpvv" Jun 15 01:57:38.279074 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:38.279033 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hgb7\" (UniqueName: \"kubernetes.io/projected/9f01c072-54ff-4306-8403-72eec96b09ef-kube-api-access-8hgb7\") pod \"payload-processing-699b6cd6c-pl44j\" (UID: \"9f01c072-54ff-4306-8403-72eec96b09ef\") " pod="openshift-ingress/payload-processing-699b6cd6c-pl44j" Jun 15 01:57:38.279303 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:38.279199 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwl7x\" (UniqueName: \"kubernetes.io/projected/f843f2ab-3bc0-42e0-88dc-a17d7d1443a7-kube-api-access-dwl7x\") pod \"payload-pre-processing-579785bc64-clpvv\" (UID: \"f843f2ab-3bc0-42e0-88dc-a17d7d1443a7\") " pod="openshift-ingress/payload-pre-processing-579785bc64-clpvv" Jun 15 01:57:38.287720 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:38.287683 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwl7x\" (UniqueName: \"kubernetes.io/projected/f843f2ab-3bc0-42e0-88dc-a17d7d1443a7-kube-api-access-dwl7x\") pod \"payload-pre-processing-579785bc64-clpvv\" (UID: \"f843f2ab-3bc0-42e0-88dc-a17d7d1443a7\") " pod="openshift-ingress/payload-pre-processing-579785bc64-clpvv" Jun 15 01:57:38.380473 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:38.380426 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hgb7\" (UniqueName: \"kubernetes.io/projected/9f01c072-54ff-4306-8403-72eec96b09ef-kube-api-access-8hgb7\") pod \"payload-processing-699b6cd6c-pl44j\" (UID: \"9f01c072-54ff-4306-8403-72eec96b09ef\") " pod="openshift-ingress/payload-processing-699b6cd6c-pl44j" Jun 15 01:57:38.380680 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:38.380557 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0bdec198-1402-4d1a-91c8-5be0b4383a40-maas-api-tls\") pod \"maas-api-bb7bb5956-bh2nv\" (UID: \"0bdec198-1402-4d1a-91c8-5be0b4383a40\") " pod="opendatahub/maas-api-bb7bb5956-bh2nv" Jun 15 01:57:38.380763 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:57:38.380683 2578 secret.go:189] Couldn't get secret opendatahub/maas-api-serving-cert: secret "maas-api-serving-cert" not found Jun 15 01:57:38.380763 ip-10-0-143-196 kubenswrapper[2578]: E0615 01:57:38.380760 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bdec198-1402-4d1a-91c8-5be0b4383a40-maas-api-tls podName:0bdec198-1402-4d1a-91c8-5be0b4383a40 nodeName:}" failed. No retries permitted until 2026-06-15 01:57:39.380740137 +0000 UTC m=+783.094466090 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0bdec198-1402-4d1a-91c8-5be0b4383a40-maas-api-tls") pod "maas-api-bb7bb5956-bh2nv" (UID: "0bdec198-1402-4d1a-91c8-5be0b4383a40") : secret "maas-api-serving-cert" not found Jun 15 01:57:38.401970 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:38.401934 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hgb7\" (UniqueName: \"kubernetes.io/projected/9f01c072-54ff-4306-8403-72eec96b09ef-kube-api-access-8hgb7\") pod \"payload-processing-699b6cd6c-pl44j\" (UID: \"9f01c072-54ff-4306-8403-72eec96b09ef\") " pod="openshift-ingress/payload-processing-699b6cd6c-pl44j" Jun 15 01:57:38.433336 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:38.433305 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-579785bc64-clpvv" Jun 15 01:57:38.479678 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:38.479640 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-699b6cd6c-pl44j" Jun 15 01:57:38.589894 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:38.589811 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-579785bc64-clpvv"] Jun 15 01:57:38.591931 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:57:38.591901 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf843f2ab_3bc0_42e0_88dc_a17d7d1443a7.slice/crio-1a26e49cbceec6b1954098b751eb34e09c35ec1699999da5c631e5c631f894ad WatchSource:0}: Error finding container 1a26e49cbceec6b1954098b751eb34e09c35ec1699999da5c631e5c631f894ad: Status 404 returned error can't find the container with id 1a26e49cbceec6b1954098b751eb34e09c35ec1699999da5c631e5c631f894ad Jun 15 01:57:38.639406 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:38.639375 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-699b6cd6c-pl44j"] Jun 15 01:57:38.640362 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:57:38.640333 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f01c072_54ff_4306_8403_72eec96b09ef.slice/crio-151630967fd0d51cce58ceb1696121f5c11f337f087790e54b4132374062f643 WatchSource:0}: Error finding container 151630967fd0d51cce58ceb1696121f5c11f337f087790e54b4132374062f643: Status 404 returned error can't find the container with id 151630967fd0d51cce58ceb1696121f5c11f337f087790e54b4132374062f643 Jun 15 01:57:39.390889 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:39.390850 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0bdec198-1402-4d1a-91c8-5be0b4383a40-maas-api-tls\") pod \"maas-api-bb7bb5956-bh2nv\" (UID: \"0bdec198-1402-4d1a-91c8-5be0b4383a40\") " pod="opendatahub/maas-api-bb7bb5956-bh2nv" Jun 15 01:57:39.396572 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:39.396537 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0bdec198-1402-4d1a-91c8-5be0b4383a40-maas-api-tls\") pod \"maas-api-bb7bb5956-bh2nv\" (UID: \"0bdec198-1402-4d1a-91c8-5be0b4383a40\") " pod="opendatahub/maas-api-bb7bb5956-bh2nv" Jun 15 01:57:39.479470 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:39.479429 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-bb7bb5956-bh2nv" Jun 15 01:57:39.515886 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:39.515754 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-699b6cd6c-pl44j" event={"ID":"9f01c072-54ff-4306-8403-72eec96b09ef","Type":"ContainerStarted","Data":"151630967fd0d51cce58ceb1696121f5c11f337f087790e54b4132374062f643"} Jun 15 01:57:39.519141 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:39.519106 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-579785bc64-clpvv" event={"ID":"f843f2ab-3bc0-42e0-88dc-a17d7d1443a7","Type":"ContainerStarted","Data":"1a26e49cbceec6b1954098b751eb34e09c35ec1699999da5c631e5c631f894ad"} Jun 15 01:57:39.681543 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:39.681427 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-bb7bb5956-bh2nv"] Jun 15 01:57:39.689145 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:57:39.689109 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bdec198_1402_4d1a_91c8_5be0b4383a40.slice/crio-36b2ce0a71f320669d1969b49553c94e848fa61a8c7c2a6d3961e43b3a04ba86 WatchSource:0}: Error finding container 36b2ce0a71f320669d1969b49553c94e848fa61a8c7c2a6d3961e43b3a04ba86: Status 404 returned error can't find the container with id 36b2ce0a71f320669d1969b49553c94e848fa61a8c7c2a6d3961e43b3a04ba86 Jun 15 01:57:40.527777 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:40.527735 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-bb7bb5956-bh2nv" event={"ID":"0bdec198-1402-4d1a-91c8-5be0b4383a40","Type":"ContainerStarted","Data":"36b2ce0a71f320669d1969b49553c94e848fa61a8c7c2a6d3961e43b3a04ba86"} Jun 15 01:57:41.546024 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:41.545978 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-699b6cd6c-pl44j" event={"ID":"9f01c072-54ff-4306-8403-72eec96b09ef","Type":"ContainerStarted","Data":"3a42fe2f16bece2792108f2bd2496d37b74ddd91eca87c9a4912499251799804"} Jun 15 01:57:41.546643 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:41.546623 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-699b6cd6c-pl44j" Jun 15 01:57:41.548685 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:41.548659 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-579785bc64-clpvv" event={"ID":"f843f2ab-3bc0-42e0-88dc-a17d7d1443a7","Type":"ContainerStarted","Data":"3fff450f6ec2767a9668a9d99c43503d4e57a34dfbe4d1488008d9e82b2ba721"} Jun 15 01:57:41.548802 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:41.548717 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-579785bc64-clpvv" Jun 15 01:57:41.566571 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:41.566514 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-699b6cd6c-pl44j" podStartSLOduration=1.119033353 podStartE2EDuration="3.566493146s" totalImagesPullingTime="2.447459793s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 01:57:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 01:57:41.564344921 +0000 UTC m=+785.278070896" watchObservedRunningTime="2026-06-15 01:57:41.566493146 +0000 UTC m=+785.280219119" Jun 15 01:57:41.580845 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:41.580778 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-579785bc64-clpvv" podStartSLOduration=1.09261047 podStartE2EDuration="3.580758764s" totalImagesPullingTime="2.488148294s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 01:57:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 01:57:41.580419484 +0000 UTC m=+785.294145451" watchObservedRunningTime="2026-06-15 01:57:41.580758764 +0000 UTC m=+785.294484738" Jun 15 01:57:43.564219 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:43.564187 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-bb7bb5956-bh2nv" event={"ID":"0bdec198-1402-4d1a-91c8-5be0b4383a40","Type":"ContainerStarted","Data":"1d2c8481604e690cb00319900f0fa3d60833c1f6461f7cf7ffb8af869214790a"} Jun 15 01:57:43.564688 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:43.564282 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-bb7bb5956-bh2nv" Jun 15 01:57:43.582800 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:43.582748 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-bb7bb5956-bh2nv" podStartSLOduration=3.286608366 podStartE2EDuration="6.582732971s" totalImagesPullingTime="3.296124605s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 01:57:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 01:57:43.580889498 +0000 UTC m=+787.294615465" watchObservedRunningTime="2026-06-15 01:57:43.582732971 +0000 UTC m=+787.296458942" Jun 15 01:57:49.576334 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:49.576303 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-bb7bb5956-bh2nv" Jun 15 01:57:50.888143 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:50.888104 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 15 01:57:50.888600 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:50.888302 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="keycloak-system/maas-keycloak-0" podUID="d95cf931-e057-4768-b92f-6b11cf85694d" containerName="keycloak" containerID="cri-o://6d560d2f494e7ff8b07555e300bee12746544446b072e63d79a3bf99e1f6c233" gracePeriod=30 Jun 15 01:57:52.559339 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:52.559248 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-579785bc64-clpvv" Jun 15 01:57:52.620207 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:52.620169 2578 generic.go:356] "Generic (PLEG): container finished" podID="d95cf931-e057-4768-b92f-6b11cf85694d" containerID="6d560d2f494e7ff8b07555e300bee12746544446b072e63d79a3bf99e1f6c233" exitCode=143 Jun 15 01:57:52.620383 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:52.620243 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"d95cf931-e057-4768-b92f-6b11cf85694d","Type":"ContainerDied","Data":"6d560d2f494e7ff8b07555e300bee12746544446b072e63d79a3bf99e1f6c233"} Jun 15 01:57:52.945402 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:52.945375 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 15 01:57:53.016988 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:53.016955 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h7fxc\" (UniqueName: \"kubernetes.io/projected/d95cf931-e057-4768-b92f-6b11cf85694d-kube-api-access-h7fxc\") pod \"d95cf931-e057-4768-b92f-6b11cf85694d\" (UID: \"d95cf931-e057-4768-b92f-6b11cf85694d\") " Jun 15 01:57:53.019356 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:53.019328 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d95cf931-e057-4768-b92f-6b11cf85694d-kube-api-access-h7fxc" (OuterVolumeSpecName: "kube-api-access-h7fxc") pod "d95cf931-e057-4768-b92f-6b11cf85694d" (UID: "d95cf931-e057-4768-b92f-6b11cf85694d"). InnerVolumeSpecName "kube-api-access-h7fxc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 01:57:53.118043 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:53.117950 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h7fxc\" (UniqueName: \"kubernetes.io/projected/d95cf931-e057-4768-b92f-6b11cf85694d-kube-api-access-h7fxc\") on node \"ip-10-0-143-196.ec2.internal\" DevicePath \"\"" Jun 15 01:57:53.566294 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:53.566255 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-699b6cd6c-pl44j" Jun 15 01:57:53.628382 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:53.628345 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"d95cf931-e057-4768-b92f-6b11cf85694d","Type":"ContainerDied","Data":"64ee3eaa44e714a2a1a37c7b333b3cea2796addf4acae987fe8dc50f3090d605"} Jun 15 01:57:53.628551 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:53.628409 2578 scope.go:117] "RemoveContainer" containerID="6d560d2f494e7ff8b07555e300bee12746544446b072e63d79a3bf99e1f6c233" Jun 15 01:57:53.628551 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:53.628359 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 15 01:57:53.650786 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:53.650758 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 15 01:57:53.654748 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:53.654722 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 15 01:57:53.675301 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:53.675266 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 15 01:57:53.675670 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:53.675658 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d95cf931-e057-4768-b92f-6b11cf85694d" containerName="keycloak" Jun 15 01:57:53.675719 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:53.675672 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d95cf931-e057-4768-b92f-6b11cf85694d" containerName="keycloak" Jun 15 01:57:53.675754 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:53.675739 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d95cf931-e057-4768-b92f-6b11cf85694d" containerName="keycloak" Jun 15 01:57:53.680223 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:53.680205 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 15 01:57:53.682767 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:53.682742 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 15 01:57:53.682767 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:53.682773 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 15 01:57:53.682938 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:53.682773 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 15 01:57:53.682938 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:53.682896 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"keycloak-test-realms\"" Jun 15 01:57:53.683039 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:53.682975 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-cxqpg\"" Jun 15 01:57:53.691376 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:53.691350 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 15 01:57:53.825012 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:53.824917 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzp74\" (UniqueName: \"kubernetes.io/projected/28ff6f0b-ca0e-41e2-b99b-dd7a750a5717-kube-api-access-nzp74\") pod \"maas-keycloak-0\" (UID: \"28ff6f0b-ca0e-41e2-b99b-dd7a750a5717\") " pod="keycloak-system/maas-keycloak-0" Jun 15 01:57:53.825012 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:53.824956 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/28ff6f0b-ca0e-41e2-b99b-dd7a750a5717-test-realms\") pod \"maas-keycloak-0\" (UID: \"28ff6f0b-ca0e-41e2-b99b-dd7a750a5717\") " pod="keycloak-system/maas-keycloak-0" Jun 15 01:57:53.926420 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:53.926386 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzp74\" (UniqueName: \"kubernetes.io/projected/28ff6f0b-ca0e-41e2-b99b-dd7a750a5717-kube-api-access-nzp74\") pod \"maas-keycloak-0\" (UID: \"28ff6f0b-ca0e-41e2-b99b-dd7a750a5717\") " pod="keycloak-system/maas-keycloak-0" Jun 15 01:57:53.926420 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:53.926423 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/28ff6f0b-ca0e-41e2-b99b-dd7a750a5717-test-realms\") pod \"maas-keycloak-0\" (UID: \"28ff6f0b-ca0e-41e2-b99b-dd7a750a5717\") " pod="keycloak-system/maas-keycloak-0" Jun 15 01:57:53.927208 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:53.927189 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/28ff6f0b-ca0e-41e2-b99b-dd7a750a5717-test-realms\") pod \"maas-keycloak-0\" (UID: \"28ff6f0b-ca0e-41e2-b99b-dd7a750a5717\") " pod="keycloak-system/maas-keycloak-0" Jun 15 01:57:53.935712 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:53.935692 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzp74\" (UniqueName: \"kubernetes.io/projected/28ff6f0b-ca0e-41e2-b99b-dd7a750a5717-kube-api-access-nzp74\") pod \"maas-keycloak-0\" (UID: \"28ff6f0b-ca0e-41e2-b99b-dd7a750a5717\") " pod="keycloak-system/maas-keycloak-0" Jun 15 01:57:53.991768 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:53.991730 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 15 01:57:54.128317 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:54.128283 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 15 01:57:54.129706 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:57:54.129679 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28ff6f0b_ca0e_41e2_b99b_dd7a750a5717.slice/crio-2e688705ccff4a4437ac80b72711635d1a39c0a4ba7e4ef062f9d21d7847bf62 WatchSource:0}: Error finding container 2e688705ccff4a4437ac80b72711635d1a39c0a4ba7e4ef062f9d21d7847bf62: Status 404 returned error can't find the container with id 2e688705ccff4a4437ac80b72711635d1a39c0a4ba7e4ef062f9d21d7847bf62 Jun 15 01:57:54.637594 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:54.637559 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"28ff6f0b-ca0e-41e2-b99b-dd7a750a5717","Type":"ContainerStarted","Data":"7762b637857209725b0424d260642deb911122915f7fbe67192e49a5019292bd"} Jun 15 01:57:54.638050 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:54.637614 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"28ff6f0b-ca0e-41e2-b99b-dd7a750a5717","Type":"ContainerStarted","Data":"2e688705ccff4a4437ac80b72711635d1a39c0a4ba7e4ef062f9d21d7847bf62"} Jun 15 01:57:54.655040 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:54.654969 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=1.346687622 podStartE2EDuration="1.654947624s" totalImagesPullingTime="308.260002ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 01:57:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 01:57:54.654130149 +0000 UTC m=+798.367856122" watchObservedRunningTime="2026-06-15 01:57:54.654947624 +0000 UTC m=+798.368673597" Jun 15 01:57:54.770900 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:54.770807 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d95cf931-e057-4768-b92f-6b11cf85694d" path="/var/lib/kubelet/pods/d95cf931-e057-4768-b92f-6b11cf85694d/volumes" Jun 15 01:57:54.992563 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:54.992521 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jun 15 01:57:54.994293 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:54.994255 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="28ff6f0b-ca0e-41e2-b99b-dd7a750a5717" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.56:9000/health/started\": dial tcp 10.132.0.56:9000: connect: connection refused" Jun 15 01:57:55.993057 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:55.992920 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="28ff6f0b-ca0e-41e2-b99b-dd7a750a5717" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.56:9000/health/started\": dial tcp 10.132.0.56:9000: connect: connection refused" Jun 15 01:57:56.992798 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:56.992742 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="28ff6f0b-ca0e-41e2-b99b-dd7a750a5717" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.56:9000/health/started\": dial tcp 10.132.0.56:9000: connect: connection refused" Jun 15 01:57:57.992933 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:57.992872 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="28ff6f0b-ca0e-41e2-b99b-dd7a750a5717" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.56:9000/health/started\": dial tcp 10.132.0.56:9000: connect: connection refused" Jun 15 01:57:58.992276 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:58.992225 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="28ff6f0b-ca0e-41e2-b99b-dd7a750a5717" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.56:9000/health/started\": dial tcp 10.132.0.56:9000: connect: connection refused" Jun 15 01:57:59.993187 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:57:59.992737 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="28ff6f0b-ca0e-41e2-b99b-dd7a750a5717" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.56:9000/health/started\": dial tcp 10.132.0.56:9000: connect: connection refused" Jun 15 01:58:00.992241 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:00.992194 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="28ff6f0b-ca0e-41e2-b99b-dd7a750a5717" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.56:9000/health/started\": dial tcp 10.132.0.56:9000: connect: connection refused" Jun 15 01:58:01.993059 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:01.993004 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="28ff6f0b-ca0e-41e2-b99b-dd7a750a5717" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.56:9000/health/started\": dial tcp 10.132.0.56:9000: connect: connection refused" Jun 15 01:58:02.992742 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:02.992678 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="28ff6f0b-ca0e-41e2-b99b-dd7a750a5717" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.56:9000/health/started\": dial tcp 10.132.0.56:9000: connect: connection refused" Jun 15 01:58:03.991848 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:03.991805 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jun 15 01:58:03.992377 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:03.992108 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="28ff6f0b-ca0e-41e2-b99b-dd7a750a5717" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.56:9000/health/started\": dial tcp 10.132.0.56:9000: connect: connection refused" Jun 15 01:58:04.992822 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:04.992774 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="28ff6f0b-ca0e-41e2-b99b-dd7a750a5717" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.56:9000/health/started\": dial tcp 10.132.0.56:9000: connect: connection refused" Jun 15 01:58:05.992393 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:05.992341 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="28ff6f0b-ca0e-41e2-b99b-dd7a750a5717" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.56:9000/health/started\": dial tcp 10.132.0.56:9000: connect: connection refused" Jun 15 01:58:06.992615 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:06.992568 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="28ff6f0b-ca0e-41e2-b99b-dd7a750a5717" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.56:9000/health/started\": dial tcp 10.132.0.56:9000: connect: connection refused" Jun 15 01:58:08.123450 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:08.123404 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jun 15 01:58:08.143811 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:08.143291 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="28ff6f0b-ca0e-41e2-b99b-dd7a750a5717" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 15 01:58:18.128921 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:18.128879 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jun 15 01:58:45.731057 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:45.731017 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-2jvsx"] Jun 15 01:58:45.736387 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:45.736364 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-2jvsx" Jun 15 01:58:45.739305 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:45.739283 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jun 15 01:58:45.739305 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:45.739294 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jun 15 01:58:45.740462 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:45.740446 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-7xlhx\"" Jun 15 01:58:45.740538 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:45.740496 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jun 15 01:58:45.746229 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:45.746201 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-2jvsx"] Jun 15 01:58:45.914850 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:45.914815 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7310e2ee-80eb-4bfa-a811-0d22509b33ff-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-2jvsx\" (UID: \"7310e2ee-80eb-4bfa-a811-0d22509b33ff\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-2jvsx" Jun 15 01:58:45.915065 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:45.914876 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7310e2ee-80eb-4bfa-a811-0d22509b33ff-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-2jvsx\" (UID: \"7310e2ee-80eb-4bfa-a811-0d22509b33ff\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-2jvsx" Jun 15 01:58:45.915065 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:45.914946 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7310e2ee-80eb-4bfa-a811-0d22509b33ff-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-2jvsx\" (UID: \"7310e2ee-80eb-4bfa-a811-0d22509b33ff\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-2jvsx" Jun 15 01:58:45.915065 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:45.914998 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7310e2ee-80eb-4bfa-a811-0d22509b33ff-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-2jvsx\" (UID: \"7310e2ee-80eb-4bfa-a811-0d22509b33ff\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-2jvsx" Jun 15 01:58:45.915065 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:45.915024 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7310e2ee-80eb-4bfa-a811-0d22509b33ff-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-2jvsx\" (UID: \"7310e2ee-80eb-4bfa-a811-0d22509b33ff\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-2jvsx" Jun 15 01:58:45.915065 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:45.915068 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vd89r\" (UniqueName: \"kubernetes.io/projected/7310e2ee-80eb-4bfa-a811-0d22509b33ff-kube-api-access-vd89r\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-2jvsx\" (UID: \"7310e2ee-80eb-4bfa-a811-0d22509b33ff\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-2jvsx" Jun 15 01:58:46.016520 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:46.016427 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7310e2ee-80eb-4bfa-a811-0d22509b33ff-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-2jvsx\" (UID: \"7310e2ee-80eb-4bfa-a811-0d22509b33ff\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-2jvsx" Jun 15 01:58:46.016520 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:46.016463 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7310e2ee-80eb-4bfa-a811-0d22509b33ff-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-2jvsx\" (UID: \"7310e2ee-80eb-4bfa-a811-0d22509b33ff\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-2jvsx" Jun 15 01:58:46.016520 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:46.016486 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vd89r\" (UniqueName: \"kubernetes.io/projected/7310e2ee-80eb-4bfa-a811-0d22509b33ff-kube-api-access-vd89r\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-2jvsx\" (UID: \"7310e2ee-80eb-4bfa-a811-0d22509b33ff\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-2jvsx" Jun 15 01:58:46.016977 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:46.016546 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7310e2ee-80eb-4bfa-a811-0d22509b33ff-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-2jvsx\" (UID: \"7310e2ee-80eb-4bfa-a811-0d22509b33ff\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-2jvsx" Jun 15 01:58:46.016977 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:46.016591 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7310e2ee-80eb-4bfa-a811-0d22509b33ff-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-2jvsx\" (UID: \"7310e2ee-80eb-4bfa-a811-0d22509b33ff\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-2jvsx" Jun 15 01:58:46.016977 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:46.016615 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7310e2ee-80eb-4bfa-a811-0d22509b33ff-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-2jvsx\" (UID: \"7310e2ee-80eb-4bfa-a811-0d22509b33ff\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-2jvsx" Jun 15 01:58:46.017557 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:46.017531 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7310e2ee-80eb-4bfa-a811-0d22509b33ff-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-2jvsx\" (UID: \"7310e2ee-80eb-4bfa-a811-0d22509b33ff\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-2jvsx" Jun 15 01:58:46.017699 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:46.017637 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7310e2ee-80eb-4bfa-a811-0d22509b33ff-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-2jvsx\" (UID: \"7310e2ee-80eb-4bfa-a811-0d22509b33ff\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-2jvsx" Jun 15 01:58:46.017794 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:46.016916 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7310e2ee-80eb-4bfa-a811-0d22509b33ff-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-2jvsx\" (UID: \"7310e2ee-80eb-4bfa-a811-0d22509b33ff\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-2jvsx" Jun 15 01:58:46.027403 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:46.019681 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7310e2ee-80eb-4bfa-a811-0d22509b33ff-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-2jvsx\" (UID: \"7310e2ee-80eb-4bfa-a811-0d22509b33ff\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-2jvsx" Jun 15 01:58:46.027403 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:46.019852 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7310e2ee-80eb-4bfa-a811-0d22509b33ff-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-2jvsx\" (UID: \"7310e2ee-80eb-4bfa-a811-0d22509b33ff\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-2jvsx" Jun 15 01:58:46.027403 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:46.024460 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vd89r\" (UniqueName: \"kubernetes.io/projected/7310e2ee-80eb-4bfa-a811-0d22509b33ff-kube-api-access-vd89r\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-2jvsx\" (UID: \"7310e2ee-80eb-4bfa-a811-0d22509b33ff\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-2jvsx" Jun 15 01:58:46.047715 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:46.047678 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-2jvsx" Jun 15 01:58:46.193393 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:46.193360 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-2jvsx"] Jun 15 01:58:46.194558 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:58:46.194529 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7310e2ee_80eb_4bfa_a811_0d22509b33ff.slice/crio-9911d0de0c5812a066d23200c39dc5aff1eb60ccac23c4464d0a65894c24f82f WatchSource:0}: Error finding container 9911d0de0c5812a066d23200c39dc5aff1eb60ccac23c4464d0a65894c24f82f: Status 404 returned error can't find the container with id 9911d0de0c5812a066d23200c39dc5aff1eb60ccac23c4464d0a65894c24f82f Jun 15 01:58:47.011583 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:47.011536 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-2jvsx" event={"ID":"7310e2ee-80eb-4bfa-a811-0d22509b33ff","Type":"ContainerStarted","Data":"9911d0de0c5812a066d23200c39dc5aff1eb60ccac23c4464d0a65894c24f82f"} Jun 15 01:58:50.033503 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:50.033469 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-2jvsx" event={"ID":"7310e2ee-80eb-4bfa-a811-0d22509b33ff","Type":"ContainerStarted","Data":"db5919f7ab6a83027b04feb86038fc1a82864ecacc9f06646582d1b9c1302aa9"} Jun 15 01:58:53.528383 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:53.528347 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-5cdff5474d-4mhwb"] Jun 15 01:58:53.532329 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:53.532302 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-4mhwb" Jun 15 01:58:53.535416 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:53.535390 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jun 15 01:58:53.544774 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:53.544746 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-5cdff5474d-4mhwb"] Jun 15 01:58:53.690772 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:53.690732 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8191ab9b-318c-42ac-ab5f-27675d8ff918-model-cache\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-4mhwb\" (UID: \"8191ab9b-318c-42ac-ab5f-27675d8ff918\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-4mhwb" Jun 15 01:58:53.690772 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:53.690781 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8191ab9b-318c-42ac-ab5f-27675d8ff918-home\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-4mhwb\" (UID: \"8191ab9b-318c-42ac-ab5f-27675d8ff918\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-4mhwb" Jun 15 01:58:53.691025 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:53.690892 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8191ab9b-318c-42ac-ab5f-27675d8ff918-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-4mhwb\" (UID: \"8191ab9b-318c-42ac-ab5f-27675d8ff918\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-4mhwb" Jun 15 01:58:53.691025 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:53.690979 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vgfh\" (UniqueName: \"kubernetes.io/projected/8191ab9b-318c-42ac-ab5f-27675d8ff918-kube-api-access-5vgfh\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-4mhwb\" (UID: \"8191ab9b-318c-42ac-ab5f-27675d8ff918\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-4mhwb" Jun 15 01:58:53.691143 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:53.691062 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8191ab9b-318c-42ac-ab5f-27675d8ff918-dshm\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-4mhwb\" (UID: \"8191ab9b-318c-42ac-ab5f-27675d8ff918\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-4mhwb" Jun 15 01:58:53.691143 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:53.691117 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8191ab9b-318c-42ac-ab5f-27675d8ff918-tls-certs\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-4mhwb\" (UID: \"8191ab9b-318c-42ac-ab5f-27675d8ff918\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-4mhwb" Jun 15 01:58:53.791862 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:53.791760 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vgfh\" (UniqueName: \"kubernetes.io/projected/8191ab9b-318c-42ac-ab5f-27675d8ff918-kube-api-access-5vgfh\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-4mhwb\" (UID: \"8191ab9b-318c-42ac-ab5f-27675d8ff918\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-4mhwb" Jun 15 01:58:53.791862 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:53.791809 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8191ab9b-318c-42ac-ab5f-27675d8ff918-dshm\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-4mhwb\" (UID: \"8191ab9b-318c-42ac-ab5f-27675d8ff918\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-4mhwb" Jun 15 01:58:53.791862 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:53.791835 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8191ab9b-318c-42ac-ab5f-27675d8ff918-tls-certs\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-4mhwb\" (UID: \"8191ab9b-318c-42ac-ab5f-27675d8ff918\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-4mhwb" Jun 15 01:58:53.792195 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:53.791879 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8191ab9b-318c-42ac-ab5f-27675d8ff918-model-cache\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-4mhwb\" (UID: \"8191ab9b-318c-42ac-ab5f-27675d8ff918\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-4mhwb" Jun 15 01:58:53.792195 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:53.791907 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8191ab9b-318c-42ac-ab5f-27675d8ff918-home\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-4mhwb\" (UID: \"8191ab9b-318c-42ac-ab5f-27675d8ff918\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-4mhwb" Jun 15 01:58:53.792195 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:53.792050 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8191ab9b-318c-42ac-ab5f-27675d8ff918-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-4mhwb\" (UID: \"8191ab9b-318c-42ac-ab5f-27675d8ff918\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-4mhwb" Jun 15 01:58:53.792360 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:53.792328 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8191ab9b-318c-42ac-ab5f-27675d8ff918-home\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-4mhwb\" (UID: \"8191ab9b-318c-42ac-ab5f-27675d8ff918\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-4mhwb" Jun 15 01:58:53.792416 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:53.792380 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8191ab9b-318c-42ac-ab5f-27675d8ff918-model-cache\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-4mhwb\" (UID: \"8191ab9b-318c-42ac-ab5f-27675d8ff918\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-4mhwb" Jun 15 01:58:53.792471 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:53.792430 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8191ab9b-318c-42ac-ab5f-27675d8ff918-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-4mhwb\" (UID: \"8191ab9b-318c-42ac-ab5f-27675d8ff918\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-4mhwb" Jun 15 01:58:53.794309 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:53.794287 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8191ab9b-318c-42ac-ab5f-27675d8ff918-dshm\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-4mhwb\" (UID: \"8191ab9b-318c-42ac-ab5f-27675d8ff918\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-4mhwb" Jun 15 01:58:53.794612 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:53.794593 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8191ab9b-318c-42ac-ab5f-27675d8ff918-tls-certs\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-4mhwb\" (UID: \"8191ab9b-318c-42ac-ab5f-27675d8ff918\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-4mhwb" Jun 15 01:58:53.800269 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:53.800242 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vgfh\" (UniqueName: \"kubernetes.io/projected/8191ab9b-318c-42ac-ab5f-27675d8ff918-kube-api-access-5vgfh\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-4mhwb\" (UID: \"8191ab9b-318c-42ac-ab5f-27675d8ff918\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-4mhwb" Jun 15 01:58:53.845230 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:53.845186 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-4mhwb" Jun 15 01:58:53.991377 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:53.991341 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-5cdff5474d-4mhwb"] Jun 15 01:58:53.992535 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:58:53.992502 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8191ab9b_318c_42ac_ab5f_27675d8ff918.slice/crio-2405793eca383a616ee871877b93906cabbf0b4208a3cac9f0abe01f7e66bf10 WatchSource:0}: Error finding container 2405793eca383a616ee871877b93906cabbf0b4208a3cac9f0abe01f7e66bf10: Status 404 returned error can't find the container with id 2405793eca383a616ee871877b93906cabbf0b4208a3cac9f0abe01f7e66bf10 Jun 15 01:58:54.059722 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:54.059691 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-4mhwb" event={"ID":"8191ab9b-318c-42ac-ab5f-27675d8ff918","Type":"ContainerStarted","Data":"2405793eca383a616ee871877b93906cabbf0b4208a3cac9f0abe01f7e66bf10"} Jun 15 01:58:55.073109 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:55.067376 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-4mhwb" event={"ID":"8191ab9b-318c-42ac-ab5f-27675d8ff918","Type":"ContainerStarted","Data":"48263cec2c63c61a4fb14e3e2ae9d9c50c0e2a8aa6b2df1d3d931dad0f964954"} Jun 15 01:58:57.080231 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:57.080195 2578 generic.go:356] "Generic (PLEG): container finished" podID="7310e2ee-80eb-4bfa-a811-0d22509b33ff" containerID="db5919f7ab6a83027b04feb86038fc1a82864ecacc9f06646582d1b9c1302aa9" exitCode=0 Jun 15 01:58:57.080709 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:57.080268 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-2jvsx" event={"ID":"7310e2ee-80eb-4bfa-a811-0d22509b33ff","Type":"ContainerDied","Data":"db5919f7ab6a83027b04feb86038fc1a82864ecacc9f06646582d1b9c1302aa9"} Jun 15 01:58:59.093729 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:59.093685 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-2jvsx" event={"ID":"7310e2ee-80eb-4bfa-a811-0d22509b33ff","Type":"ContainerStarted","Data":"10526da37551f511fbb13edd4568d88142be29f8dc60e6fb5bc575da03f2b9df"} Jun 15 01:58:59.113141 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:58:59.113063 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-2jvsx" podStartSLOduration=2.496041337 podStartE2EDuration="14.113046375s" totalImagesPullingTime="4.176169526s" totalInitContainerRuntime="7.440835512s" isStatefulPod=false podCreationTimestamp="2026-06-15 01:58:45 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 01:58:59.111013492 +0000 UTC m=+862.824739467" watchObservedRunningTime="2026-06-15 01:58:59.113046375 +0000 UTC m=+862.826772346" Jun 15 01:59:06.048244 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:59:06.048212 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-2jvsx" Jun 15 01:59:06.048644 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:59:06.048300 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-2jvsx" Jun 15 01:59:06.060910 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:59:06.060887 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-2jvsx" Jun 15 01:59:06.144364 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:59:06.144322 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-2jvsx" Jun 15 01:59:17.714724 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:59:17.714688 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-6f969c879b-9vkqs"] Jun 15 01:59:17.730592 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:59:17.730549 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-6f969c879b-9vkqs"] Jun 15 01:59:17.730755 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:59:17.730683 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-9vkqs" Jun 15 01:59:17.733399 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:59:17.733374 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jun 15 01:59:17.813052 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:59:17.813021 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/66576744-3833-4f9a-bf9f-dc4c7b0c51b7-model-cache\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-9vkqs\" (UID: \"66576744-3833-4f9a-bf9f-dc4c7b0c51b7\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-9vkqs" Jun 15 01:59:17.813205 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:59:17.813113 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/66576744-3833-4f9a-bf9f-dc4c7b0c51b7-home\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-9vkqs\" (UID: \"66576744-3833-4f9a-bf9f-dc4c7b0c51b7\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-9vkqs" Jun 15 01:59:17.813205 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:59:17.813147 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86zs5\" (UniqueName: \"kubernetes.io/projected/66576744-3833-4f9a-bf9f-dc4c7b0c51b7-kube-api-access-86zs5\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-9vkqs\" (UID: \"66576744-3833-4f9a-bf9f-dc4c7b0c51b7\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-9vkqs" Jun 15 01:59:17.813320 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:59:17.813211 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/66576744-3833-4f9a-bf9f-dc4c7b0c51b7-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-9vkqs\" (UID: \"66576744-3833-4f9a-bf9f-dc4c7b0c51b7\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-9vkqs" Jun 15 01:59:17.813320 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:59:17.813260 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/66576744-3833-4f9a-bf9f-dc4c7b0c51b7-dshm\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-9vkqs\" (UID: \"66576744-3833-4f9a-bf9f-dc4c7b0c51b7\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-9vkqs" Jun 15 01:59:17.813320 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:59:17.813303 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/66576744-3833-4f9a-bf9f-dc4c7b0c51b7-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-9vkqs\" (UID: \"66576744-3833-4f9a-bf9f-dc4c7b0c51b7\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-9vkqs" Jun 15 01:59:17.914016 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:59:17.913974 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/66576744-3833-4f9a-bf9f-dc4c7b0c51b7-model-cache\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-9vkqs\" (UID: \"66576744-3833-4f9a-bf9f-dc4c7b0c51b7\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-9vkqs" Jun 15 01:59:17.914249 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:59:17.914043 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/66576744-3833-4f9a-bf9f-dc4c7b0c51b7-home\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-9vkqs\" (UID: \"66576744-3833-4f9a-bf9f-dc4c7b0c51b7\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-9vkqs" Jun 15 01:59:17.914249 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:59:17.914071 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86zs5\" (UniqueName: \"kubernetes.io/projected/66576744-3833-4f9a-bf9f-dc4c7b0c51b7-kube-api-access-86zs5\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-9vkqs\" (UID: \"66576744-3833-4f9a-bf9f-dc4c7b0c51b7\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-9vkqs" Jun 15 01:59:17.914249 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:59:17.914150 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/66576744-3833-4f9a-bf9f-dc4c7b0c51b7-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-9vkqs\" (UID: \"66576744-3833-4f9a-bf9f-dc4c7b0c51b7\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-9vkqs" Jun 15 01:59:17.914249 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:59:17.914198 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/66576744-3833-4f9a-bf9f-dc4c7b0c51b7-dshm\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-9vkqs\" (UID: \"66576744-3833-4f9a-bf9f-dc4c7b0c51b7\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-9vkqs" Jun 15 01:59:17.914249 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:59:17.914227 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/66576744-3833-4f9a-bf9f-dc4c7b0c51b7-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-9vkqs\" (UID: \"66576744-3833-4f9a-bf9f-dc4c7b0c51b7\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-9vkqs" Jun 15 01:59:17.914525 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:59:17.914403 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/66576744-3833-4f9a-bf9f-dc4c7b0c51b7-model-cache\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-9vkqs\" (UID: \"66576744-3833-4f9a-bf9f-dc4c7b0c51b7\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-9vkqs" Jun 15 01:59:17.914525 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:59:17.914456 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/66576744-3833-4f9a-bf9f-dc4c7b0c51b7-home\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-9vkqs\" (UID: \"66576744-3833-4f9a-bf9f-dc4c7b0c51b7\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-9vkqs" Jun 15 01:59:17.914632 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:59:17.914529 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/66576744-3833-4f9a-bf9f-dc4c7b0c51b7-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-9vkqs\" (UID: \"66576744-3833-4f9a-bf9f-dc4c7b0c51b7\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-9vkqs" Jun 15 01:59:17.916606 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:59:17.916581 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/66576744-3833-4f9a-bf9f-dc4c7b0c51b7-dshm\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-9vkqs\" (UID: \"66576744-3833-4f9a-bf9f-dc4c7b0c51b7\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-9vkqs" Jun 15 01:59:17.916890 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:59:17.916871 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/66576744-3833-4f9a-bf9f-dc4c7b0c51b7-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-9vkqs\" (UID: \"66576744-3833-4f9a-bf9f-dc4c7b0c51b7\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-9vkqs" Jun 15 01:59:17.922140 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:59:17.922117 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-86zs5\" (UniqueName: \"kubernetes.io/projected/66576744-3833-4f9a-bf9f-dc4c7b0c51b7-kube-api-access-86zs5\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-9vkqs\" (UID: \"66576744-3833-4f9a-bf9f-dc4c7b0c51b7\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-9vkqs" Jun 15 01:59:18.064277 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:59:18.064191 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-9vkqs" Jun 15 01:59:18.213027 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:59:18.213001 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-6f969c879b-9vkqs"] Jun 15 01:59:18.213992 ip-10-0-143-196 kubenswrapper[2578]: W0615 01:59:18.213969 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66576744_3833_4f9a_bf9f_dc4c7b0c51b7.slice/crio-2cb8e88c9cc7fe7c55157df1f4d9e7098f1bc43ed1e8f36236cf32217e8f4551 WatchSource:0}: Error finding container 2cb8e88c9cc7fe7c55157df1f4d9e7098f1bc43ed1e8f36236cf32217e8f4551: Status 404 returned error can't find the container with id 2cb8e88c9cc7fe7c55157df1f4d9e7098f1bc43ed1e8f36236cf32217e8f4551 Jun 15 01:59:19.216745 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:59:19.216710 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-9vkqs" event={"ID":"66576744-3833-4f9a-bf9f-dc4c7b0c51b7","Type":"ContainerStarted","Data":"df7985d04f608c24749971ab9301c22a327a7a91e933c9f5b97f9370eb12ba23"} Jun 15 01:59:19.217224 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:59:19.216762 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-9vkqs" event={"ID":"66576744-3833-4f9a-bf9f-dc4c7b0c51b7","Type":"ContainerStarted","Data":"2cb8e88c9cc7fe7c55157df1f4d9e7098f1bc43ed1e8f36236cf32217e8f4551"} Jun 15 01:59:20.224502 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:59:20.224470 2578 generic.go:356] "Generic (PLEG): container finished" podID="66576744-3833-4f9a-bf9f-dc4c7b0c51b7" containerID="df7985d04f608c24749971ab9301c22a327a7a91e933c9f5b97f9370eb12ba23" exitCode=0 Jun 15 01:59:20.224908 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:59:20.224526 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-9vkqs" event={"ID":"66576744-3833-4f9a-bf9f-dc4c7b0c51b7","Type":"ContainerDied","Data":"df7985d04f608c24749971ab9301c22a327a7a91e933c9f5b97f9370eb12ba23"} Jun 15 01:59:21.232582 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:59:21.232545 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-9vkqs" event={"ID":"66576744-3833-4f9a-bf9f-dc4c7b0c51b7","Type":"ContainerStarted","Data":"e9b444261ff9f9a5ac08bb7b0c71a15c9391f47400908cd130e9c4d53cec450f"} Jun 15 01:59:21.252271 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:59:21.252219 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-9vkqs" podStartSLOduration=2.7149084070000002 podStartE2EDuration="4.252201732s" totalImagesPullingTime="178.684031ms" totalInitContainerRuntime="1.358609294s" isStatefulPod=false podCreationTimestamp="2026-06-15 01:59:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 01:59:21.250202477 +0000 UTC m=+884.963928467" watchObservedRunningTime="2026-06-15 01:59:21.252201732 +0000 UTC m=+884.965927703" Jun 15 01:59:28.064715 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:59:28.064677 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-9vkqs" Jun 15 01:59:28.065224 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:59:28.064764 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-9vkqs" Jun 15 01:59:28.081053 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:59:28.081027 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-9vkqs" Jun 15 01:59:28.290696 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:59:28.290669 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-9vkqs" Jun 15 01:59:36.808895 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:59:36.808865 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cf286_64f557b0-78d8-45a6-9f54-756417231d6c/ovn-acl-logging/0.log" Jun 15 01:59:36.810124 ip-10-0-143-196 kubenswrapper[2578]: I0615 01:59:36.810073 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cf286_64f557b0-78d8-45a6-9f54-756417231d6c/ovn-acl-logging/0.log" Jun 15 02:00:38.695947 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:00:38.695914 2578 generic.go:356] "Generic (PLEG): container finished" podID="8191ab9b-318c-42ac-ab5f-27675d8ff918" containerID="48263cec2c63c61a4fb14e3e2ae9d9c50c0e2a8aa6b2df1d3d931dad0f964954" exitCode=0 Jun 15 02:00:38.696509 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:00:38.695998 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-4mhwb" event={"ID":"8191ab9b-318c-42ac-ab5f-27675d8ff918","Type":"ContainerDied","Data":"48263cec2c63c61a4fb14e3e2ae9d9c50c0e2a8aa6b2df1d3d931dad0f964954"} Jun 15 02:00:39.703357 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:00:39.703317 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-4mhwb" event={"ID":"8191ab9b-318c-42ac-ab5f-27675d8ff918","Type":"ContainerStarted","Data":"8407f2c809b2bf0df433fd0869798cb5201d6972eaf242b8b1947f5f5b10e4f6"} Jun 15 02:00:39.722374 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:00:39.722328 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-4mhwb" podStartSLOduration=2.040958961 podStartE2EDuration="1m46.722314369s" totalImagesPullingTime="191.951611ms" totalInitContainerRuntime="1m44.489403797s" isStatefulPod=false podCreationTimestamp="2026-06-15 01:58:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 02:00:39.720478286 +0000 UTC m=+963.434204259" watchObservedRunningTime="2026-06-15 02:00:39.722314369 +0000 UTC m=+963.436040356" Jun 15 02:00:43.845503 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:00:43.845457 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-4mhwb" Jun 15 02:00:43.845947 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:00:43.845536 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-4mhwb" Jun 15 02:00:43.858588 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:00:43.858560 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-4mhwb" Jun 15 02:00:44.752522 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:00:44.752493 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-4mhwb" Jun 15 02:04:36.860912 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:04:36.860887 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cf286_64f557b0-78d8-45a6-9f54-756417231d6c/ovn-acl-logging/0.log" Jun 15 02:04:36.863408 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:04:36.863387 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cf286_64f557b0-78d8-45a6-9f54-756417231d6c/ovn-acl-logging/0.log" Jun 15 02:06:24.386044 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:24.386012 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-79b56865f7-btnp2_f890d1a2-5c4a-4eac-bd8f-8f7c11c07a57/nginx/0.log" Jun 15 02:06:24.605913 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:24.605882 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-69fc866d76-4mzv8_019c220f-42d7-4691-b5d4-183b9f27c964/manager/0.log" Jun 15 02:06:24.716605 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:24.716523 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-864bfd7bc9-6z22w_f694f872-296b-4205-98ef-d03f361e25ae/manager/0.log" Jun 15 02:06:24.822710 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:24.822680 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-bb7bb5956-bh2nv_0bdec198-1402-4d1a-91c8-5be0b4383a40/maas-api/0.log" Jun 15 02:06:25.173686 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:25.173653 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-5dd488fc48-l7wlm_40c9ba7d-66ce-49f9-a472-5445a57d1968/server/0.log" Jun 15 02:06:25.293893 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:25.293850 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-6dc94f689f-jntw9_4cb41eb9-34d4-495b-8534-0a7b7f4b42d2/manager/0.log" Jun 15 02:06:25.522123 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:25.522028 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d494fbff9-9ndp7_729a1b01-1fa0-44fb-88e2-c23c5cad4deb/copy-manifests/0.log" Jun 15 02:06:25.528417 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:25.528390 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d494fbff9-9ndp7_729a1b01-1fa0-44fb-88e2-c23c5cad4deb/manager/0.log" Jun 15 02:06:25.771423 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:25.771394 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-mg9dq_e8bb261a-c997-4d6a-a00a-9a009562f23c/postgres/0.log" Jun 15 02:06:26.517146 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:26.517118 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p7crd_43f8bbf1-6346-42b0-9581-547004591362/pull/0.log" Jun 15 02:06:26.523335 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:26.523311 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p7crd_43f8bbf1-6346-42b0-9581-547004591362/extract/0.log" Jun 15 02:06:26.529131 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:26.529111 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p7crd_43f8bbf1-6346-42b0-9581-547004591362/util/0.log" Jun 15 02:06:26.634747 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:26.634715 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v2wpk_f483827b-baf7-4a44-8bbb-8027562a973e/util/0.log" Jun 15 02:06:26.641227 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:26.641204 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v2wpk_f483827b-baf7-4a44-8bbb-8027562a973e/pull/0.log" Jun 15 02:06:26.647294 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:26.647275 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v2wpk_f483827b-baf7-4a44-8bbb-8027562a973e/extract/0.log" Jun 15 02:06:26.755820 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:26.755775 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wmpwk_e4f239ad-a463-4488-95af-aa5739b6a334/extract/0.log" Jun 15 02:06:26.761570 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:26.761539 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wmpwk_e4f239ad-a463-4488-95af-aa5739b6a334/util/0.log" Jun 15 02:06:26.767308 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:26.767250 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wmpwk_e4f239ad-a463-4488-95af-aa5739b6a334/pull/0.log" Jun 15 02:06:26.874975 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:26.874937 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12tpbp_9ccbd467-a8f2-43a8-ad76-b08698c29c54/util/0.log" Jun 15 02:06:26.881267 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:26.881246 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12tpbp_9ccbd467-a8f2-43a8-ad76-b08698c29c54/pull/0.log" Jun 15 02:06:26.887375 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:26.887348 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12tpbp_9ccbd467-a8f2-43a8-ad76-b08698c29c54/extract/0.log" Jun 15 02:06:27.567300 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:27.567272 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-64fsh_6fa80792-4f41-4d2e-9fab-722a4fc58910/manager/0.log" Jun 15 02:06:28.239707 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:28.239677 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-4nlft_b50e4740-5402-4abc-8737-a2aebb7e69c7/discovery/0.log" Jun 15 02:06:28.345621 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:28.345593 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-644c7647f-gszx8_0fcdf2ea-d316-456b-8f67-3454dec3558f/kube-auth-proxy/0.log" Jun 15 02:06:28.667951 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:28.667927 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-579785bc64-clpvv_f843f2ab-3bc0-42e0-88dc-a17d7d1443a7/payload-pre-processing/0.log" Jun 15 02:06:28.772778 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:28.772752 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-699b6cd6c-pl44j_9f01c072-54ff-4306-8403-72eec96b09ef/payload-processing/0.log" Jun 15 02:06:28.881058 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:28.881026 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-58cf549cb4-mc482_d4979e05-89c5-482b-a895-4f1f9ae54e93/router/0.log" Jun 15 02:06:29.221746 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:29.221711 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-f5c55bdc-2jvsx_7310e2ee-80eb-4bfa-a811-0d22509b33ff/storage-initializer/0.log" Jun 15 02:06:29.228830 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:29.228796 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-f5c55bdc-2jvsx_7310e2ee-80eb-4bfa-a811-0d22509b33ff/main/0.log" Jun 15 02:06:29.348067 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:29.348035 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-5cdff5474d-4mhwb_8191ab9b-318c-42ac-ab5f-27675d8ff918/storage-initializer/0.log" Jun 15 02:06:29.368376 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:29.368348 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-5cdff5474d-4mhwb_8191ab9b-318c-42ac-ab5f-27675d8ff918/main/0.log" Jun 15 02:06:29.710669 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:29.710635 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-6f969c879b-9vkqs_66576744-3833-4f9a-bf9f-dc4c7b0c51b7/storage-initializer/0.log" Jun 15 02:06:29.718290 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:29.718259 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-6f969c879b-9vkqs_66576744-3833-4f9a-bf9f-dc4c7b0c51b7/main/0.log" Jun 15 02:06:37.004329 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:37.004300 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-6272r_a6368c58-4355-4d1c-b621-75c5bf0e7394/global-pull-secret-syncer/0.log" Jun 15 02:06:37.181972 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:37.181939 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-tnmwb_d9f8083b-6138-4b4d-b8b8-d11ca6811a9d/konnectivity-agent/0.log" Jun 15 02:06:37.243037 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:37.243006 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-196.ec2.internal_3461aba319c72ca2a9e406f3c9bf17da/haproxy/0.log" Jun 15 02:06:41.358241 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:41.358214 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-operators_servicemesh-operator3-57f65f65fb-k7kj6_6576348b-5d9e-4960-b3b7-351d766d83cf/sail-operator/0.log" Jun 15 02:06:42.337146 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:42.337106 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p7crd_43f8bbf1-6346-42b0-9581-547004591362/extract/0.log" Jun 15 02:06:42.361207 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:42.361172 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p7crd_43f8bbf1-6346-42b0-9581-547004591362/util/0.log" Jun 15 02:06:42.382891 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:42.382867 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p7crd_43f8bbf1-6346-42b0-9581-547004591362/pull/0.log" Jun 15 02:06:42.416000 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:42.415960 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v2wpk_f483827b-baf7-4a44-8bbb-8027562a973e/extract/0.log" Jun 15 02:06:42.435241 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:42.435211 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v2wpk_f483827b-baf7-4a44-8bbb-8027562a973e/util/0.log" Jun 15 02:06:42.454314 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:42.454285 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v2wpk_f483827b-baf7-4a44-8bbb-8027562a973e/pull/0.log" Jun 15 02:06:42.478773 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:42.478708 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wmpwk_e4f239ad-a463-4488-95af-aa5739b6a334/extract/0.log" Jun 15 02:06:42.498999 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:42.498970 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wmpwk_e4f239ad-a463-4488-95af-aa5739b6a334/util/0.log" Jun 15 02:06:42.517044 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:42.517015 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wmpwk_e4f239ad-a463-4488-95af-aa5739b6a334/pull/0.log" Jun 15 02:06:42.540963 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:42.540938 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12tpbp_9ccbd467-a8f2-43a8-ad76-b08698c29c54/extract/0.log" Jun 15 02:06:42.566544 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:42.566520 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12tpbp_9ccbd467-a8f2-43a8-ad76-b08698c29c54/util/0.log" Jun 15 02:06:42.585816 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:42.585785 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12tpbp_9ccbd467-a8f2-43a8-ad76-b08698c29c54/pull/0.log" Jun 15 02:06:42.777528 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:42.777497 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-64fsh_6fa80792-4f41-4d2e-9fab-722a4fc58910/manager/0.log" Jun 15 02:06:44.504883 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:44.504848 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-589576dfc7-9kvt9_52ed7008-98b2-43b1-910c-b75cde0911e3/monitoring-plugin/0.log" Jun 15 02:06:44.603846 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:44.603821 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nbzph_46e7619f-0f36-4119-841d-e8a8713a5ee5/node-exporter/0.log" Jun 15 02:06:44.632452 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:44.632422 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nbzph_46e7619f-0f36-4119-841d-e8a8713a5ee5/kube-rbac-proxy/0.log" Jun 15 02:06:44.674314 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:44.674290 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nbzph_46e7619f-0f36-4119-841d-e8a8713a5ee5/init-textfile/0.log" Jun 15 02:06:44.794021 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:44.793944 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-6b8d965dd5-78tw8_c4bb542e-3182-427d-ae55-e6e0897abf87/kube-rbac-proxy-main/0.log" Jun 15 02:06:44.814733 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:44.814703 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-6b8d965dd5-78tw8_c4bb542e-3182-427d-ae55-e6e0897abf87/kube-rbac-proxy-self/0.log" Jun 15 02:06:44.836180 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:44.836155 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-6b8d965dd5-78tw8_c4bb542e-3182-427d-ae55-e6e0897abf87/openshift-state-metrics/0.log" Jun 15 02:06:45.084548 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:45.084472 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5cf667749c-bzm9j_8f715911-523f-4405-be5a-9e902db5609a/prometheus-operator-admission-webhook/0.log" Jun 15 02:06:45.192370 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:45.192336 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d8dcf9d4c-n6wws_fbcce109-1cea-4846-86ef-198e1bd347e4/thanos-query/0.log" Jun 15 02:06:45.212506 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:45.212478 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d8dcf9d4c-n6wws_fbcce109-1cea-4846-86ef-198e1bd347e4/kube-rbac-proxy-web/0.log" Jun 15 02:06:45.241158 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:45.241126 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d8dcf9d4c-n6wws_fbcce109-1cea-4846-86ef-198e1bd347e4/kube-rbac-proxy/0.log" Jun 15 02:06:45.259687 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:45.259658 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d8dcf9d4c-n6wws_fbcce109-1cea-4846-86ef-198e1bd347e4/prom-label-proxy/0.log" Jun 15 02:06:45.281070 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:45.281030 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d8dcf9d4c-n6wws_fbcce109-1cea-4846-86ef-198e1bd347e4/kube-rbac-proxy-rules/0.log" Jun 15 02:06:45.299495 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:45.299477 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d8dcf9d4c-n6wws_fbcce109-1cea-4846-86ef-198e1bd347e4/kube-rbac-proxy-metrics/0.log" Jun 15 02:06:47.239780 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:47.239745 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-b4ldg/perf-node-gather-daemonset-zrghn"] Jun 15 02:06:47.243546 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:47.243530 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-b4ldg/perf-node-gather-daemonset-zrghn" Jun 15 02:06:47.246223 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:47.246203 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-b4ldg\"/\"openshift-service-ca.crt\"" Jun 15 02:06:47.246333 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:47.246268 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-b4ldg\"/\"default-dockercfg-cr4bj\"" Jun 15 02:06:47.247415 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:47.247396 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-b4ldg\"/\"kube-root-ca.crt\"" Jun 15 02:06:47.253722 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:47.253696 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-b4ldg/perf-node-gather-daemonset-zrghn"] Jun 15 02:06:47.338930 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:47.338893 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/afee26a3-e804-4dab-97e3-a38a58419f8c-proc\") pod \"perf-node-gather-daemonset-zrghn\" (UID: \"afee26a3-e804-4dab-97e3-a38a58419f8c\") " pod="openshift-must-gather-b4ldg/perf-node-gather-daemonset-zrghn" Jun 15 02:06:47.339174 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:47.338938 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/afee26a3-e804-4dab-97e3-a38a58419f8c-podres\") pod \"perf-node-gather-daemonset-zrghn\" (UID: \"afee26a3-e804-4dab-97e3-a38a58419f8c\") " pod="openshift-must-gather-b4ldg/perf-node-gather-daemonset-zrghn" Jun 15 02:06:47.339174 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:47.338996 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/afee26a3-e804-4dab-97e3-a38a58419f8c-sys\") pod \"perf-node-gather-daemonset-zrghn\" (UID: \"afee26a3-e804-4dab-97e3-a38a58419f8c\") " pod="openshift-must-gather-b4ldg/perf-node-gather-daemonset-zrghn" Jun 15 02:06:47.339174 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:47.339011 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxncn\" (UniqueName: \"kubernetes.io/projected/afee26a3-e804-4dab-97e3-a38a58419f8c-kube-api-access-kxncn\") pod \"perf-node-gather-daemonset-zrghn\" (UID: \"afee26a3-e804-4dab-97e3-a38a58419f8c\") " pod="openshift-must-gather-b4ldg/perf-node-gather-daemonset-zrghn" Jun 15 02:06:47.339327 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:47.339171 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/afee26a3-e804-4dab-97e3-a38a58419f8c-lib-modules\") pod \"perf-node-gather-daemonset-zrghn\" (UID: \"afee26a3-e804-4dab-97e3-a38a58419f8c\") " pod="openshift-must-gather-b4ldg/perf-node-gather-daemonset-zrghn" Jun 15 02:06:47.439831 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:47.439783 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/afee26a3-e804-4dab-97e3-a38a58419f8c-proc\") pod \"perf-node-gather-daemonset-zrghn\" (UID: \"afee26a3-e804-4dab-97e3-a38a58419f8c\") " pod="openshift-must-gather-b4ldg/perf-node-gather-daemonset-zrghn" Jun 15 02:06:47.440036 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:47.439846 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/afee26a3-e804-4dab-97e3-a38a58419f8c-podres\") pod \"perf-node-gather-daemonset-zrghn\" (UID: \"afee26a3-e804-4dab-97e3-a38a58419f8c\") " pod="openshift-must-gather-b4ldg/perf-node-gather-daemonset-zrghn" Jun 15 02:06:47.440036 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:47.439908 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/afee26a3-e804-4dab-97e3-a38a58419f8c-sys\") pod \"perf-node-gather-daemonset-zrghn\" (UID: \"afee26a3-e804-4dab-97e3-a38a58419f8c\") " pod="openshift-must-gather-b4ldg/perf-node-gather-daemonset-zrghn" Jun 15 02:06:47.440036 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:47.439919 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/afee26a3-e804-4dab-97e3-a38a58419f8c-proc\") pod \"perf-node-gather-daemonset-zrghn\" (UID: \"afee26a3-e804-4dab-97e3-a38a58419f8c\") " pod="openshift-must-gather-b4ldg/perf-node-gather-daemonset-zrghn" Jun 15 02:06:47.440036 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:47.439931 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxncn\" (UniqueName: \"kubernetes.io/projected/afee26a3-e804-4dab-97e3-a38a58419f8c-kube-api-access-kxncn\") pod \"perf-node-gather-daemonset-zrghn\" (UID: \"afee26a3-e804-4dab-97e3-a38a58419f8c\") " pod="openshift-must-gather-b4ldg/perf-node-gather-daemonset-zrghn" Jun 15 02:06:47.440036 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:47.439965 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/afee26a3-e804-4dab-97e3-a38a58419f8c-podres\") pod \"perf-node-gather-daemonset-zrghn\" (UID: \"afee26a3-e804-4dab-97e3-a38a58419f8c\") " pod="openshift-must-gather-b4ldg/perf-node-gather-daemonset-zrghn" Jun 15 02:06:47.440036 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:47.440030 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/afee26a3-e804-4dab-97e3-a38a58419f8c-lib-modules\") pod \"perf-node-gather-daemonset-zrghn\" (UID: \"afee26a3-e804-4dab-97e3-a38a58419f8c\") " pod="openshift-must-gather-b4ldg/perf-node-gather-daemonset-zrghn" Jun 15 02:06:47.440364 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:47.440080 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/afee26a3-e804-4dab-97e3-a38a58419f8c-sys\") pod \"perf-node-gather-daemonset-zrghn\" (UID: \"afee26a3-e804-4dab-97e3-a38a58419f8c\") " pod="openshift-must-gather-b4ldg/perf-node-gather-daemonset-zrghn" Jun 15 02:06:47.440364 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:47.440224 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/afee26a3-e804-4dab-97e3-a38a58419f8c-lib-modules\") pod \"perf-node-gather-daemonset-zrghn\" (UID: \"afee26a3-e804-4dab-97e3-a38a58419f8c\") " pod="openshift-must-gather-b4ldg/perf-node-gather-daemonset-zrghn" Jun 15 02:06:47.447991 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:47.447968 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxncn\" (UniqueName: \"kubernetes.io/projected/afee26a3-e804-4dab-97e3-a38a58419f8c-kube-api-access-kxncn\") pod \"perf-node-gather-daemonset-zrghn\" (UID: \"afee26a3-e804-4dab-97e3-a38a58419f8c\") " pod="openshift-must-gather-b4ldg/perf-node-gather-daemonset-zrghn" Jun 15 02:06:47.555385 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:47.555276 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-b4ldg/perf-node-gather-daemonset-zrghn" Jun 15 02:06:47.696961 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:47.696924 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-b4ldg/perf-node-gather-daemonset-zrghn"] Jun 15 02:06:47.697346 ip-10-0-143-196 kubenswrapper[2578]: W0615 02:06:47.697311 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podafee26a3_e804_4dab_97e3_a38a58419f8c.slice/crio-932eeb8d40c392728df424f6c60e33b0503f6da87396685c67b68355e7dacd1e WatchSource:0}: Error finding container 932eeb8d40c392728df424f6c60e33b0503f6da87396685c67b68355e7dacd1e: Status 404 returned error can't find the container with id 932eeb8d40c392728df424f6c60e33b0503f6da87396685c67b68355e7dacd1e Jun 15 02:06:47.760707 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:47.760680 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6798ff8d9c-bx88t_8bec6edd-4845-4d89-8532-e95b73315323/console/0.log" Jun 15 02:06:47.948239 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:47.948192 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b4ldg/perf-node-gather-daemonset-zrghn" event={"ID":"afee26a3-e804-4dab-97e3-a38a58419f8c","Type":"ContainerStarted","Data":"222aa055dc8c0488acba08882c7c8275c6d9b519951908b78acfb29ac3ee8006"} Jun 15 02:06:47.948457 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:47.948288 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b4ldg/perf-node-gather-daemonset-zrghn" event={"ID":"afee26a3-e804-4dab-97e3-a38a58419f8c","Type":"ContainerStarted","Data":"932eeb8d40c392728df424f6c60e33b0503f6da87396685c67b68355e7dacd1e"} Jun 15 02:06:47.948457 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:47.948314 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-b4ldg/perf-node-gather-daemonset-zrghn" Jun 15 02:06:47.966744 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:47.966689 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-b4ldg/perf-node-gather-daemonset-zrghn" podStartSLOduration=0.966673336 podStartE2EDuration="966.673336ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 02:06:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 02:06:47.96401006 +0000 UTC m=+1331.677736032" watchObservedRunningTime="2026-06-15 02:06:47.966673336 +0000 UTC m=+1331.680399308" Jun 15 02:06:49.481331 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:49.481303 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bmrhc_7f6afa77-1931-4efb-aeae-7a7fdef97959/dns/0.log" Jun 15 02:06:49.499531 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:49.499506 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bmrhc_7f6afa77-1931-4efb-aeae-7a7fdef97959/kube-rbac-proxy/0.log" Jun 15 02:06:49.650173 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:49.650078 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pmkl7_3cf2d35f-dae7-42c6-8f35-5b0ccb5e4750/dns-node-resolver/0.log" Jun 15 02:06:50.227247 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:50.227217 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-r8dqf_17743aa6-c9cb-458c-b3d3-16c6a774171a/node-ca/0.log" Jun 15 02:06:51.161929 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:51.161890 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-4nlft_b50e4740-5402-4abc-8737-a2aebb7e69c7/discovery/0.log" Jun 15 02:06:51.182998 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:51.182970 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-644c7647f-gszx8_0fcdf2ea-d316-456b-8f67-3454dec3558f/kube-auth-proxy/0.log" Jun 15 02:06:51.245979 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:51.245952 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-579785bc64-clpvv_f843f2ab-3bc0-42e0-88dc-a17d7d1443a7/payload-pre-processing/0.log" Jun 15 02:06:51.267147 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:51.267114 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-699b6cd6c-pl44j_9f01c072-54ff-4306-8403-72eec96b09ef/payload-processing/0.log" Jun 15 02:06:51.290643 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:51.290614 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-58cf549cb4-mc482_d4979e05-89c5-482b-a895-4f1f9ae54e93/router/0.log" Jun 15 02:06:51.848699 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:51.848671 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-48xtp_878da7f4-63c4-45b4-a4c6-fc6a72556516/serve-healthcheck-canary/0.log" Jun 15 02:06:52.463833 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:52.463804 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lgxjj_47a7fc6e-077c-4eb7-9850-a31a72bec0d1/kube-rbac-proxy/0.log" Jun 15 02:06:52.482902 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:52.482872 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lgxjj_47a7fc6e-077c-4eb7-9850-a31a72bec0d1/exporter/0.log" Jun 15 02:06:52.501741 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:52.501716 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lgxjj_47a7fc6e-077c-4eb7-9850-a31a72bec0d1/extractor/0.log" Jun 15 02:06:53.964721 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:53.964693 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-b4ldg/perf-node-gather-daemonset-zrghn" Jun 15 02:06:54.718588 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:54.718561 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-79b56865f7-btnp2_f890d1a2-5c4a-4eac-bd8f-8f7c11c07a57/nginx/0.log" Jun 15 02:06:54.763933 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:54.763906 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-69fc866d76-4mzv8_019c220f-42d7-4691-b5d4-183b9f27c964/manager/0.log" Jun 15 02:06:54.799591 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:54.799551 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-864bfd7bc9-6z22w_f694f872-296b-4205-98ef-d03f361e25ae/manager/0.log" Jun 15 02:06:54.818549 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:54.818526 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-bb7bb5956-bh2nv_0bdec198-1402-4d1a-91c8-5be0b4383a40/maas-api/0.log" Jun 15 02:06:54.881633 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:54.881609 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-5dd488fc48-l7wlm_40c9ba7d-66ce-49f9-a472-5445a57d1968/server/0.log" Jun 15 02:06:54.924052 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:54.924019 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-6dc94f689f-jntw9_4cb41eb9-34d4-495b-8534-0a7b7f4b42d2/manager/0.log" Jun 15 02:06:54.995849 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:54.995764 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d494fbff9-9ndp7_729a1b01-1fa0-44fb-88e2-c23c5cad4deb/manager/0.log" Jun 15 02:06:55.013386 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:55.013358 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d494fbff9-9ndp7_729a1b01-1fa0-44fb-88e2-c23c5cad4deb/copy-manifests/0.log" Jun 15 02:06:55.136103 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:06:55.136064 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-mg9dq_e8bb261a-c997-4d6a-a00a-9a009562f23c/postgres/0.log" Jun 15 02:07:01.870309 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:07:01.870274 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-4nlft_b50e4740-5402-4abc-8737-a2aebb7e69c7/discovery/0.log" Jun 15 02:07:01.880532 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:07:01.880509 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-644c7647f-gszx8_0fcdf2ea-d316-456b-8f67-3454dec3558f/kube-auth-proxy/0.log" Jun 15 02:07:01.926540 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:07:01.926510 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-579785bc64-clpvv_f843f2ab-3bc0-42e0-88dc-a17d7d1443a7/payload-pre-processing/0.log" Jun 15 02:07:01.946479 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:07:01.946455 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-699b6cd6c-pl44j_9f01c072-54ff-4306-8403-72eec96b09ef/payload-processing/0.log" Jun 15 02:07:01.964112 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:07:01.964074 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-58cf549cb4-mc482_d4979e05-89c5-482b-a895-4f1f9ae54e93/router/0.log" Jun 15 02:07:03.151077 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:07:03.151041 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-4nlft_b50e4740-5402-4abc-8737-a2aebb7e69c7/discovery/0.log" Jun 15 02:07:03.162302 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:07:03.162280 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-644c7647f-gszx8_0fcdf2ea-d316-456b-8f67-3454dec3558f/kube-auth-proxy/0.log" Jun 15 02:07:03.214514 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:07:03.214485 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-579785bc64-clpvv_f843f2ab-3bc0-42e0-88dc-a17d7d1443a7/payload-pre-processing/0.log" Jun 15 02:07:03.229263 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:07:03.229238 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-699b6cd6c-pl44j_9f01c072-54ff-4306-8403-72eec96b09ef/payload-processing/0.log" Jun 15 02:07:03.246919 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:07:03.246895 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-58cf549cb4-mc482_d4979e05-89c5-482b-a895-4f1f9ae54e93/router/0.log" Jun 15 02:07:03.603117 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:07:03.603028 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-4xrm5_134d75e6-f2f5-4264-9f35-c231e06bdbb9/kube-multus/0.log" Jun 15 02:07:03.625833 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:07:03.625810 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8r9kz_9a743adf-5ee6-4357-9206-1ccada80d911/kube-multus-additional-cni-plugins/0.log" Jun 15 02:07:03.644794 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:07:03.644767 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8r9kz_9a743adf-5ee6-4357-9206-1ccada80d911/egress-router-binary-copy/0.log" Jun 15 02:07:03.665139 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:07:03.665114 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8r9kz_9a743adf-5ee6-4357-9206-1ccada80d911/cni-plugins/0.log" Jun 15 02:07:03.684110 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:07:03.684068 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8r9kz_9a743adf-5ee6-4357-9206-1ccada80d911/bond-cni-plugin/0.log" Jun 15 02:07:03.702346 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:07:03.702320 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8r9kz_9a743adf-5ee6-4357-9206-1ccada80d911/routeoverride-cni/0.log" Jun 15 02:07:03.720812 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:07:03.720784 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8r9kz_9a743adf-5ee6-4357-9206-1ccada80d911/whereabouts-cni-bincopy/0.log" Jun 15 02:07:03.742010 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:07:03.741983 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8r9kz_9a743adf-5ee6-4357-9206-1ccada80d911/whereabouts-cni/0.log" Jun 15 02:07:04.098322 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:07:04.098295 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-n4p82_e1719195-2a1b-4c96-abe3-c25bda154d3d/network-metrics-daemon/0.log" Jun 15 02:07:04.115870 ip-10-0-143-196 kubenswrapper[2578]: I0615 02:07:04.115842 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-n4p82_e1719195-2a1b-4c96-abe3-c25bda154d3d/kube-rbac-proxy/0.log"