Jun 15 02:26:14.446252 ip-10-0-143-51 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jun 15 02:26:14.446262 ip-10-0-143-51 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jun 15 02:26:14.446269 ip-10-0-143-51 systemd[1]: kubelet.service: Failed with result 'resources'. Jun 15 02:26:14.446506 ip-10-0-143-51 systemd[1]: Failed to start Kubernetes Kubelet. Jun 15 02:26:24.453990 ip-10-0-143-51 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jun 15 02:26:24.454004 ip-10-0-143-51 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 6694816bf0a94ea38d2bd9ead3a53dd2 -- Jun 15 02:28:27.040999 ip-10-0-143-51 systemd[1]: Starting Kubernetes Kubelet... Jun 15 02:28:27.345080 ip-10-0-143-51 kubenswrapper[2579]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 15 02:28:27.345080 ip-10-0-143-51 kubenswrapper[2579]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 15 02:28:27.345080 ip-10-0-143-51 kubenswrapper[2579]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 15 02:28:27.345080 ip-10-0-143-51 kubenswrapper[2579]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 15 02:28:27.345080 ip-10-0-143-51 kubenswrapper[2579]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 15 02:28:27.346423 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.346355 2579 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 15 02:28:27.353546 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353526 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 15 02:28:27.353546 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353544 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 15 02:28:27.353546 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353549 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 15 02:28:27.353753 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353553 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 15 02:28:27.353753 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353557 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 15 02:28:27.353753 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353562 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 15 02:28:27.353753 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353566 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 15 02:28:27.353753 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353570 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 15 02:28:27.353753 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353574 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 15 02:28:27.353753 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353577 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 15 02:28:27.353753 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353581 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 15 02:28:27.353753 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353585 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 15 02:28:27.353753 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353591 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 15 02:28:27.353753 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353596 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 15 02:28:27.353753 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353600 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 15 02:28:27.353753 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353604 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 15 02:28:27.353753 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353608 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 15 02:28:27.353753 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353612 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 15 02:28:27.353753 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353616 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 15 02:28:27.353753 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353621 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 15 02:28:27.353753 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353625 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 15 02:28:27.353753 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353658 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 15 02:28:27.353753 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353663 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 15 02:28:27.354567 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353668 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 15 02:28:27.354567 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353671 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 15 02:28:27.354567 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353676 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 15 02:28:27.354567 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353679 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 15 02:28:27.354567 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353684 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 15 02:28:27.354567 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353688 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 15 02:28:27.354567 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353692 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 15 02:28:27.354567 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353698 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 15 02:28:27.354567 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353702 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 15 02:28:27.354567 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353708 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 15 02:28:27.354567 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353712 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 15 02:28:27.354567 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353716 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 15 02:28:27.354567 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353720 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 15 02:28:27.354567 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353725 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 15 02:28:27.354567 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353730 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 15 02:28:27.354567 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353734 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 15 02:28:27.354567 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353738 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 15 02:28:27.354567 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353742 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 15 02:28:27.354567 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353746 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 15 02:28:27.355377 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353750 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 15 02:28:27.355377 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353754 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 15 02:28:27.355377 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353759 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 15 02:28:27.355377 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353764 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 15 02:28:27.355377 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353769 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 15 02:28:27.355377 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353773 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 15 02:28:27.355377 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353777 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 15 02:28:27.355377 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353782 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 15 02:28:27.355377 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353787 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 15 02:28:27.355377 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353791 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 15 02:28:27.355377 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353798 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 15 02:28:27.355377 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353805 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 15 02:28:27.355377 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353810 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 15 02:28:27.355377 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353815 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 15 02:28:27.355377 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353820 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 15 02:28:27.355377 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353824 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 15 02:28:27.355377 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353829 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 15 02:28:27.355377 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353834 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 15 02:28:27.355377 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353838 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 15 02:28:27.355377 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353843 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 15 02:28:27.356220 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353847 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 15 02:28:27.356220 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353851 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 15 02:28:27.356220 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353855 2579 feature_gate.go:328] unrecognized feature gate: Example Jun 15 02:28:27.356220 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353859 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 15 02:28:27.356220 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353863 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 15 02:28:27.356220 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353868 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 15 02:28:27.356220 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353872 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 15 02:28:27.356220 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353877 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 15 02:28:27.356220 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353881 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 15 02:28:27.356220 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353885 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jun 15 02:28:27.356220 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353889 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 15 02:28:27.356220 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353893 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 15 02:28:27.356220 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353899 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 15 02:28:27.356220 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353903 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 15 02:28:27.356220 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353908 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 15 02:28:27.356220 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353912 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 15 02:28:27.356220 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353917 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 15 02:28:27.356220 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353921 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 15 02:28:27.356220 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353925 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 15 02:28:27.356220 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353930 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 15 02:28:27.356220 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353934 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 15 02:28:27.357225 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353939 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 15 02:28:27.357225 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353944 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 15 02:28:27.357225 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353948 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 15 02:28:27.357225 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353952 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 15 02:28:27.357225 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353956 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 15 02:28:27.357225 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353960 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 15 02:28:27.357225 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353965 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 15 02:28:27.357225 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353969 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 15 02:28:27.357225 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353975 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 15 02:28:27.357225 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353980 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 15 02:28:27.357225 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353985 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 15 02:28:27.357225 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353989 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 15 02:28:27.357225 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353993 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 15 02:28:27.357225 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.353998 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 15 02:28:27.357225 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.354002 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 15 02:28:27.357225 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.354006 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 15 02:28:27.357225 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355139 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 15 02:28:27.357225 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355151 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 15 02:28:27.357225 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355156 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 15 02:28:27.358106 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355160 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 15 02:28:27.358106 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355165 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 15 02:28:27.358106 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355169 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 15 02:28:27.358106 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355173 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 15 02:28:27.358106 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355177 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 15 02:28:27.358106 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355181 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 15 02:28:27.358106 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355186 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 15 02:28:27.358106 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355190 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 15 02:28:27.358106 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355194 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 15 02:28:27.358106 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355198 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 15 02:28:27.358106 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355202 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 15 02:28:27.358106 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355207 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 15 02:28:27.358106 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355211 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 15 02:28:27.358106 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355215 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 15 02:28:27.358106 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355220 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 15 02:28:27.358106 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355225 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 15 02:28:27.358106 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355229 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 15 02:28:27.358106 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355233 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 15 02:28:27.358106 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355237 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 15 02:28:27.358106 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355241 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 15 02:28:27.358699 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355245 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 15 02:28:27.358699 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355249 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 15 02:28:27.358699 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355254 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 15 02:28:27.358699 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355258 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 15 02:28:27.358699 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355262 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 15 02:28:27.358699 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355266 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 15 02:28:27.358699 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355271 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 15 02:28:27.358699 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355275 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 15 02:28:27.358699 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355280 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 15 02:28:27.358699 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355284 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 15 02:28:27.358699 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355288 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 15 02:28:27.358699 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355292 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 15 02:28:27.358699 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355297 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 15 02:28:27.358699 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355301 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 15 02:28:27.358699 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355305 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 15 02:28:27.358699 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355309 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 15 02:28:27.358699 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355313 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 15 02:28:27.358699 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355318 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 15 02:28:27.358699 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355322 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 15 02:28:27.358699 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355326 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 15 02:28:27.359314 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355330 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 15 02:28:27.359314 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355334 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 15 02:28:27.359314 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355338 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 15 02:28:27.359314 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355342 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 15 02:28:27.359314 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355346 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 15 02:28:27.359314 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355351 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 15 02:28:27.359314 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355355 2579 feature_gate.go:328] unrecognized feature gate: Example Jun 15 02:28:27.359314 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355359 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 15 02:28:27.359314 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355364 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 15 02:28:27.359314 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355368 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 15 02:28:27.359314 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355372 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 15 02:28:27.359314 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355376 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 15 02:28:27.359314 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355380 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 15 02:28:27.359314 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355384 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 15 02:28:27.359314 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355388 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 15 02:28:27.359314 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355394 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 15 02:28:27.359314 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355401 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 15 02:28:27.359314 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355407 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 15 02:28:27.359314 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355411 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 15 02:28:27.359314 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355421 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 15 02:28:27.359979 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355426 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 15 02:28:27.359979 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355430 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 15 02:28:27.359979 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355435 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 15 02:28:27.359979 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355440 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 15 02:28:27.359979 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355444 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 15 02:28:27.359979 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355448 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 15 02:28:27.359979 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355453 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 15 02:28:27.359979 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355457 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 15 02:28:27.359979 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355463 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 15 02:28:27.359979 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355470 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 15 02:28:27.359979 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355475 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 15 02:28:27.359979 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355479 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 15 02:28:27.359979 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355485 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 15 02:28:27.359979 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355492 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 15 02:28:27.359979 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355497 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 15 02:28:27.359979 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355501 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 15 02:28:27.359979 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355506 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 15 02:28:27.359979 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355510 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 15 02:28:27.359979 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355515 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 15 02:28:27.360449 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355519 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 15 02:28:27.360449 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355525 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 15 02:28:27.360449 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355529 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 15 02:28:27.360449 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355533 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jun 15 02:28:27.360449 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355538 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 15 02:28:27.360449 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355543 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 15 02:28:27.360449 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355547 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 15 02:28:27.360449 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355552 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 15 02:28:27.360449 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355556 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 15 02:28:27.360449 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355560 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 15 02:28:27.360449 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355564 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 15 02:28:27.360449 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355568 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 15 02:28:27.360449 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355574 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 15 02:28:27.360449 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355579 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 15 02:28:27.360449 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355583 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 15 02:28:27.360449 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355587 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 15 02:28:27.360449 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.355592 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 15 02:28:27.360449 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356205 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 15 02:28:27.360449 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356214 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 15 02:28:27.360449 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356218 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 15 02:28:27.361089 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356222 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 15 02:28:27.361089 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356227 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 15 02:28:27.361089 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356231 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 15 02:28:27.361089 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356235 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 15 02:28:27.361089 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356240 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 15 02:28:27.361089 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356244 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 15 02:28:27.361089 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356249 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 15 02:28:27.361089 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356253 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 15 02:28:27.361089 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356258 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 15 02:28:27.361089 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356263 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 15 02:28:27.361089 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356267 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 15 02:28:27.361089 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356271 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 15 02:28:27.361089 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356276 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 15 02:28:27.361089 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356281 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 15 02:28:27.361089 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356286 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 15 02:28:27.361089 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356290 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 15 02:28:27.361089 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356294 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 15 02:28:27.361089 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356299 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 15 02:28:27.361089 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356303 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 15 02:28:27.361089 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356307 2579 feature_gate.go:328] unrecognized feature gate: Example Jun 15 02:28:27.361645 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356311 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 15 02:28:27.361645 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356316 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 15 02:28:27.361645 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356320 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 15 02:28:27.361645 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356324 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 15 02:28:27.361645 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356329 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 15 02:28:27.361645 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356332 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 15 02:28:27.361645 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356336 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 15 02:28:27.361645 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356341 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 15 02:28:27.361645 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356345 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 15 02:28:27.361645 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356349 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 15 02:28:27.361645 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356353 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 15 02:28:27.361645 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356358 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 15 02:28:27.361645 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356362 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 15 02:28:27.361645 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356366 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 15 02:28:27.361645 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356370 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 15 02:28:27.361645 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356374 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 15 02:28:27.361645 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356378 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 15 02:28:27.361645 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356382 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 15 02:28:27.361645 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356386 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 15 02:28:27.361645 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356392 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 15 02:28:27.362144 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356396 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 15 02:28:27.362144 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356400 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 15 02:28:27.362144 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356404 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jun 15 02:28:27.362144 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356409 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 15 02:28:27.362144 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356413 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 15 02:28:27.362144 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356417 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 15 02:28:27.362144 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356422 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 15 02:28:27.362144 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356426 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 15 02:28:27.362144 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356430 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 15 02:28:27.362144 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356434 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 15 02:28:27.362144 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356438 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 15 02:28:27.362144 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356442 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 15 02:28:27.362144 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356446 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 15 02:28:27.362144 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356451 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 15 02:28:27.362144 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356455 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 15 02:28:27.362144 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356458 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 15 02:28:27.362144 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356463 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 15 02:28:27.362144 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356468 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 15 02:28:27.362144 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356472 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 15 02:28:27.362144 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356479 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 15 02:28:27.362658 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356484 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 15 02:28:27.362658 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356490 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 15 02:28:27.362658 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356496 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 15 02:28:27.362658 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356500 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 15 02:28:27.362658 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356505 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 15 02:28:27.362658 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356509 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 15 02:28:27.362658 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356513 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 15 02:28:27.362658 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356518 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 15 02:28:27.362658 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356525 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 15 02:28:27.362658 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356529 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 15 02:28:27.362658 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356533 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 15 02:28:27.362658 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356538 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 15 02:28:27.362658 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356542 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 15 02:28:27.362658 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356546 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 15 02:28:27.362658 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356550 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 15 02:28:27.362658 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356555 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 15 02:28:27.362658 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356559 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 15 02:28:27.362658 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356563 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 15 02:28:27.362658 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356567 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 15 02:28:27.363129 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356571 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 15 02:28:27.363129 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356575 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 15 02:28:27.363129 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356579 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 15 02:28:27.363129 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356583 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 15 02:28:27.363129 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356587 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 15 02:28:27.363129 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356592 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 15 02:28:27.363129 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356596 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 15 02:28:27.363129 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356600 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 15 02:28:27.363129 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356604 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 15 02:28:27.363129 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356609 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 15 02:28:27.363129 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356613 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 15 02:28:27.363129 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356617 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 15 02:28:27.363129 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356623 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 15 02:28:27.363129 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356648 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 15 02:28:27.363129 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356653 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 15 02:28:27.363129 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356657 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 15 02:28:27.363129 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.356662 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 15 02:28:27.363129 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.356757 2579 flags.go:64] FLAG: --address="0.0.0.0" Jun 15 02:28:27.363129 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.356767 2579 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 15 02:28:27.363129 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.356777 2579 flags.go:64] FLAG: --anonymous-auth="true" Jun 15 02:28:27.363688 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.356784 2579 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 15 02:28:27.363688 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.356793 2579 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 15 02:28:27.363688 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.356798 2579 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 15 02:28:27.363688 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.356805 2579 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 15 02:28:27.363688 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.356812 2579 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 15 02:28:27.363688 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.356817 2579 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 15 02:28:27.363688 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.356822 2579 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 15 02:28:27.363688 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.356828 2579 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 15 02:28:27.363688 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.356833 2579 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 15 02:28:27.363688 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.356838 2579 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 15 02:28:27.363688 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.356843 2579 flags.go:64] FLAG: --cgroup-root="" Jun 15 02:28:27.363688 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.356847 2579 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 15 02:28:27.363688 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.356852 2579 flags.go:64] FLAG: --client-ca-file="" Jun 15 02:28:27.363688 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.356857 2579 flags.go:64] FLAG: --cloud-provider="external" Jun 15 02:28:27.363688 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.356862 2579 flags.go:64] FLAG: --cluster-dns="[]" Jun 15 02:28:27.363688 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.356868 2579 flags.go:64] FLAG: --cluster-domain="" Jun 15 02:28:27.363688 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.356873 2579 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 15 02:28:27.363688 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.356878 2579 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jun 15 02:28:27.363688 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.356883 2579 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 15 02:28:27.363688 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.356888 2579 flags.go:64] FLAG: --container-log-max-files="5" Jun 15 02:28:27.363688 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.356893 2579 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 15 02:28:27.363688 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.356898 2579 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 15 02:28:27.363688 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.356903 2579 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 15 02:28:27.364241 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.356909 2579 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 15 02:28:27.364241 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.356913 2579 flags.go:64] FLAG: --contention-profiling="false" Jun 15 02:28:27.364241 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.356918 2579 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 15 02:28:27.364241 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.356922 2579 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 15 02:28:27.364241 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.356927 2579 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 15 02:28:27.364241 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.356931 2579 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 15 02:28:27.364241 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.356938 2579 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 15 02:28:27.364241 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.356943 2579 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 15 02:28:27.364241 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.356948 2579 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 15 02:28:27.364241 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.356952 2579 flags.go:64] FLAG: --enable-load-reader="false" Jun 15 02:28:27.364241 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.356958 2579 flags.go:64] FLAG: --enable-server="true" Jun 15 02:28:27.364241 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.356963 2579 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 15 02:28:27.364241 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.356970 2579 flags.go:64] FLAG: --event-burst="100" Jun 15 02:28:27.364241 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.356975 2579 flags.go:64] FLAG: --event-qps="50" Jun 15 02:28:27.364241 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.356979 2579 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 15 02:28:27.364241 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.356985 2579 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 15 02:28:27.364241 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.356989 2579 flags.go:64] FLAG: --eviction-hard="" Jun 15 02:28:27.364241 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.356995 2579 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 15 02:28:27.364241 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357000 2579 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 15 02:28:27.364241 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357004 2579 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 15 02:28:27.364241 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357009 2579 flags.go:64] FLAG: --eviction-soft="" Jun 15 02:28:27.364241 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357013 2579 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 15 02:28:27.364241 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357018 2579 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 15 02:28:27.364241 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357024 2579 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 15 02:28:27.364241 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357028 2579 flags.go:64] FLAG: --experimental-mounter-path="" Jun 15 02:28:27.364945 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357033 2579 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 15 02:28:27.364945 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357037 2579 flags.go:64] FLAG: --fail-swap-on="true" Jun 15 02:28:27.364945 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357042 2579 flags.go:64] FLAG: --feature-gates="" Jun 15 02:28:27.364945 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357048 2579 flags.go:64] FLAG: --file-check-frequency="20s" Jun 15 02:28:27.364945 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357052 2579 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 15 02:28:27.364945 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357059 2579 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 15 02:28:27.364945 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357064 2579 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 15 02:28:27.364945 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357069 2579 flags.go:64] FLAG: --healthz-port="10248" Jun 15 02:28:27.364945 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357073 2579 flags.go:64] FLAG: --help="false" Jun 15 02:28:27.364945 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357078 2579 flags.go:64] FLAG: --hostname-override="ip-10-0-143-51.ec2.internal" Jun 15 02:28:27.364945 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357083 2579 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 15 02:28:27.364945 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357089 2579 flags.go:64] FLAG: --http-check-frequency="20s" Jun 15 02:28:27.364945 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357093 2579 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 15 02:28:27.364945 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357098 2579 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 15 02:28:27.364945 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357104 2579 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 15 02:28:27.364945 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357109 2579 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 15 02:28:27.364945 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357113 2579 flags.go:64] FLAG: --image-service-endpoint="" Jun 15 02:28:27.364945 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357119 2579 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 15 02:28:27.364945 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357123 2579 flags.go:64] FLAG: --kube-api-burst="100" Jun 15 02:28:27.364945 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357127 2579 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 15 02:28:27.364945 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357131 2579 flags.go:64] FLAG: --kube-api-qps="50" Jun 15 02:28:27.364945 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357136 2579 flags.go:64] FLAG: --kube-reserved="" Jun 15 02:28:27.364945 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357140 2579 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 15 02:28:27.364945 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357144 2579 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 15 02:28:27.365539 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357148 2579 flags.go:64] FLAG: --kubelet-cgroups="" Jun 15 02:28:27.365539 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357153 2579 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 15 02:28:27.365539 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357157 2579 flags.go:64] FLAG: --lock-file="" Jun 15 02:28:27.365539 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357161 2579 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 15 02:28:27.365539 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357166 2579 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 15 02:28:27.365539 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357170 2579 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 15 02:28:27.365539 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357178 2579 flags.go:64] FLAG: --log-json-split-stream="false" Jun 15 02:28:27.365539 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357183 2579 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 15 02:28:27.365539 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357187 2579 flags.go:64] FLAG: --log-text-split-stream="false" Jun 15 02:28:27.365539 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357192 2579 flags.go:64] FLAG: --logging-format="text" Jun 15 02:28:27.365539 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357196 2579 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 15 02:28:27.365539 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357202 2579 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 15 02:28:27.365539 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357206 2579 flags.go:64] FLAG: --manifest-url="" Jun 15 02:28:27.365539 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357212 2579 flags.go:64] FLAG: --manifest-url-header="" Jun 15 02:28:27.365539 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357219 2579 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 15 02:28:27.365539 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357223 2579 flags.go:64] FLAG: --max-open-files="1000000" Jun 15 02:28:27.365539 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357230 2579 flags.go:64] FLAG: --max-pods="110" Jun 15 02:28:27.365539 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357234 2579 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 15 02:28:27.365539 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357239 2579 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 15 02:28:27.365539 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357243 2579 flags.go:64] FLAG: --memory-manager-policy="None" Jun 15 02:28:27.365539 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357247 2579 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 15 02:28:27.365539 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357251 2579 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 15 02:28:27.365539 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357256 2579 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 15 02:28:27.365539 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357261 2579 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 15 02:28:27.365539 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357272 2579 flags.go:64] FLAG: --node-status-max-images="50" Jun 15 02:28:27.366172 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357278 2579 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 15 02:28:27.366172 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357283 2579 flags.go:64] FLAG: --oom-score-adj="-999" Jun 15 02:28:27.366172 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357287 2579 flags.go:64] FLAG: --pod-cidr="" Jun 15 02:28:27.366172 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357292 2579 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4a2e90c8ef7d783e4d635e24fb09435f5e1f6e954583512c0944f9e57bc77b52" Jun 15 02:28:27.366172 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357301 2579 flags.go:64] FLAG: --pod-manifest-path="" Jun 15 02:28:27.366172 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357305 2579 flags.go:64] FLAG: --pod-max-pids="-1" Jun 15 02:28:27.366172 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357310 2579 flags.go:64] FLAG: --pods-per-core="0" Jun 15 02:28:27.366172 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357314 2579 flags.go:64] FLAG: --port="10250" Jun 15 02:28:27.366172 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357319 2579 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 15 02:28:27.366172 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357324 2579 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0e713fdbfa2ecbeae" Jun 15 02:28:27.366172 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357329 2579 flags.go:64] FLAG: --qos-reserved="" Jun 15 02:28:27.366172 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357334 2579 flags.go:64] FLAG: --read-only-port="10255" Jun 15 02:28:27.366172 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357338 2579 flags.go:64] FLAG: --register-node="true" Jun 15 02:28:27.366172 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357343 2579 flags.go:64] FLAG: --register-with-taints="" Jun 15 02:28:27.366172 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357349 2579 flags.go:64] FLAG: --registry-burst="10" Jun 15 02:28:27.366172 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357355 2579 flags.go:64] FLAG: --registry-qps="5" Jun 15 02:28:27.366172 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357359 2579 flags.go:64] FLAG: --reserved-cpus="" Jun 15 02:28:27.366172 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357364 2579 flags.go:64] FLAG: --reserved-memory="" Jun 15 02:28:27.366172 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357370 2579 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 15 02:28:27.366172 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357375 2579 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 15 02:28:27.366172 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357380 2579 flags.go:64] FLAG: --rotate-certificates="false" Jun 15 02:28:27.366172 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357386 2579 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 15 02:28:27.366172 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357391 2579 flags.go:64] FLAG: --runonce="false" Jun 15 02:28:27.366172 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357395 2579 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 15 02:28:27.366172 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357400 2579 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 15 02:28:27.366799 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357405 2579 flags.go:64] FLAG: --seccomp-default="false" Jun 15 02:28:27.366799 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357409 2579 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 15 02:28:27.366799 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357414 2579 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 15 02:28:27.366799 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357418 2579 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 15 02:28:27.366799 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357424 2579 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 15 02:28:27.366799 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357429 2579 flags.go:64] FLAG: --storage-driver-password="root" Jun 15 02:28:27.366799 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357434 2579 flags.go:64] FLAG: --storage-driver-secure="false" Jun 15 02:28:27.366799 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357440 2579 flags.go:64] FLAG: --storage-driver-table="stats" Jun 15 02:28:27.366799 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357445 2579 flags.go:64] FLAG: --storage-driver-user="root" Jun 15 02:28:27.366799 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357449 2579 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 15 02:28:27.366799 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357454 2579 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 15 02:28:27.366799 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357458 2579 flags.go:64] FLAG: --system-cgroups="" Jun 15 02:28:27.366799 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357463 2579 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 15 02:28:27.366799 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357472 2579 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 15 02:28:27.366799 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357476 2579 flags.go:64] FLAG: --tls-cert-file="" Jun 15 02:28:27.366799 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357480 2579 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 15 02:28:27.366799 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357487 2579 flags.go:64] FLAG: --tls-min-version="" Jun 15 02:28:27.366799 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357491 2579 flags.go:64] FLAG: --tls-private-key-file="" Jun 15 02:28:27.366799 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357496 2579 flags.go:64] FLAG: --topology-manager-policy="none" Jun 15 02:28:27.366799 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357501 2579 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 15 02:28:27.366799 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357505 2579 flags.go:64] FLAG: --topology-manager-scope="container" Jun 15 02:28:27.366799 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357510 2579 flags.go:64] FLAG: --v="2" Jun 15 02:28:27.366799 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357519 2579 flags.go:64] FLAG: --version="false" Jun 15 02:28:27.366799 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357534 2579 flags.go:64] FLAG: --vmodule="" Jun 15 02:28:27.366799 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357539 2579 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 15 02:28:27.367409 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.357544 2579 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 15 02:28:27.367409 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.357742 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 15 02:28:27.367409 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.357751 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 15 02:28:27.367409 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.357758 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 15 02:28:27.367409 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.357762 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 15 02:28:27.367409 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.357767 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 15 02:28:27.367409 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.357771 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 15 02:28:27.367409 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.357775 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 15 02:28:27.367409 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.357779 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 15 02:28:27.367409 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.357784 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 15 02:28:27.367409 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.357789 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 15 02:28:27.367409 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.357793 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 15 02:28:27.367409 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.357797 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 15 02:28:27.367409 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.357801 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 15 02:28:27.367409 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.357809 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 15 02:28:27.367409 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.357813 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 15 02:28:27.367409 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.357817 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 15 02:28:27.367409 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.357821 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 15 02:28:27.367409 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.357826 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 15 02:28:27.367409 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.357830 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 15 02:28:27.367913 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.357834 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 15 02:28:27.367913 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.357839 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 15 02:28:27.367913 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.357843 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 15 02:28:27.367913 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.357847 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 15 02:28:27.367913 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.357851 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 15 02:28:27.367913 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.357855 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 15 02:28:27.367913 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.357859 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 15 02:28:27.367913 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.357863 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 15 02:28:27.367913 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.357868 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 15 02:28:27.367913 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.357872 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 15 02:28:27.367913 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.357876 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 15 02:28:27.367913 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.357881 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 15 02:28:27.367913 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.357885 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 15 02:28:27.367913 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.357889 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 15 02:28:27.367913 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.357893 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 15 02:28:27.367913 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.357899 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 15 02:28:27.367913 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.357905 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 15 02:28:27.367913 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.357912 2579 feature_gate.go:328] unrecognized feature gate: Example Jun 15 02:28:27.367913 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.357917 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 15 02:28:27.367913 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.357921 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 15 02:28:27.368407 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.357925 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 15 02:28:27.368407 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.357929 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 15 02:28:27.368407 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.357934 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 15 02:28:27.368407 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.357938 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 15 02:28:27.368407 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.357942 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 15 02:28:27.368407 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.357946 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 15 02:28:27.368407 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.357952 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 15 02:28:27.368407 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.357957 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 15 02:28:27.368407 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.357961 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 15 02:28:27.368407 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.357966 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 15 02:28:27.368407 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.357970 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 15 02:28:27.368407 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.357974 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 15 02:28:27.368407 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.357979 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 15 02:28:27.368407 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.357983 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 15 02:28:27.368407 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.357987 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 15 02:28:27.368407 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.357991 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 15 02:28:27.368407 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.357995 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 15 02:28:27.368407 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.358002 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 15 02:28:27.368407 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.358008 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 15 02:28:27.368888 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.358034 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 15 02:28:27.368888 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.358040 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 15 02:28:27.368888 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.358045 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 15 02:28:27.368888 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.358050 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 15 02:28:27.368888 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.358054 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 15 02:28:27.368888 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.358059 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 15 02:28:27.368888 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.358063 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 15 02:28:27.368888 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.358067 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 15 02:28:27.368888 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.358073 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 15 02:28:27.368888 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.358077 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 15 02:28:27.368888 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.358081 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 15 02:28:27.368888 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.358086 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 15 02:28:27.368888 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.358089 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 15 02:28:27.368888 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.358094 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jun 15 02:28:27.368888 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.358098 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 15 02:28:27.368888 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.358102 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 15 02:28:27.368888 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.358106 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 15 02:28:27.368888 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.358110 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 15 02:28:27.368888 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.358114 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 15 02:28:27.368888 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.358121 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 15 02:28:27.369376 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.358125 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 15 02:28:27.369376 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.358129 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 15 02:28:27.369376 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.358134 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 15 02:28:27.369376 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.358138 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 15 02:28:27.369376 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.358143 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 15 02:28:27.369376 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.358147 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 15 02:28:27.369376 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.358151 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 15 02:28:27.369376 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.358158 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 15 02:28:27.369376 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.358163 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 15 02:28:27.369376 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.358168 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 15 02:28:27.369376 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.358172 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 15 02:28:27.369376 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.358176 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 15 02:28:27.369376 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.358181 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 15 02:28:27.369376 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.358186 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 15 02:28:27.369376 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.358191 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 15 02:28:27.369376 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.358195 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 15 02:28:27.369376 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.358199 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 15 02:28:27.369376 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.358203 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 15 02:28:27.369376 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.358208 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 15 02:28:27.369866 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.358213 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 15 02:28:27.369866 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.358218 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 15 02:28:27.369866 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.358762 2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 15 02:28:27.369866 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.364774 2579 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jun 15 02:28:27.369866 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.364787 2579 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 15 02:28:27.369866 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.365388 2579 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jun 15 02:28:27.369866 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.365395 2579 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 15 02:28:27.369866 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365452 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 15 02:28:27.369866 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365457 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 15 02:28:27.369866 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365460 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 15 02:28:27.369866 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365464 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 15 02:28:27.369866 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365467 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 15 02:28:27.369866 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365470 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 15 02:28:27.369866 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365473 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 15 02:28:27.370258 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365476 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 15 02:28:27.370258 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365479 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 15 02:28:27.370258 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365482 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 15 02:28:27.370258 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365484 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 15 02:28:27.370258 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365487 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 15 02:28:27.370258 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365490 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 15 02:28:27.370258 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365493 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 15 02:28:27.370258 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365496 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 15 02:28:27.370258 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365498 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 15 02:28:27.370258 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365501 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 15 02:28:27.370258 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365503 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 15 02:28:27.370258 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365506 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 15 02:28:27.370258 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365508 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 15 02:28:27.370258 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365511 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 15 02:28:27.370258 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365514 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 15 02:28:27.370258 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365518 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 15 02:28:27.370258 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365522 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 15 02:28:27.370258 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365529 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 15 02:28:27.370258 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365532 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 15 02:28:27.370751 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365536 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 15 02:28:27.370751 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365540 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 15 02:28:27.370751 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365543 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 15 02:28:27.370751 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365547 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jun 15 02:28:27.370751 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365550 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 15 02:28:27.370751 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365553 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 15 02:28:27.370751 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365556 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 15 02:28:27.370751 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365559 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 15 02:28:27.370751 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365561 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 15 02:28:27.370751 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365564 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 15 02:28:27.370751 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365567 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 15 02:28:27.370751 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365569 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 15 02:28:27.370751 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365572 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 15 02:28:27.370751 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365575 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 15 02:28:27.370751 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365577 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 15 02:28:27.370751 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365580 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 15 02:28:27.370751 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365582 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 15 02:28:27.370751 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365585 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 15 02:28:27.370751 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365588 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 15 02:28:27.370751 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365590 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 15 02:28:27.371243 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365593 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 15 02:28:27.371243 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365595 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 15 02:28:27.371243 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365598 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 15 02:28:27.371243 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365600 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 15 02:28:27.371243 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365603 2579 feature_gate.go:328] unrecognized feature gate: Example Jun 15 02:28:27.371243 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365605 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 15 02:28:27.371243 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365608 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 15 02:28:27.371243 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365611 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 15 02:28:27.371243 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365613 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 15 02:28:27.371243 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365616 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 15 02:28:27.371243 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365618 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 15 02:28:27.371243 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365621 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 15 02:28:27.371243 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365624 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 15 02:28:27.371243 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365626 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 15 02:28:27.371243 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365641 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 15 02:28:27.371243 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365645 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 15 02:28:27.371243 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365648 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 15 02:28:27.371243 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365650 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 15 02:28:27.371243 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365653 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 15 02:28:27.371243 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365656 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 15 02:28:27.371769 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365659 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 15 02:28:27.371769 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365661 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 15 02:28:27.371769 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365664 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 15 02:28:27.371769 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365666 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 15 02:28:27.371769 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365669 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 15 02:28:27.371769 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365671 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 15 02:28:27.371769 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365674 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 15 02:28:27.371769 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365677 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 15 02:28:27.371769 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365679 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 15 02:28:27.371769 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365682 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 15 02:28:27.371769 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365684 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 15 02:28:27.371769 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365687 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 15 02:28:27.371769 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365690 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 15 02:28:27.371769 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365692 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 15 02:28:27.371769 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365696 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 15 02:28:27.371769 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365700 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 15 02:28:27.371769 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365703 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 15 02:28:27.371769 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365706 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 15 02:28:27.371769 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365709 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 15 02:28:27.372214 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365711 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 15 02:28:27.372214 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365714 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 15 02:28:27.372214 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365716 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 15 02:28:27.372214 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365719 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 15 02:28:27.372214 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365721 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 15 02:28:27.372214 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365725 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 15 02:28:27.372214 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365727 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 15 02:28:27.372214 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365730 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 15 02:28:27.372214 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365733 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 15 02:28:27.372214 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365736 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 15 02:28:27.372214 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365738 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 15 02:28:27.372214 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365741 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 15 02:28:27.372214 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365744 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 15 02:28:27.372214 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365746 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 15 02:28:27.372214 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.365751 2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 15 02:28:27.372579 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365865 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 15 02:28:27.372579 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365869 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 15 02:28:27.372579 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365873 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 15 02:28:27.372579 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365876 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 15 02:28:27.372579 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365879 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 15 02:28:27.372579 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365882 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 15 02:28:27.372579 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365884 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 15 02:28:27.372579 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365887 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 15 02:28:27.372579 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365890 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 15 02:28:27.372579 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365893 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 15 02:28:27.372579 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365895 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 15 02:28:27.372579 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365898 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 15 02:28:27.372579 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365900 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 15 02:28:27.372579 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365903 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 15 02:28:27.372579 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365906 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 15 02:28:27.372579 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365908 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 15 02:28:27.372579 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365910 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 15 02:28:27.372579 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365913 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 15 02:28:27.372579 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365915 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 15 02:28:27.372579 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365918 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 15 02:28:27.373118 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365921 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 15 02:28:27.373118 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365923 2579 feature_gate.go:328] unrecognized feature gate: Example Jun 15 02:28:27.373118 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365926 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 15 02:28:27.373118 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365929 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 15 02:28:27.373118 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365932 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 15 02:28:27.373118 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365936 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 15 02:28:27.373118 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365940 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 15 02:28:27.373118 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365942 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 15 02:28:27.373118 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365945 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 15 02:28:27.373118 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365948 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 15 02:28:27.373118 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365951 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 15 02:28:27.373118 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365954 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 15 02:28:27.373118 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365956 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 15 02:28:27.373118 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365959 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 15 02:28:27.373118 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365962 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 15 02:28:27.373118 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365964 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 15 02:28:27.373118 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365967 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 15 02:28:27.373118 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365969 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 15 02:28:27.373118 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365972 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 15 02:28:27.373118 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365974 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 15 02:28:27.373622 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365977 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 15 02:28:27.373622 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365979 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 15 02:28:27.373622 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365983 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 15 02:28:27.373622 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365986 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 15 02:28:27.373622 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365989 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 15 02:28:27.373622 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365992 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 15 02:28:27.373622 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365994 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 15 02:28:27.373622 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365997 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 15 02:28:27.373622 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.365999 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 15 02:28:27.373622 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.366002 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 15 02:28:27.373622 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.366004 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 15 02:28:27.373622 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.366007 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 15 02:28:27.373622 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.366009 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 15 02:28:27.373622 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.366012 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 15 02:28:27.373622 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.366014 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 15 02:28:27.373622 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.366017 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 15 02:28:27.373622 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.366020 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 15 02:28:27.373622 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.366023 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 15 02:28:27.373622 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.366026 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 15 02:28:27.374113 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.366028 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 15 02:28:27.374113 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.366031 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 15 02:28:27.374113 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.366034 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 15 02:28:27.374113 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.366036 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 15 02:28:27.374113 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.366039 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 15 02:28:27.374113 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.366041 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 15 02:28:27.374113 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.366045 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 15 02:28:27.374113 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.366048 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 15 02:28:27.374113 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.366051 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 15 02:28:27.374113 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.366054 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 15 02:28:27.374113 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.366057 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 15 02:28:27.374113 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.366060 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 15 02:28:27.374113 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.366063 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 15 02:28:27.374113 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.366065 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 15 02:28:27.374113 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.366068 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 15 02:28:27.374113 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.366070 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 15 02:28:27.374113 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.366073 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 15 02:28:27.374113 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.366075 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 15 02:28:27.374113 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.366078 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 15 02:28:27.374579 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.366080 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jun 15 02:28:27.374579 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.366083 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 15 02:28:27.374579 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.366085 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 15 02:28:27.374579 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.366088 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 15 02:28:27.374579 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.366090 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 15 02:28:27.374579 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.366093 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 15 02:28:27.374579 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.366095 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 15 02:28:27.374579 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.366098 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 15 02:28:27.374579 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.366100 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 15 02:28:27.374579 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.366103 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 15 02:28:27.374579 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.366106 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 15 02:28:27.374579 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.366109 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 15 02:28:27.374579 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.366111 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 15 02:28:27.374579 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.366114 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 15 02:28:27.374579 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.366116 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 15 02:28:27.374579 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.366119 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 15 02:28:27.374579 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.366121 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 15 02:28:27.374579 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.366124 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 15 02:28:27.374579 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.366126 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 15 02:28:27.374579 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.366129 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 15 02:28:27.375174 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:27.366131 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 15 02:28:27.375174 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.366135 2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 15 02:28:27.375174 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.366220 2579 server.go:962] "Client rotation is on, will bootstrap in background" Jun 15 02:28:27.375174 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.368207 2579 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 15 02:28:27.375174 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.369617 2579 server.go:1019] "Starting client certificate rotation" Jun 15 02:28:27.375174 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.369720 2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 15 02:28:27.375174 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.369756 2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 15 02:28:27.375174 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.371685 2579 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 15 02:28:27.377297 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.377281 2579 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 15 02:28:27.387784 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.387764 2579 log.go:25] "Validated CRI v1 runtime API" Jun 15 02:28:27.390555 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.390540 2579 log.go:25] "Validated CRI v1 image API" Jun 15 02:28:27.391291 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.391277 2579 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 15 02:28:27.393272 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.393251 2579 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 cc2379cc-c7f4-4fd3-9ffd-6716fddc26e6:/dev/nvme0n1p4 f1ea7fe9-f0e6-4d79-99aa-72a326923902:/dev/nvme0n1p3] Jun 15 02:28:27.393342 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.393273 2579 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 15 02:28:27.398086 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.397979 2579 manager.go:216] Machine: {Timestamp:2026-06-15 02:28:27.396904802 +0000 UTC m=+0.264689538 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100112 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec22669e6e3490b7bddcc6c87c9be06e SystemUUID:ec22669e-6e34-90b7-bddc-c6c87c9be06e BootID:6694816b-f0a9-4ea3-8d2b-d9ead3a53dd2 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:81:13:45:90:33 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:81:13:45:90:33 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:3a:ac:c2:7e:f1:d7 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 15 02:28:27.398086 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.398077 2579 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 15 02:28:27.398215 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.398140 2579 manager.go:232] Version: {KernelVersion:5.14.0-570.120.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260608-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 15 02:28:27.398574 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.398553 2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 15 02:28:27.401167 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.401139 2579 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 15 02:28:27.401331 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.401175 2579 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-51.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 15 02:28:27.401871 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.401860 2579 topology_manager.go:138] "Creating topology manager with none policy" Jun 15 02:28:27.401932 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.401876 2579 container_manager_linux.go:309] "Creating device plugin manager" Jun 15 02:28:27.401932 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.401886 2579 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 15 02:28:27.402475 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.402458 2579 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 15 02:28:27.403695 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.403684 2579 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jun 15 02:28:27.407487 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.407473 2579 state_mem.go:36] "Initialized new in-memory state store" Jun 15 02:28:27.407606 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.407596 2579 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jun 15 02:28:27.407679 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.407671 2579 kubelet.go:486] "Attempting to sync node with API server" Jun 15 02:28:27.407740 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.407686 2579 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 15 02:28:27.407740 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.407701 2579 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 15 02:28:27.407740 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.407714 2579 kubelet.go:398] "Adding apiserver pod source" Jun 15 02:28:27.407740 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.407726 2579 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 15 02:28:27.408052 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.408038 2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 15 02:28:27.408117 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.408062 2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 15 02:28:27.409785 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.409768 2579 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jun 15 02:28:27.411398 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.411385 2579 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 15 02:28:27.411437 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.411425 2579 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jun 15 02:28:27.411661 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.411650 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 15 02:28:27.411698 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.411675 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 15 02:28:27.411698 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.411687 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 15 02:28:27.411698 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.411696 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 15 02:28:27.411775 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.411708 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 15 02:28:27.411775 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.411716 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 15 02:28:27.412077 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.412067 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 15 02:28:27.412105 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.412077 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 15 02:28:27.412105 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.412087 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 15 02:28:27.412105 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.412093 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 15 02:28:27.412193 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.412107 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 15 02:28:27.412193 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.412125 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 15 02:28:27.412717 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.412708 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 15 02:28:27.412717 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.412717 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 15 02:28:27.415913 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.415899 2579 server.go:1268] "Started kubelet" Jun 15 02:28:27.416024 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.415972 2579 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 15 02:28:27.416084 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.416028 2579 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 15 02:28:27.416153 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.416133 2579 server_v1.go:49] "podresources" method="list" useActivePods=true Jun 15 02:28:27.416605 ip-10-0-143-51 systemd[1]: Started Kubernetes Kubelet. Jun 15 02:28:27.417822 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.417805 2579 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 15 02:28:27.419723 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.419682 2579 server.go:310] "Adding debug handlers to kubelet server" Jun 15 02:28:27.420416 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:27.420378 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-51.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 15 02:28:27.420520 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:27.420419 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 15 02:28:27.420520 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.420488 2579 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-51.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 15 02:28:27.422216 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:27.421527 2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-51.ec2.internal.18b9202d8140a142 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-51.ec2.internal,UID:ip-10-0-143-51.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-51.ec2.internal,},FirstTimestamp:2026-06-15 02:28:27.415855426 +0000 UTC m=+0.283640162,LastTimestamp:2026-06-15 02:28:27.415855426 +0000 UTC m=+0.283640162,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-51.ec2.internal,}" Jun 15 02:28:27.423392 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:27.423369 2579 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 15 02:28:27.424133 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.424120 2579 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 15 02:28:27.424205 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.424129 2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 15 02:28:27.424205 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:27.424189 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-51.ec2.internal\" not found" Jun 15 02:28:27.424895 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.424875 2579 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jun 15 02:28:27.424987 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.424921 2579 volume_manager.go:311] "The desired_state_of_world populator starts" Jun 15 02:28:27.424987 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.424934 2579 volume_manager.go:313] "Starting Kubelet Volume Manager" Jun 15 02:28:27.425092 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.425065 2579 reconstruct.go:97] "Volume reconstruction finished" Jun 15 02:28:27.425092 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.425077 2579 reconciler.go:29] "Reconciler: start to sync state" Jun 15 02:28:27.425092 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.425080 2579 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 15 02:28:27.425092 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.425092 2579 factory.go:55] Registering systemd factory Jun 15 02:28:27.425270 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.425102 2579 factory.go:223] Registration of the systemd container factory successfully Jun 15 02:28:27.425342 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.425326 2579 factory.go:153] Registering CRI-O factory Jun 15 02:28:27.425342 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.425340 2579 factory.go:223] Registration of the crio container factory successfully Jun 15 02:28:27.425342 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.425357 2579 factory.go:103] Registering Raw factory Jun 15 02:28:27.425544 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.425367 2579 manager.go:1196] Started watching for new ooms in manager Jun 15 02:28:27.425799 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.425785 2579 manager.go:349] Starting recovery of all containers Jun 15 02:28:27.434553 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:27.434268 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jun 15 02:28:27.434553 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:27.434309 2579 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-51.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jun 15 02:28:27.435954 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.435939 2579 manager.go:354] Recovery completed Jun 15 02:28:27.440194 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.440183 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 15 02:28:27.442938 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.442916 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-51.ec2.internal" event="NodeHasSufficientMemory" Jun 15 02:28:27.443007 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.442950 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-51.ec2.internal" event="NodeHasNoDiskPressure" Jun 15 02:28:27.443007 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.442960 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-51.ec2.internal" event="NodeHasSufficientPID" Jun 15 02:28:27.443460 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.443442 2579 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 15 02:28:27.443460 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.443453 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 15 02:28:27.443460 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.443458 2579 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 15 02:28:27.443594 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.443476 2579 state_mem.go:36] "Initialized new in-memory state store" Jun 15 02:28:27.444163 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.444149 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-51.ec2.internal" event="NodeHasSufficientMemory" Jun 15 02:28:27.444226 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.444174 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-51.ec2.internal" event="NodeHasNoDiskPressure" Jun 15 02:28:27.444226 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.444183 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-51.ec2.internal" event="NodeHasSufficientPID" Jun 15 02:28:27.445488 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.445474 2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-g85hn" Jun 15 02:28:27.445657 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.445644 2579 policy_none.go:49] "None policy: Start" Jun 15 02:28:27.445703 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.445662 2579 memory_manager.go:187] "Starting memorymanager" policy="None" Jun 15 02:28:27.445703 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.445672 2579 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jun 15 02:28:27.447657 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.447647 2579 policy_none.go:47] "Start" Jun 15 02:28:27.453727 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:27.453668 2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-51.ec2.internal.18b9202d82dde03e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-51.ec2.internal,UID:ip-10-0-143-51.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-51.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-51.ec2.internal,},FirstTimestamp:2026-06-15 02:28:27.442937918 +0000 UTC m=+0.310722654,LastTimestamp:2026-06-15 02:28:27.442937918 +0000 UTC m=+0.310722654,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-51.ec2.internal,}" Jun 15 02:28:27.454230 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.454213 2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-g85hn" Jun 15 02:28:27.500511 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.483939 2579 manager.go:337] "Starting Device Plugin manager" Jun 15 02:28:27.500511 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:27.483964 2579 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 15 02:28:27.500511 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.483973 2579 server.go:87] "Starting device plugin registration server" Jun 15 02:28:27.500511 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.484180 2579 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 15 02:28:27.500511 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.484193 2579 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 15 02:28:27.500511 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.484297 2579 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 15 02:28:27.500511 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.484363 2579 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 15 02:28:27.500511 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.484371 2579 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 15 02:28:27.500511 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.484656 2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jun 15 02:28:27.500511 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.486172 2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jun 15 02:28:27.500511 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.486204 2579 status_manager.go:244] "Starting to sync pod status with apiserver" Jun 15 02:28:27.500511 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.486222 2579 kubelet.go:2448] "Starting kubelet main sync loop" Jun 15 02:28:27.500511 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:27.486273 2579 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 15 02:28:27.500511 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:27.487373 2579 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 15 02:28:27.500511 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:27.487561 2579 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-51.ec2.internal\" not found" Jun 15 02:28:27.500511 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.488290 2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 15 02:28:27.584429 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.584413 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 15 02:28:27.585145 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.585129 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-51.ec2.internal" event="NodeHasSufficientMemory" Jun 15 02:28:27.585216 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.585156 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-51.ec2.internal" event="NodeHasNoDiskPressure" Jun 15 02:28:27.585216 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.585167 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-51.ec2.internal" event="NodeHasSufficientPID" Jun 15 02:28:27.585216 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.585185 2579 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-51.ec2.internal" Jun 15 02:28:27.586847 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.586832 2579 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-51.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-51.ec2.internal"] Jun 15 02:28:27.586903 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.586882 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 15 02:28:27.588383 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.588367 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-51.ec2.internal" event="NodeHasSufficientMemory" Jun 15 02:28:27.588467 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.588394 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-51.ec2.internal" event="NodeHasNoDiskPressure" Jun 15 02:28:27.588467 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.588403 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-51.ec2.internal" event="NodeHasSufficientPID" Jun 15 02:28:27.590669 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.590657 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 15 02:28:27.590760 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.590746 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-51.ec2.internal" Jun 15 02:28:27.590811 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.590780 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 15 02:28:27.591317 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.591302 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-51.ec2.internal" event="NodeHasSufficientMemory" Jun 15 02:28:27.591402 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.591328 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-51.ec2.internal" event="NodeHasNoDiskPressure" Jun 15 02:28:27.591402 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.591342 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-51.ec2.internal" event="NodeHasSufficientPID" Jun 15 02:28:27.591402 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.591301 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-51.ec2.internal" event="NodeHasSufficientMemory" Jun 15 02:28:27.591402 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.591399 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-51.ec2.internal" event="NodeHasNoDiskPressure" Jun 15 02:28:27.591576 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.591411 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-51.ec2.internal" event="NodeHasSufficientPID" Jun 15 02:28:27.593644 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.593614 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-51.ec2.internal" Jun 15 02:28:27.593708 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.593652 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 15 02:28:27.593752 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.593743 2579 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-51.ec2.internal" Jun 15 02:28:27.593803 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:27.593761 2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-51.ec2.internal\": node \"ip-10-0-143-51.ec2.internal\" not found" Jun 15 02:28:27.594317 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.594300 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-51.ec2.internal" event="NodeHasSufficientMemory" Jun 15 02:28:27.594381 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.594327 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-51.ec2.internal" event="NodeHasNoDiskPressure" Jun 15 02:28:27.594381 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.594337 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-51.ec2.internal" event="NodeHasSufficientPID" Jun 15 02:28:27.608252 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:27.608230 2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-51.ec2.internal\" not found" node="ip-10-0-143-51.ec2.internal" Jun 15 02:28:27.612002 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:27.611989 2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-51.ec2.internal\" not found" node="ip-10-0-143-51.ec2.internal" Jun 15 02:28:27.625543 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.625520 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f8074a2ff640a6c74fbf4752467e2525-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-51.ec2.internal\" (UID: \"f8074a2ff640a6c74fbf4752467e2525\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-51.ec2.internal" Jun 15 02:28:27.625597 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.625544 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/66c462553606b0900fb46b9deb100df6-config\") pod \"kube-apiserver-proxy-ip-10-0-143-51.ec2.internal\" (UID: \"66c462553606b0900fb46b9deb100df6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-51.ec2.internal" Jun 15 02:28:27.625644 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.625609 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f8074a2ff640a6c74fbf4752467e2525-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-51.ec2.internal\" (UID: \"f8074a2ff640a6c74fbf4752467e2525\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-51.ec2.internal" Jun 15 02:28:27.628923 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:27.628909 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-51.ec2.internal\" not found" Jun 15 02:28:27.725815 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.725797 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f8074a2ff640a6c74fbf4752467e2525-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-51.ec2.internal\" (UID: \"f8074a2ff640a6c74fbf4752467e2525\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-51.ec2.internal" Jun 15 02:28:27.725896 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.725822 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f8074a2ff640a6c74fbf4752467e2525-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-51.ec2.internal\" (UID: \"f8074a2ff640a6c74fbf4752467e2525\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-51.ec2.internal" Jun 15 02:28:27.725896 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.725839 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/66c462553606b0900fb46b9deb100df6-config\") pod \"kube-apiserver-proxy-ip-10-0-143-51.ec2.internal\" (UID: \"66c462553606b0900fb46b9deb100df6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-51.ec2.internal" Jun 15 02:28:27.725896 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.725891 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f8074a2ff640a6c74fbf4752467e2525-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-51.ec2.internal\" (UID: \"f8074a2ff640a6c74fbf4752467e2525\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-51.ec2.internal" Jun 15 02:28:27.725988 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.725893 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/66c462553606b0900fb46b9deb100df6-config\") pod \"kube-apiserver-proxy-ip-10-0-143-51.ec2.internal\" (UID: \"66c462553606b0900fb46b9deb100df6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-51.ec2.internal" Jun 15 02:28:27.725988 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.725921 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f8074a2ff640a6c74fbf4752467e2525-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-51.ec2.internal\" (UID: \"f8074a2ff640a6c74fbf4752467e2525\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-51.ec2.internal" Jun 15 02:28:27.729916 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:27.729901 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-51.ec2.internal\" not found" Jun 15 02:28:27.830448 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:27.830419 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-51.ec2.internal\" not found" Jun 15 02:28:27.910872 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.910833 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-51.ec2.internal" Jun 15 02:28:27.914781 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:27.914730 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-51.ec2.internal" Jun 15 02:28:27.930819 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:27.930802 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-51.ec2.internal\" not found" Jun 15 02:28:28.031284 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:28.031258 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-51.ec2.internal\" not found" Jun 15 02:28:28.131805 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:28.131785 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-51.ec2.internal\" not found" Jun 15 02:28:28.232403 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:28.232355 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-51.ec2.internal\" not found" Jun 15 02:28:28.333048 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:28.332927 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-51.ec2.internal\" not found" Jun 15 02:28:28.369088 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:28.369070 2579 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 15 02:28:28.369605 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:28.369219 2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 15 02:28:28.433764 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:28.433741 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-51.ec2.internal\" not found" Jun 15 02:28:28.459274 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:28.459244 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-14 02:23:27 +0000 UTC" deadline="2028-02-12 13:27:16.53634957 +0000 UTC" Jun 15 02:28:28.459274 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:28.459272 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14578h58m48.077081119s" Jun 15 02:28:28.478252 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:28.478224 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66c462553606b0900fb46b9deb100df6.slice/crio-a2a4abd8bff001372ab58cadd12d0f775740bd16b052c80abb11a24979640610 WatchSource:0}: Error finding container a2a4abd8bff001372ab58cadd12d0f775740bd16b052c80abb11a24979640610: Status 404 returned error can't find the container with id a2a4abd8bff001372ab58cadd12d0f775740bd16b052c80abb11a24979640610 Jun 15 02:28:28.479022 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:28.478994 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8074a2ff640a6c74fbf4752467e2525.slice/crio-e3654e220a56f0d8b88d8c598049763494830c951c0d304e1f3d6436fe574ab1 WatchSource:0}: Error finding container e3654e220a56f0d8b88d8c598049763494830c951c0d304e1f3d6436fe574ab1: Status 404 returned error can't find the container with id e3654e220a56f0d8b88d8c598049763494830c951c0d304e1f3d6436fe574ab1 Jun 15 02:28:28.483458 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:28.483437 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 15 02:28:28.490651 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:28.490607 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-51.ec2.internal" event={"ID":"f8074a2ff640a6c74fbf4752467e2525","Type":"ContainerStarted","Data":"e3654e220a56f0d8b88d8c598049763494830c951c0d304e1f3d6436fe574ab1"} Jun 15 02:28:28.491561 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:28.491543 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-51.ec2.internal" event={"ID":"66c462553606b0900fb46b9deb100df6","Type":"ContainerStarted","Data":"a2a4abd8bff001372ab58cadd12d0f775740bd16b052c80abb11a24979640610"} Jun 15 02:28:28.534088 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:28.534055 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-51.ec2.internal\" not found" Jun 15 02:28:28.557205 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:28.557187 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 15 02:28:28.557970 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:28.557955 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-51.ec2.internal" event="NodeHasSufficientMemory" Jun 15 02:28:28.558024 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:28.557983 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-51.ec2.internal" event="NodeHasNoDiskPressure" Jun 15 02:28:28.558024 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:28.557998 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-51.ec2.internal" event="NodeHasSufficientPID" Jun 15 02:28:28.634896 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:28.634872 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-51.ec2.internal\" not found" Jun 15 02:28:28.735389 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:28.735346 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-51.ec2.internal\" not found" Jun 15 02:28:28.739715 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:28.739692 2579 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 15 02:28:28.782359 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:28.782338 2579 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 15 02:28:28.825000 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:28.824984 2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-51.ec2.internal" Jun 15 02:28:28.834457 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:28.834438 2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 15 02:28:28.835808 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:28.835789 2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-51.ec2.internal" Jun 15 02:28:28.846083 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:28.846067 2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 15 02:28:28.889218 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:28.889198 2579 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 15 02:28:29.408709 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.408678 2579 apiserver.go:52] "Watching apiserver" Jun 15 02:28:29.414028 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.414007 2579 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 15 02:28:29.414997 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.414972 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-143-51.ec2.internal","openshift-dns/node-resolver-wthd7","openshift-multus/network-metrics-daemon-954mx","openshift-network-operator/iptables-alerter-p76zd","openshift-ovn-kubernetes/ovnkube-node-qg4vc","kube-system/konnectivity-agent-tsd5w","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v956g","openshift-cluster-node-tuning-operator/tuned-wpq45","openshift-image-registry/node-ca-9zrhr","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-51.ec2.internal","openshift-multus/multus-additional-cni-plugins-5hrsv","openshift-multus/multus-lnjtl","openshift-network-diagnostics/network-check-target-5qfdl","kube-system/global-pull-secret-syncer-qnxs7"] Jun 15 02:28:29.417451 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.417426 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v956g" Jun 15 02:28:29.419774 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.419616 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5qfdl" Jun 15 02:28:29.419774 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:29.419700 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5qfdl" podUID="91285de0-6ca5-4b23-b554-0c06718dc1df" Jun 15 02:28:29.419774 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.419766 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-nx76n\"" Jun 15 02:28:29.419922 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.419774 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 15 02:28:29.419986 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.419967 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 15 02:28:29.421083 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.421062 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 15 02:28:29.421862 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.421846 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-954mx" Jun 15 02:28:29.421931 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:29.421912 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-954mx" podUID="0c770be9-5ffa-4da5-bc85-58fd4c82b027" Jun 15 02:28:29.424849 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.424833 2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 15 02:28:29.424947 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.424935 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-p76zd" Jun 15 02:28:29.426819 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.426799 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 15 02:28:29.426940 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.426883 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-5kmfn\"" Jun 15 02:28:29.427074 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.427019 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 15 02:28:29.427152 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.427138 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 15 02:28:29.427256 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.427243 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.428962 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.428943 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 15 02:28:29.429182 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.429163 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 15 02:28:29.429262 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.429167 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 15 02:28:29.429262 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.429169 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 15 02:28:29.429542 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.429523 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 15 02:28:29.429620 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.429558 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 15 02:28:29.429620 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.429575 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-bv4sn\"" Jun 15 02:28:29.429729 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.429575 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-tsd5w" Jun 15 02:28:29.431509 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.431493 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 15 02:28:29.431571 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.431498 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-6xrlf\"" Jun 15 02:28:29.431644 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.431604 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 15 02:28:29.431886 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.431871 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9zrhr" Jun 15 02:28:29.433482 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.433464 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 15 02:28:29.434054 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.434037 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qnxs7" Jun 15 02:28:29.434128 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.434073 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 15 02:28:29.434128 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:29.434094 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qnxs7" podUID="d436483f-cbb4-4ce9-b4d5-a21564cf6e97" Jun 15 02:28:29.434234 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.434160 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-fmfnv\"" Jun 15 02:28:29.434234 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.434168 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 15 02:28:29.434234 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.434182 2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 15 02:28:29.434965 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.434946 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztfbc\" (UniqueName: \"kubernetes.io/projected/f2a4bcb6-c233-4ccc-9fc1-88c268ae2fb6-kube-api-access-ztfbc\") pod \"iptables-alerter-p76zd\" (UID: \"f2a4bcb6-c233-4ccc-9fc1-88c268ae2fb6\") " pod="openshift-network-operator/iptables-alerter-p76zd" Jun 15 02:28:29.435053 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.434971 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ea5abfee-bd02-41ce-a0f9-189e5c238bec-device-dir\") pod \"aws-ebs-csi-driver-node-v956g\" (UID: \"ea5abfee-bd02-41ce-a0f9-189e5c238bec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v956g" Jun 15 02:28:29.435053 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.434985 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ea5abfee-bd02-41ce-a0f9-189e5c238bec-etc-selinux\") pod \"aws-ebs-csi-driver-node-v956g\" (UID: \"ea5abfee-bd02-41ce-a0f9-189e5c238bec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v956g" Jun 15 02:28:29.435053 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.435010 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ea5abfee-bd02-41ce-a0f9-189e5c238bec-sys-fs\") pod \"aws-ebs-csi-driver-node-v956g\" (UID: \"ea5abfee-bd02-41ce-a0f9-189e5c238bec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v956g" Jun 15 02:28:29.435197 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.435076 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvrg6\" (UniqueName: \"kubernetes.io/projected/0c770be9-5ffa-4da5-bc85-58fd4c82b027-kube-api-access-cvrg6\") pod \"network-metrics-daemon-954mx\" (UID: \"0c770be9-5ffa-4da5-bc85-58fd4c82b027\") " pod="openshift-multus/network-metrics-daemon-954mx" Jun 15 02:28:29.435197 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.435126 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-host-cni-netd\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.435518 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.435497 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-host-slash\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.435600 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.435558 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-run-systemd\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.435600 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.435583 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-host-kubelet\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.435715 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.435611 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-run-ovn\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.435715 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.435657 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-host-cni-bin\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.435715 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.435687 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-ovnkube-config\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.435854 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.435727 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-env-overrides\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.435854 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.435753 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nntp6\" (UniqueName: \"kubernetes.io/projected/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-kube-api-access-nntp6\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.435854 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.435784 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l75z9\" (UniqueName: \"kubernetes.io/projected/91285de0-6ca5-4b23-b554-0c06718dc1df-kube-api-access-l75z9\") pod \"network-check-target-5qfdl\" (UID: \"91285de0-6ca5-4b23-b554-0c06718dc1df\") " pod="openshift-network-diagnostics/network-check-target-5qfdl" Jun 15 02:28:29.435854 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.435837 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f2a4bcb6-c233-4ccc-9fc1-88c268ae2fb6-iptables-alerter-script\") pod \"iptables-alerter-p76zd\" (UID: \"f2a4bcb6-c233-4ccc-9fc1-88c268ae2fb6\") " pod="openshift-network-operator/iptables-alerter-p76zd" Jun 15 02:28:29.436072 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.435878 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-host-run-netns\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.436072 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.435994 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-run-openvswitch\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.436072 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.436026 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-ovnkube-script-lib\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.436208 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.436084 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvblx\" (UniqueName: \"kubernetes.io/projected/ea5abfee-bd02-41ce-a0f9-189e5c238bec-kube-api-access-hvblx\") pod \"aws-ebs-csi-driver-node-v956g\" (UID: \"ea5abfee-bd02-41ce-a0f9-189e5c238bec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v956g" Jun 15 02:28:29.436671 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.436147 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-systemd-units\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.436761 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.436695 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-var-lib-openvswitch\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.436761 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.436729 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.436858 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.436758 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-ovn-node-metrics-cert\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.436858 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.436785 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ea5abfee-bd02-41ce-a0f9-189e5c238bec-kubelet-dir\") pod \"aws-ebs-csi-driver-node-v956g\" (UID: \"ea5abfee-bd02-41ce-a0f9-189e5c238bec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v956g" Jun 15 02:28:29.436858 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.436809 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ea5abfee-bd02-41ce-a0f9-189e5c238bec-socket-dir\") pod \"aws-ebs-csi-driver-node-v956g\" (UID: \"ea5abfee-bd02-41ce-a0f9-189e5c238bec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v956g" Jun 15 02:28:29.436858 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.436836 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0c770be9-5ffa-4da5-bc85-58fd4c82b027-metrics-certs\") pod \"network-metrics-daemon-954mx\" (UID: \"0c770be9-5ffa-4da5-bc85-58fd4c82b027\") " pod="openshift-multus/network-metrics-daemon-954mx" Jun 15 02:28:29.437027 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.436861 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f2a4bcb6-c233-4ccc-9fc1-88c268ae2fb6-host-slash\") pod \"iptables-alerter-p76zd\" (UID: \"f2a4bcb6-c233-4ccc-9fc1-88c268ae2fb6\") " pod="openshift-network-operator/iptables-alerter-p76zd" Jun 15 02:28:29.437027 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.436891 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-etc-openvswitch\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.437027 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.436920 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-host-run-ovn-kubernetes\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.437027 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.436944 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ea5abfee-bd02-41ce-a0f9-189e5c238bec-registration-dir\") pod \"aws-ebs-csi-driver-node-v956g\" (UID: \"ea5abfee-bd02-41ce-a0f9-189e5c238bec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v956g" Jun 15 02:28:29.437027 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.436971 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-node-log\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.437027 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.436997 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-log-socket\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.438411 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.438381 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wthd7" Jun 15 02:28:29.440472 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.440452 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 15 02:28:29.440560 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.440525 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 15 02:28:29.440619 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.440584 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-xkflz\"" Jun 15 02:28:29.442159 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.442143 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lnjtl" Jun 15 02:28:29.443920 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.443904 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 15 02:28:29.444030 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.444013 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-8jnpj\"" Jun 15 02:28:29.444030 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.444019 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 15 02:28:29.444181 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.444163 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 15 02:28:29.444272 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.444211 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 15 02:28:29.444491 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.444478 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5hrsv" Jun 15 02:28:29.446161 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.446148 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 15 02:28:29.446385 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.446371 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 15 02:28:29.446466 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.446376 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-bf2kz\"" Jun 15 02:28:29.446698 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.446681 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wpq45" Jun 15 02:28:29.448436 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.448421 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-htb2v\"" Jun 15 02:28:29.448524 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.448438 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 15 02:28:29.448524 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.448421 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 15 02:28:29.453557 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.453542 2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-jlxnf" Jun 15 02:28:29.460382 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.460363 2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-jlxnf" Jun 15 02:28:29.526535 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.526515 2579 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jun 15 02:28:29.538072 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.538053 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-host-slash\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.538162 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.538083 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-run-systemd\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.538162 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.538106 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4-host-run-netns\") pod \"multus-lnjtl\" (UID: \"e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4\") " pod="openshift-multus/multus-lnjtl" Jun 15 02:28:29.538162 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.538122 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4-host-var-lib-cni-multus\") pod \"multus-lnjtl\" (UID: \"e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4\") " pod="openshift-multus/multus-lnjtl" Jun 15 02:28:29.538162 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.538138 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdnx7\" (UniqueName: \"kubernetes.io/projected/6798a601-4421-44c1-8ef7-3b2527cad953-kube-api-access-sdnx7\") pod \"node-ca-9zrhr\" (UID: \"6798a601-4421-44c1-8ef7-3b2527cad953\") " pod="openshift-image-registry/node-ca-9zrhr" Jun 15 02:28:29.538367 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.538197 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-host-kubelet\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.538367 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.538245 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-ovnkube-config\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.538367 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.538258 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-run-systemd\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.538367 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.538271 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nntp6\" (UniqueName: \"kubernetes.io/projected/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-kube-api-access-nntp6\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.538367 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.538292 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-host-slash\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.538367 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.538296 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l75z9\" (UniqueName: \"kubernetes.io/projected/91285de0-6ca5-4b23-b554-0c06718dc1df-kube-api-access-l75z9\") pod \"network-check-target-5qfdl\" (UID: \"91285de0-6ca5-4b23-b554-0c06718dc1df\") " pod="openshift-network-diagnostics/network-check-target-5qfdl" Jun 15 02:28:29.538367 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.538324 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4-system-cni-dir\") pod \"multus-lnjtl\" (UID: \"e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4\") " pod="openshift-multus/multus-lnjtl" Jun 15 02:28:29.538367 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.538332 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-host-kubelet\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.538367 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.538347 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/95473860-f2a8-4daa-b746-dac03c5462be-lib-modules\") pod \"tuned-wpq45\" (UID: \"95473860-f2a8-4daa-b746-dac03c5462be\") " pod="openshift-cluster-node-tuning-operator/tuned-wpq45" Jun 15 02:28:29.538870 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.538376 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f2a4bcb6-c233-4ccc-9fc1-88c268ae2fb6-iptables-alerter-script\") pod \"iptables-alerter-p76zd\" (UID: \"f2a4bcb6-c233-4ccc-9fc1-88c268ae2fb6\") " pod="openshift-network-operator/iptables-alerter-p76zd" Jun 15 02:28:29.538870 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.538406 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-ovnkube-script-lib\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.538870 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.538430 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/95473860-f2a8-4daa-b746-dac03c5462be-etc-sysctl-d\") pod \"tuned-wpq45\" (UID: \"95473860-f2a8-4daa-b746-dac03c5462be\") " pod="openshift-cluster-node-tuning-operator/tuned-wpq45" Jun 15 02:28:29.538870 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.538466 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/54db0e1c-0d25-46d2-a3e3-aebf9339e272-os-release\") pod \"multus-additional-cni-plugins-5hrsv\" (UID: \"54db0e1c-0d25-46d2-a3e3-aebf9339e272\") " pod="openshift-multus/multus-additional-cni-plugins-5hrsv" Jun 15 02:28:29.538870 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.538491 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fc66bce4-865f-4b30-aca2-93cedd05ffe3-konnectivity-ca\") pod \"konnectivity-agent-tsd5w\" (UID: \"fc66bce4-865f-4b30-aca2-93cedd05ffe3\") " pod="kube-system/konnectivity-agent-tsd5w" Jun 15 02:28:29.538870 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.538516 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-ovn-node-metrics-cert\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.538870 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.538541 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ea5abfee-bd02-41ce-a0f9-189e5c238bec-socket-dir\") pod \"aws-ebs-csi-driver-node-v956g\" (UID: \"ea5abfee-bd02-41ce-a0f9-189e5c238bec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v956g" Jun 15 02:28:29.538870 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.538565 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4-cni-binary-copy\") pod \"multus-lnjtl\" (UID: \"e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4\") " pod="openshift-multus/multus-lnjtl" Jun 15 02:28:29.538870 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.538608 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/95473860-f2a8-4daa-b746-dac03c5462be-sys\") pod \"tuned-wpq45\" (UID: \"95473860-f2a8-4daa-b746-dac03c5462be\") " pod="openshift-cluster-node-tuning-operator/tuned-wpq45" Jun 15 02:28:29.538870 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.538646 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/95473860-f2a8-4daa-b746-dac03c5462be-host\") pod \"tuned-wpq45\" (UID: \"95473860-f2a8-4daa-b746-dac03c5462be\") " pod="openshift-cluster-node-tuning-operator/tuned-wpq45" Jun 15 02:28:29.538870 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.538686 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/54db0e1c-0d25-46d2-a3e3-aebf9339e272-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5hrsv\" (UID: \"54db0e1c-0d25-46d2-a3e3-aebf9339e272\") " pod="openshift-multus/multus-additional-cni-plugins-5hrsv" Jun 15 02:28:29.538870 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.538725 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f2a4bcb6-c233-4ccc-9fc1-88c268ae2fb6-host-slash\") pod \"iptables-alerter-p76zd\" (UID: \"f2a4bcb6-c233-4ccc-9fc1-88c268ae2fb6\") " pod="openshift-network-operator/iptables-alerter-p76zd" Jun 15 02:28:29.538870 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.538750 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-etc-openvswitch\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.538870 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.538775 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-host-run-ovn-kubernetes\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.538870 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.538794 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f2a4bcb6-c233-4ccc-9fc1-88c268ae2fb6-host-slash\") pod \"iptables-alerter-p76zd\" (UID: \"f2a4bcb6-c233-4ccc-9fc1-88c268ae2fb6\") " pod="openshift-network-operator/iptables-alerter-p76zd" Jun 15 02:28:29.538870 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.538798 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ea5abfee-bd02-41ce-a0f9-189e5c238bec-socket-dir\") pod \"aws-ebs-csi-driver-node-v956g\" (UID: \"ea5abfee-bd02-41ce-a0f9-189e5c238bec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v956g" Jun 15 02:28:29.538870 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.538799 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ea5abfee-bd02-41ce-a0f9-189e5c238bec-registration-dir\") pod \"aws-ebs-csi-driver-node-v956g\" (UID: \"ea5abfee-bd02-41ce-a0f9-189e5c238bec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v956g" Jun 15 02:28:29.539571 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.538831 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-etc-openvswitch\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.539571 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.538837 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-node-log\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.539571 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.538838 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-host-run-ovn-kubernetes\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.539571 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.538867 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-log-socket\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.539571 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.538886 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-node-log\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.539571 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.538895 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/95473860-f2a8-4daa-b746-dac03c5462be-etc-systemd\") pod \"tuned-wpq45\" (UID: \"95473860-f2a8-4daa-b746-dac03c5462be\") " pod="openshift-cluster-node-tuning-operator/tuned-wpq45" Jun 15 02:28:29.539571 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.538899 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-ovnkube-config\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.539571 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.538920 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztfbc\" (UniqueName: \"kubernetes.io/projected/f2a4bcb6-c233-4ccc-9fc1-88c268ae2fb6-kube-api-access-ztfbc\") pod \"iptables-alerter-p76zd\" (UID: \"f2a4bcb6-c233-4ccc-9fc1-88c268ae2fb6\") " pod="openshift-network-operator/iptables-alerter-p76zd" Jun 15 02:28:29.539571 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.538936 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ea5abfee-bd02-41ce-a0f9-189e5c238bec-registration-dir\") pod \"aws-ebs-csi-driver-node-v956g\" (UID: \"ea5abfee-bd02-41ce-a0f9-189e5c238bec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v956g" Jun 15 02:28:29.539571 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.538960 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ea5abfee-bd02-41ce-a0f9-189e5c238bec-etc-selinux\") pod \"aws-ebs-csi-driver-node-v956g\" (UID: \"ea5abfee-bd02-41ce-a0f9-189e5c238bec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v956g" Jun 15 02:28:29.539571 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.539442 2579 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 15 02:28:29.539571 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.538965 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-ovnkube-script-lib\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.539571 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.539571 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-log-socket\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.540148 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.539606 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ea5abfee-bd02-41ce-a0f9-189e5c238bec-sys-fs\") pod \"aws-ebs-csi-driver-node-v956g\" (UID: \"ea5abfee-bd02-41ce-a0f9-189e5c238bec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v956g" Jun 15 02:28:29.540148 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.539660 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/95473860-f2a8-4daa-b746-dac03c5462be-etc-sysconfig\") pod \"tuned-wpq45\" (UID: \"95473860-f2a8-4daa-b746-dac03c5462be\") " pod="openshift-cluster-node-tuning-operator/tuned-wpq45" Jun 15 02:28:29.540148 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.539696 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-host-cni-netd\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.540148 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.539660 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ea5abfee-bd02-41ce-a0f9-189e5c238bec-etc-selinux\") pod \"aws-ebs-csi-driver-node-v956g\" (UID: \"ea5abfee-bd02-41ce-a0f9-189e5c238bec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v956g" Jun 15 02:28:29.540148 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.539711 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f2a4bcb6-c233-4ccc-9fc1-88c268ae2fb6-iptables-alerter-script\") pod \"iptables-alerter-p76zd\" (UID: \"f2a4bcb6-c233-4ccc-9fc1-88c268ae2fb6\") " pod="openshift-network-operator/iptables-alerter-p76zd" Jun 15 02:28:29.540148 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.539698 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ea5abfee-bd02-41ce-a0f9-189e5c238bec-sys-fs\") pod \"aws-ebs-csi-driver-node-v956g\" (UID: \"ea5abfee-bd02-41ce-a0f9-189e5c238bec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v956g" Jun 15 02:28:29.540148 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.539746 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4-multus-socket-dir-parent\") pod \"multus-lnjtl\" (UID: \"e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4\") " pod="openshift-multus/multus-lnjtl" Jun 15 02:28:29.540148 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.539760 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-host-cni-netd\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.540148 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.539790 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9m26g\" (UniqueName: \"kubernetes.io/projected/e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4-kube-api-access-9m26g\") pod \"multus-lnjtl\" (UID: \"e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4\") " pod="openshift-multus/multus-lnjtl" Jun 15 02:28:29.540148 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.539825 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4v8dm\" (UniqueName: \"kubernetes.io/projected/95473860-f2a8-4daa-b746-dac03c5462be-kube-api-access-4v8dm\") pod \"tuned-wpq45\" (UID: \"95473860-f2a8-4daa-b746-dac03c5462be\") " pod="openshift-cluster-node-tuning-operator/tuned-wpq45" Jun 15 02:28:29.540148 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.539878 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6798a601-4421-44c1-8ef7-3b2527cad953-serviceca\") pod \"node-ca-9zrhr\" (UID: \"6798a601-4421-44c1-8ef7-3b2527cad953\") " pod="openshift-image-registry/node-ca-9zrhr" Jun 15 02:28:29.540148 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.539925 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d436483f-cbb4-4ce9-b4d5-a21564cf6e97-kubelet-config\") pod \"global-pull-secret-syncer-qnxs7\" (UID: \"d436483f-cbb4-4ce9-b4d5-a21564cf6e97\") " pod="kube-system/global-pull-secret-syncer-qnxs7" Jun 15 02:28:29.540148 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.539956 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/54db0e1c-0d25-46d2-a3e3-aebf9339e272-system-cni-dir\") pod \"multus-additional-cni-plugins-5hrsv\" (UID: \"54db0e1c-0d25-46d2-a3e3-aebf9339e272\") " pod="openshift-multus/multus-additional-cni-plugins-5hrsv" Jun 15 02:28:29.540148 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.539985 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/54db0e1c-0d25-46d2-a3e3-aebf9339e272-cni-binary-copy\") pod \"multus-additional-cni-plugins-5hrsv\" (UID: \"54db0e1c-0d25-46d2-a3e3-aebf9339e272\") " pod="openshift-multus/multus-additional-cni-plugins-5hrsv" Jun 15 02:28:29.540148 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.540014 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4-os-release\") pod \"multus-lnjtl\" (UID: \"e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4\") " pod="openshift-multus/multus-lnjtl" Jun 15 02:28:29.540148 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.540037 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/95473860-f2a8-4daa-b746-dac03c5462be-etc-modprobe-d\") pod \"tuned-wpq45\" (UID: \"95473860-f2a8-4daa-b746-dac03c5462be\") " pod="openshift-cluster-node-tuning-operator/tuned-wpq45" Jun 15 02:28:29.540148 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.540066 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/95473860-f2a8-4daa-b746-dac03c5462be-etc-sysctl-conf\") pod \"tuned-wpq45\" (UID: \"95473860-f2a8-4daa-b746-dac03c5462be\") " pod="openshift-cluster-node-tuning-operator/tuned-wpq45" Jun 15 02:28:29.540847 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.540092 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d436483f-cbb4-4ce9-b4d5-a21564cf6e97-dbus\") pod \"global-pull-secret-syncer-qnxs7\" (UID: \"d436483f-cbb4-4ce9-b4d5-a21564cf6e97\") " pod="kube-system/global-pull-secret-syncer-qnxs7" Jun 15 02:28:29.540847 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.540122 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f96eb18e-3121-4b4e-a6fb-a0c8bd807285-tmp-dir\") pod \"node-resolver-wthd7\" (UID: \"f96eb18e-3121-4b4e-a6fb-a0c8bd807285\") " pod="openshift-dns/node-resolver-wthd7" Jun 15 02:28:29.540847 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.540153 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-run-ovn\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.540847 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.540178 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-host-cni-bin\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.540847 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.540207 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-env-overrides\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.540847 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.540235 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4-host-var-lib-kubelet\") pod \"multus-lnjtl\" (UID: \"e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4\") " pod="openshift-multus/multus-lnjtl" Jun 15 02:28:29.540847 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.540266 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bfpv\" (UniqueName: \"kubernetes.io/projected/54db0e1c-0d25-46d2-a3e3-aebf9339e272-kube-api-access-2bfpv\") pod \"multus-additional-cni-plugins-5hrsv\" (UID: \"54db0e1c-0d25-46d2-a3e3-aebf9339e272\") " pod="openshift-multus/multus-additional-cni-plugins-5hrsv" Jun 15 02:28:29.540847 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.540302 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-host-run-netns\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.540847 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.540342 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-run-openvswitch\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.540847 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.540375 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvblx\" (UniqueName: \"kubernetes.io/projected/ea5abfee-bd02-41ce-a0f9-189e5c238bec-kube-api-access-hvblx\") pod \"aws-ebs-csi-driver-node-v956g\" (UID: \"ea5abfee-bd02-41ce-a0f9-189e5c238bec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v956g" Jun 15 02:28:29.540847 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.540409 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/95473860-f2a8-4daa-b746-dac03c5462be-run\") pod \"tuned-wpq45\" (UID: \"95473860-f2a8-4daa-b746-dac03c5462be\") " pod="openshift-cluster-node-tuning-operator/tuned-wpq45" Jun 15 02:28:29.540847 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.540410 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-run-ovn\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.540847 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.540445 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-host-cni-bin\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.540847 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.540525 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-host-run-netns\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.540847 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.540582 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/95473860-f2a8-4daa-b746-dac03c5462be-tmp\") pod \"tuned-wpq45\" (UID: \"95473860-f2a8-4daa-b746-dac03c5462be\") " pod="openshift-cluster-node-tuning-operator/tuned-wpq45" Jun 15 02:28:29.540847 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.540619 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fc66bce4-865f-4b30-aca2-93cedd05ffe3-agent-certs\") pod \"konnectivity-agent-tsd5w\" (UID: \"fc66bce4-865f-4b30-aca2-93cedd05ffe3\") " pod="kube-system/konnectivity-agent-tsd5w" Jun 15 02:28:29.540847 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.540677 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-systemd-units\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.541628 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.540706 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-var-lib-openvswitch\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.541628 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.540737 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.541628 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.540767 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ea5abfee-bd02-41ce-a0f9-189e5c238bec-kubelet-dir\") pod \"aws-ebs-csi-driver-node-v956g\" (UID: \"ea5abfee-bd02-41ce-a0f9-189e5c238bec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v956g" Jun 15 02:28:29.541628 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.540790 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4-multus-cni-dir\") pod \"multus-lnjtl\" (UID: \"e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4\") " pod="openshift-multus/multus-lnjtl" Jun 15 02:28:29.541628 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.540819 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/95473860-f2a8-4daa-b746-dac03c5462be-etc-tuned\") pod \"tuned-wpq45\" (UID: \"95473860-f2a8-4daa-b746-dac03c5462be\") " pod="openshift-cluster-node-tuning-operator/tuned-wpq45" Jun 15 02:28:29.541628 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.540849 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fc48h\" (UniqueName: \"kubernetes.io/projected/f96eb18e-3121-4b4e-a6fb-a0c8bd807285-kube-api-access-fc48h\") pod \"node-resolver-wthd7\" (UID: \"f96eb18e-3121-4b4e-a6fb-a0c8bd807285\") " pod="openshift-dns/node-resolver-wthd7" Jun 15 02:28:29.541628 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.540876 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-env-overrides\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.541628 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.540946 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-systemd-units\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.541628 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.540991 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-var-lib-openvswitch\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.541628 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:29.541003 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 02:28:29.541628 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.541032 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.541628 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:29.541071 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c770be9-5ffa-4da5-bc85-58fd4c82b027-metrics-certs podName:0c770be9-5ffa-4da5-bc85-58fd4c82b027 nodeName:}" failed. No retries permitted until 2026-06-15 02:28:30.041052085 +0000 UTC m=+2.908836808 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0c770be9-5ffa-4da5-bc85-58fd4c82b027-metrics-certs") pod "network-metrics-daemon-954mx" (UID: "0c770be9-5ffa-4da5-bc85-58fd4c82b027") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 02:28:29.541628 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.541084 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ea5abfee-bd02-41ce-a0f9-189e5c238bec-kubelet-dir\") pod \"aws-ebs-csi-driver-node-v956g\" (UID: \"ea5abfee-bd02-41ce-a0f9-189e5c238bec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v956g" Jun 15 02:28:29.541628 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.541125 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-run-openvswitch\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.541628 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.540881 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0c770be9-5ffa-4da5-bc85-58fd4c82b027-metrics-certs\") pod \"network-metrics-daemon-954mx\" (UID: \"0c770be9-5ffa-4da5-bc85-58fd4c82b027\") " pod="openshift-multus/network-metrics-daemon-954mx" Jun 15 02:28:29.541628 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.541171 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4-cnibin\") pod \"multus-lnjtl\" (UID: \"e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4\") " pod="openshift-multus/multus-lnjtl" Jun 15 02:28:29.541628 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.541204 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4-host-var-lib-cni-bin\") pod \"multus-lnjtl\" (UID: \"e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4\") " pod="openshift-multus/multus-lnjtl" Jun 15 02:28:29.542431 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.541236 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f96eb18e-3121-4b4e-a6fb-a0c8bd807285-hosts-file\") pod \"node-resolver-wthd7\" (UID: \"f96eb18e-3121-4b4e-a6fb-a0c8bd807285\") " pod="openshift-dns/node-resolver-wthd7" Jun 15 02:28:29.542431 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.541261 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4-host-run-k8s-cni-cncf-io\") pod \"multus-lnjtl\" (UID: \"e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4\") " pod="openshift-multus/multus-lnjtl" Jun 15 02:28:29.542431 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.541290 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4-hostroot\") pod \"multus-lnjtl\" (UID: \"e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4\") " pod="openshift-multus/multus-lnjtl" Jun 15 02:28:29.542431 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.541323 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4-etc-kubernetes\") pod \"multus-lnjtl\" (UID: \"e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4\") " pod="openshift-multus/multus-lnjtl" Jun 15 02:28:29.542431 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.541375 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95473860-f2a8-4daa-b746-dac03c5462be-etc-kubernetes\") pod \"tuned-wpq45\" (UID: \"95473860-f2a8-4daa-b746-dac03c5462be\") " pod="openshift-cluster-node-tuning-operator/tuned-wpq45" Jun 15 02:28:29.542431 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.541407 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/95473860-f2a8-4daa-b746-dac03c5462be-var-lib-kubelet\") pod \"tuned-wpq45\" (UID: \"95473860-f2a8-4daa-b746-dac03c5462be\") " pod="openshift-cluster-node-tuning-operator/tuned-wpq45" Jun 15 02:28:29.542431 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.541434 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/54db0e1c-0d25-46d2-a3e3-aebf9339e272-cnibin\") pod \"multus-additional-cni-plugins-5hrsv\" (UID: \"54db0e1c-0d25-46d2-a3e3-aebf9339e272\") " pod="openshift-multus/multus-additional-cni-plugins-5hrsv" Jun 15 02:28:29.542431 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.541460 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ea5abfee-bd02-41ce-a0f9-189e5c238bec-device-dir\") pod \"aws-ebs-csi-driver-node-v956g\" (UID: \"ea5abfee-bd02-41ce-a0f9-189e5c238bec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v956g" Jun 15 02:28:29.542431 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.541488 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4-multus-conf-dir\") pod \"multus-lnjtl\" (UID: \"e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4\") " pod="openshift-multus/multus-lnjtl" Jun 15 02:28:29.542431 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.541540 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6798a601-4421-44c1-8ef7-3b2527cad953-host\") pod \"node-ca-9zrhr\" (UID: \"6798a601-4421-44c1-8ef7-3b2527cad953\") " pod="openshift-image-registry/node-ca-9zrhr" Jun 15 02:28:29.542431 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.541569 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/54db0e1c-0d25-46d2-a3e3-aebf9339e272-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5hrsv\" (UID: \"54db0e1c-0d25-46d2-a3e3-aebf9339e272\") " pod="openshift-multus/multus-additional-cni-plugins-5hrsv" Jun 15 02:28:29.542431 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.541602 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvrg6\" (UniqueName: \"kubernetes.io/projected/0c770be9-5ffa-4da5-bc85-58fd4c82b027-kube-api-access-cvrg6\") pod \"network-metrics-daemon-954mx\" (UID: \"0c770be9-5ffa-4da5-bc85-58fd4c82b027\") " pod="openshift-multus/network-metrics-daemon-954mx" Jun 15 02:28:29.542431 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.541666 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ea5abfee-bd02-41ce-a0f9-189e5c238bec-device-dir\") pod \"aws-ebs-csi-driver-node-v956g\" (UID: \"ea5abfee-bd02-41ce-a0f9-189e5c238bec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v956g" Jun 15 02:28:29.542431 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.541717 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4-multus-daemon-config\") pod \"multus-lnjtl\" (UID: \"e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4\") " pod="openshift-multus/multus-lnjtl" Jun 15 02:28:29.542431 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.541753 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4-host-run-multus-certs\") pod \"multus-lnjtl\" (UID: \"e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4\") " pod="openshift-multus/multus-lnjtl" Jun 15 02:28:29.542431 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.541787 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d436483f-cbb4-4ce9-b4d5-a21564cf6e97-original-pull-secret\") pod \"global-pull-secret-syncer-qnxs7\" (UID: \"d436483f-cbb4-4ce9-b4d5-a21564cf6e97\") " pod="kube-system/global-pull-secret-syncer-qnxs7" Jun 15 02:28:29.543205 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.541814 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/54db0e1c-0d25-46d2-a3e3-aebf9339e272-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5hrsv\" (UID: \"54db0e1c-0d25-46d2-a3e3-aebf9339e272\") " pod="openshift-multus/multus-additional-cni-plugins-5hrsv" Jun 15 02:28:29.543205 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:29.543080 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 15 02:28:29.543205 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:29.543110 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 15 02:28:29.543205 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:29.543126 2579 projected.go:196] Error preparing data for projected volume kube-api-access-l75z9 for pod openshift-network-diagnostics/network-check-target-5qfdl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 02:28:29.543205 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.543136 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-ovn-node-metrics-cert\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.543205 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:29.543192 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/91285de0-6ca5-4b23-b554-0c06718dc1df-kube-api-access-l75z9 podName:91285de0-6ca5-4b23-b554-0c06718dc1df nodeName:}" failed. No retries permitted until 2026-06-15 02:28:30.043173176 +0000 UTC m=+2.910957917 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-l75z9" (UniqueName: "kubernetes.io/projected/91285de0-6ca5-4b23-b554-0c06718dc1df-kube-api-access-l75z9") pod "network-check-target-5qfdl" (UID: "91285de0-6ca5-4b23-b554-0c06718dc1df") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 02:28:29.545369 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.545339 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nntp6\" (UniqueName: \"kubernetes.io/projected/0b0dd3a3-9aef-4f34-8499-c293d4d81a93-kube-api-access-nntp6\") pod \"ovnkube-node-qg4vc\" (UID: \"0b0dd3a3-9aef-4f34-8499-c293d4d81a93\") " pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.548579 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.548553 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztfbc\" (UniqueName: \"kubernetes.io/projected/f2a4bcb6-c233-4ccc-9fc1-88c268ae2fb6-kube-api-access-ztfbc\") pod \"iptables-alerter-p76zd\" (UID: \"f2a4bcb6-c233-4ccc-9fc1-88c268ae2fb6\") " pod="openshift-network-operator/iptables-alerter-p76zd" Jun 15 02:28:29.550760 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.550740 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvrg6\" (UniqueName: \"kubernetes.io/projected/0c770be9-5ffa-4da5-bc85-58fd4c82b027-kube-api-access-cvrg6\") pod \"network-metrics-daemon-954mx\" (UID: \"0c770be9-5ffa-4da5-bc85-58fd4c82b027\") " pod="openshift-multus/network-metrics-daemon-954mx" Jun 15 02:28:29.550836 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.550816 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvblx\" (UniqueName: \"kubernetes.io/projected/ea5abfee-bd02-41ce-a0f9-189e5c238bec-kube-api-access-hvblx\") pod \"aws-ebs-csi-driver-node-v956g\" (UID: \"ea5abfee-bd02-41ce-a0f9-189e5c238bec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v956g" Jun 15 02:28:29.642922 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.642892 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/95473860-f2a8-4daa-b746-dac03c5462be-var-lib-kubelet\") pod \"tuned-wpq45\" (UID: \"95473860-f2a8-4daa-b746-dac03c5462be\") " pod="openshift-cluster-node-tuning-operator/tuned-wpq45" Jun 15 02:28:29.642922 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.642927 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/54db0e1c-0d25-46d2-a3e3-aebf9339e272-cnibin\") pod \"multus-additional-cni-plugins-5hrsv\" (UID: \"54db0e1c-0d25-46d2-a3e3-aebf9339e272\") " pod="openshift-multus/multus-additional-cni-plugins-5hrsv" Jun 15 02:28:29.643113 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.642945 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4-multus-conf-dir\") pod \"multus-lnjtl\" (UID: \"e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4\") " pod="openshift-multus/multus-lnjtl" Jun 15 02:28:29.643113 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.643002 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4-multus-conf-dir\") pod \"multus-lnjtl\" (UID: \"e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4\") " pod="openshift-multus/multus-lnjtl" Jun 15 02:28:29.643113 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.643012 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/95473860-f2a8-4daa-b746-dac03c5462be-var-lib-kubelet\") pod \"tuned-wpq45\" (UID: \"95473860-f2a8-4daa-b746-dac03c5462be\") " pod="openshift-cluster-node-tuning-operator/tuned-wpq45" Jun 15 02:28:29.643113 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.643019 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/54db0e1c-0d25-46d2-a3e3-aebf9339e272-cnibin\") pod \"multus-additional-cni-plugins-5hrsv\" (UID: \"54db0e1c-0d25-46d2-a3e3-aebf9339e272\") " pod="openshift-multus/multus-additional-cni-plugins-5hrsv" Jun 15 02:28:29.643113 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.643044 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6798a601-4421-44c1-8ef7-3b2527cad953-host\") pod \"node-ca-9zrhr\" (UID: \"6798a601-4421-44c1-8ef7-3b2527cad953\") " pod="openshift-image-registry/node-ca-9zrhr" Jun 15 02:28:29.643113 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.643067 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/54db0e1c-0d25-46d2-a3e3-aebf9339e272-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5hrsv\" (UID: \"54db0e1c-0d25-46d2-a3e3-aebf9339e272\") " pod="openshift-multus/multus-additional-cni-plugins-5hrsv" Jun 15 02:28:29.643113 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.643095 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4-multus-daemon-config\") pod \"multus-lnjtl\" (UID: \"e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4\") " pod="openshift-multus/multus-lnjtl" Jun 15 02:28:29.643443 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.643119 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4-host-run-multus-certs\") pod \"multus-lnjtl\" (UID: \"e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4\") " pod="openshift-multus/multus-lnjtl" Jun 15 02:28:29.643443 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.643124 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6798a601-4421-44c1-8ef7-3b2527cad953-host\") pod \"node-ca-9zrhr\" (UID: \"6798a601-4421-44c1-8ef7-3b2527cad953\") " pod="openshift-image-registry/node-ca-9zrhr" Jun 15 02:28:29.643443 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.643141 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d436483f-cbb4-4ce9-b4d5-a21564cf6e97-original-pull-secret\") pod \"global-pull-secret-syncer-qnxs7\" (UID: \"d436483f-cbb4-4ce9-b4d5-a21564cf6e97\") " pod="kube-system/global-pull-secret-syncer-qnxs7" Jun 15 02:28:29.643443 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.643165 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/54db0e1c-0d25-46d2-a3e3-aebf9339e272-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5hrsv\" (UID: \"54db0e1c-0d25-46d2-a3e3-aebf9339e272\") " pod="openshift-multus/multus-additional-cni-plugins-5hrsv" Jun 15 02:28:29.643443 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.643190 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4-host-run-netns\") pod \"multus-lnjtl\" (UID: \"e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4\") " pod="openshift-multus/multus-lnjtl" Jun 15 02:28:29.643443 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.643212 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4-host-var-lib-cni-multus\") pod \"multus-lnjtl\" (UID: \"e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4\") " pod="openshift-multus/multus-lnjtl" Jun 15 02:28:29.643443 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.643235 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdnx7\" (UniqueName: \"kubernetes.io/projected/6798a601-4421-44c1-8ef7-3b2527cad953-kube-api-access-sdnx7\") pod \"node-ca-9zrhr\" (UID: \"6798a601-4421-44c1-8ef7-3b2527cad953\") " pod="openshift-image-registry/node-ca-9zrhr" Jun 15 02:28:29.643443 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.643292 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4-system-cni-dir\") pod \"multus-lnjtl\" (UID: \"e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4\") " pod="openshift-multus/multus-lnjtl" Jun 15 02:28:29.643443 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.643313 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/95473860-f2a8-4daa-b746-dac03c5462be-lib-modules\") pod \"tuned-wpq45\" (UID: \"95473860-f2a8-4daa-b746-dac03c5462be\") " pod="openshift-cluster-node-tuning-operator/tuned-wpq45" Jun 15 02:28:29.643443 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.643337 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/95473860-f2a8-4daa-b746-dac03c5462be-etc-sysctl-d\") pod \"tuned-wpq45\" (UID: \"95473860-f2a8-4daa-b746-dac03c5462be\") " pod="openshift-cluster-node-tuning-operator/tuned-wpq45" Jun 15 02:28:29.643443 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.643359 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/54db0e1c-0d25-46d2-a3e3-aebf9339e272-os-release\") pod \"multus-additional-cni-plugins-5hrsv\" (UID: \"54db0e1c-0d25-46d2-a3e3-aebf9339e272\") " pod="openshift-multus/multus-additional-cni-plugins-5hrsv" Jun 15 02:28:29.643443 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.643383 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fc66bce4-865f-4b30-aca2-93cedd05ffe3-konnectivity-ca\") pod \"konnectivity-agent-tsd5w\" (UID: \"fc66bce4-865f-4b30-aca2-93cedd05ffe3\") " pod="kube-system/konnectivity-agent-tsd5w" Jun 15 02:28:29.643443 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.643407 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4-cni-binary-copy\") pod \"multus-lnjtl\" (UID: \"e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4\") " pod="openshift-multus/multus-lnjtl" Jun 15 02:28:29.643443 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.643429 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/95473860-f2a8-4daa-b746-dac03c5462be-sys\") pod \"tuned-wpq45\" (UID: \"95473860-f2a8-4daa-b746-dac03c5462be\") " pod="openshift-cluster-node-tuning-operator/tuned-wpq45" Jun 15 02:28:29.643443 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.643449 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/95473860-f2a8-4daa-b746-dac03c5462be-host\") pod \"tuned-wpq45\" (UID: \"95473860-f2a8-4daa-b746-dac03c5462be\") " pod="openshift-cluster-node-tuning-operator/tuned-wpq45" Jun 15 02:28:29.644155 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.643472 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/54db0e1c-0d25-46d2-a3e3-aebf9339e272-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5hrsv\" (UID: \"54db0e1c-0d25-46d2-a3e3-aebf9339e272\") " pod="openshift-multus/multus-additional-cni-plugins-5hrsv" Jun 15 02:28:29.644155 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.643502 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/95473860-f2a8-4daa-b746-dac03c5462be-etc-systemd\") pod \"tuned-wpq45\" (UID: \"95473860-f2a8-4daa-b746-dac03c5462be\") " pod="openshift-cluster-node-tuning-operator/tuned-wpq45" Jun 15 02:28:29.644155 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.643529 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/95473860-f2a8-4daa-b746-dac03c5462be-etc-sysconfig\") pod \"tuned-wpq45\" (UID: \"95473860-f2a8-4daa-b746-dac03c5462be\") " pod="openshift-cluster-node-tuning-operator/tuned-wpq45" Jun 15 02:28:29.644155 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.643554 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4-multus-socket-dir-parent\") pod \"multus-lnjtl\" (UID: \"e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4\") " pod="openshift-multus/multus-lnjtl" Jun 15 02:28:29.644155 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.643596 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9m26g\" (UniqueName: \"kubernetes.io/projected/e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4-kube-api-access-9m26g\") pod \"multus-lnjtl\" (UID: \"e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4\") " pod="openshift-multus/multus-lnjtl" Jun 15 02:28:29.644155 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.643620 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4v8dm\" (UniqueName: \"kubernetes.io/projected/95473860-f2a8-4daa-b746-dac03c5462be-kube-api-access-4v8dm\") pod \"tuned-wpq45\" (UID: \"95473860-f2a8-4daa-b746-dac03c5462be\") " pod="openshift-cluster-node-tuning-operator/tuned-wpq45" Jun 15 02:28:29.644155 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.643666 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/54db0e1c-0d25-46d2-a3e3-aebf9339e272-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5hrsv\" (UID: \"54db0e1c-0d25-46d2-a3e3-aebf9339e272\") " pod="openshift-multus/multus-additional-cni-plugins-5hrsv" Jun 15 02:28:29.644155 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.643672 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6798a601-4421-44c1-8ef7-3b2527cad953-serviceca\") pod \"node-ca-9zrhr\" (UID: \"6798a601-4421-44c1-8ef7-3b2527cad953\") " pod="openshift-image-registry/node-ca-9zrhr" Jun 15 02:28:29.644155 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.643707 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d436483f-cbb4-4ce9-b4d5-a21564cf6e97-kubelet-config\") pod \"global-pull-secret-syncer-qnxs7\" (UID: \"d436483f-cbb4-4ce9-b4d5-a21564cf6e97\") " pod="kube-system/global-pull-secret-syncer-qnxs7" Jun 15 02:28:29.644155 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.643734 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/54db0e1c-0d25-46d2-a3e3-aebf9339e272-system-cni-dir\") pod \"multus-additional-cni-plugins-5hrsv\" (UID: \"54db0e1c-0d25-46d2-a3e3-aebf9339e272\") " pod="openshift-multus/multus-additional-cni-plugins-5hrsv" Jun 15 02:28:29.644155 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.643734 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/54db0e1c-0d25-46d2-a3e3-aebf9339e272-os-release\") pod \"multus-additional-cni-plugins-5hrsv\" (UID: \"54db0e1c-0d25-46d2-a3e3-aebf9339e272\") " pod="openshift-multus/multus-additional-cni-plugins-5hrsv" Jun 15 02:28:29.644155 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.643758 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/54db0e1c-0d25-46d2-a3e3-aebf9339e272-cni-binary-copy\") pod \"multus-additional-cni-plugins-5hrsv\" (UID: \"54db0e1c-0d25-46d2-a3e3-aebf9339e272\") " pod="openshift-multus/multus-additional-cni-plugins-5hrsv" Jun 15 02:28:29.644155 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.643782 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4-os-release\") pod \"multus-lnjtl\" (UID: \"e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4\") " pod="openshift-multus/multus-lnjtl" Jun 15 02:28:29.644155 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.643790 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4-host-run-multus-certs\") pod \"multus-lnjtl\" (UID: \"e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4\") " pod="openshift-multus/multus-lnjtl" Jun 15 02:28:29.644155 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.643806 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/95473860-f2a8-4daa-b746-dac03c5462be-etc-modprobe-d\") pod \"tuned-wpq45\" (UID: \"95473860-f2a8-4daa-b746-dac03c5462be\") " pod="openshift-cluster-node-tuning-operator/tuned-wpq45" Jun 15 02:28:29.644155 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.643831 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/95473860-f2a8-4daa-b746-dac03c5462be-etc-sysctl-conf\") pod \"tuned-wpq45\" (UID: \"95473860-f2a8-4daa-b746-dac03c5462be\") " pod="openshift-cluster-node-tuning-operator/tuned-wpq45" Jun 15 02:28:29.644155 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.643854 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d436483f-cbb4-4ce9-b4d5-a21564cf6e97-dbus\") pod \"global-pull-secret-syncer-qnxs7\" (UID: \"d436483f-cbb4-4ce9-b4d5-a21564cf6e97\") " pod="kube-system/global-pull-secret-syncer-qnxs7" Jun 15 02:28:29.644155 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:29.643871 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 15 02:28:29.645026 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.643878 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f96eb18e-3121-4b4e-a6fb-a0c8bd807285-tmp-dir\") pod \"node-resolver-wthd7\" (UID: \"f96eb18e-3121-4b4e-a6fb-a0c8bd807285\") " pod="openshift-dns/node-resolver-wthd7" Jun 15 02:28:29.645026 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.643908 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4-host-var-lib-kubelet\") pod \"multus-lnjtl\" (UID: \"e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4\") " pod="openshift-multus/multus-lnjtl" Jun 15 02:28:29.645026 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:29.643922 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d436483f-cbb4-4ce9-b4d5-a21564cf6e97-original-pull-secret podName:d436483f-cbb4-4ce9-b4d5-a21564cf6e97 nodeName:}" failed. No retries permitted until 2026-06-15 02:28:30.143904362 +0000 UTC m=+3.011689100 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d436483f-cbb4-4ce9-b4d5-a21564cf6e97-original-pull-secret") pod "global-pull-secret-syncer-qnxs7" (UID: "d436483f-cbb4-4ce9-b4d5-a21564cf6e97") : object "kube-system"/"original-pull-secret" not registered Jun 15 02:28:29.645026 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.643946 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4-host-var-lib-kubelet\") pod \"multus-lnjtl\" (UID: \"e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4\") " pod="openshift-multus/multus-lnjtl" Jun 15 02:28:29.645026 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.643956 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bfpv\" (UniqueName: \"kubernetes.io/projected/54db0e1c-0d25-46d2-a3e3-aebf9339e272-kube-api-access-2bfpv\") pod \"multus-additional-cni-plugins-5hrsv\" (UID: \"54db0e1c-0d25-46d2-a3e3-aebf9339e272\") " pod="openshift-multus/multus-additional-cni-plugins-5hrsv" Jun 15 02:28:29.645026 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.643990 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/95473860-f2a8-4daa-b746-dac03c5462be-run\") pod \"tuned-wpq45\" (UID: \"95473860-f2a8-4daa-b746-dac03c5462be\") " pod="openshift-cluster-node-tuning-operator/tuned-wpq45" Jun 15 02:28:29.645026 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.643997 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d436483f-cbb4-4ce9-b4d5-a21564cf6e97-kubelet-config\") pod \"global-pull-secret-syncer-qnxs7\" (UID: \"d436483f-cbb4-4ce9-b4d5-a21564cf6e97\") " pod="kube-system/global-pull-secret-syncer-qnxs7" Jun 15 02:28:29.645026 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.644014 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/95473860-f2a8-4daa-b746-dac03c5462be-tmp\") pod \"tuned-wpq45\" (UID: \"95473860-f2a8-4daa-b746-dac03c5462be\") " pod="openshift-cluster-node-tuning-operator/tuned-wpq45" Jun 15 02:28:29.645026 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.644039 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/54db0e1c-0d25-46d2-a3e3-aebf9339e272-system-cni-dir\") pod \"multus-additional-cni-plugins-5hrsv\" (UID: \"54db0e1c-0d25-46d2-a3e3-aebf9339e272\") " pod="openshift-multus/multus-additional-cni-plugins-5hrsv" Jun 15 02:28:29.645026 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.644040 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6798a601-4421-44c1-8ef7-3b2527cad953-serviceca\") pod \"node-ca-9zrhr\" (UID: \"6798a601-4421-44c1-8ef7-3b2527cad953\") " pod="openshift-image-registry/node-ca-9zrhr" Jun 15 02:28:29.645026 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.644040 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fc66bce4-865f-4b30-aca2-93cedd05ffe3-agent-certs\") pod \"konnectivity-agent-tsd5w\" (UID: \"fc66bce4-865f-4b30-aca2-93cedd05ffe3\") " pod="kube-system/konnectivity-agent-tsd5w" Jun 15 02:28:29.645026 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.644085 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4-multus-cni-dir\") pod \"multus-lnjtl\" (UID: \"e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4\") " pod="openshift-multus/multus-lnjtl" Jun 15 02:28:29.645026 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.644108 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/95473860-f2a8-4daa-b746-dac03c5462be-etc-tuned\") pod \"tuned-wpq45\" (UID: \"95473860-f2a8-4daa-b746-dac03c5462be\") " pod="openshift-cluster-node-tuning-operator/tuned-wpq45" Jun 15 02:28:29.645026 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.644136 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fc48h\" (UniqueName: \"kubernetes.io/projected/f96eb18e-3121-4b4e-a6fb-a0c8bd807285-kube-api-access-fc48h\") pod \"node-resolver-wthd7\" (UID: \"f96eb18e-3121-4b4e-a6fb-a0c8bd807285\") " pod="openshift-dns/node-resolver-wthd7" Jun 15 02:28:29.645026 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.644173 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4-cnibin\") pod \"multus-lnjtl\" (UID: \"e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4\") " pod="openshift-multus/multus-lnjtl" Jun 15 02:28:29.645026 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.644239 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4-host-var-lib-cni-bin\") pod \"multus-lnjtl\" (UID: \"e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4\") " pod="openshift-multus/multus-lnjtl" Jun 15 02:28:29.645026 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.644265 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f96eb18e-3121-4b4e-a6fb-a0c8bd807285-hosts-file\") pod \"node-resolver-wthd7\" (UID: \"f96eb18e-3121-4b4e-a6fb-a0c8bd807285\") " pod="openshift-dns/node-resolver-wthd7" Jun 15 02:28:29.645873 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.644296 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4-host-run-k8s-cni-cncf-io\") pod \"multus-lnjtl\" (UID: \"e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4\") " pod="openshift-multus/multus-lnjtl" Jun 15 02:28:29.645873 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.644319 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4-hostroot\") pod \"multus-lnjtl\" (UID: \"e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4\") " pod="openshift-multus/multus-lnjtl" Jun 15 02:28:29.645873 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.644342 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4-etc-kubernetes\") pod \"multus-lnjtl\" (UID: \"e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4\") " pod="openshift-multus/multus-lnjtl" Jun 15 02:28:29.645873 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.644367 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95473860-f2a8-4daa-b746-dac03c5462be-etc-kubernetes\") pod \"tuned-wpq45\" (UID: \"95473860-f2a8-4daa-b746-dac03c5462be\") " pod="openshift-cluster-node-tuning-operator/tuned-wpq45" Jun 15 02:28:29.645873 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.644434 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4-multus-cni-dir\") pod \"multus-lnjtl\" (UID: \"e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4\") " pod="openshift-multus/multus-lnjtl" Jun 15 02:28:29.645873 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.644456 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95473860-f2a8-4daa-b746-dac03c5462be-etc-kubernetes\") pod \"tuned-wpq45\" (UID: \"95473860-f2a8-4daa-b746-dac03c5462be\") " pod="openshift-cluster-node-tuning-operator/tuned-wpq45" Jun 15 02:28:29.645873 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.644516 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/95473860-f2a8-4daa-b746-dac03c5462be-lib-modules\") pod \"tuned-wpq45\" (UID: \"95473860-f2a8-4daa-b746-dac03c5462be\") " pod="openshift-cluster-node-tuning-operator/tuned-wpq45" Jun 15 02:28:29.645873 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.644577 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/95473860-f2a8-4daa-b746-dac03c5462be-sys\") pod \"tuned-wpq45\" (UID: \"95473860-f2a8-4daa-b746-dac03c5462be\") " pod="openshift-cluster-node-tuning-operator/tuned-wpq45" Jun 15 02:28:29.645873 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.644573 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/95473860-f2a8-4daa-b746-dac03c5462be-host\") pod \"tuned-wpq45\" (UID: \"95473860-f2a8-4daa-b746-dac03c5462be\") " pod="openshift-cluster-node-tuning-operator/tuned-wpq45" Jun 15 02:28:29.645873 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.644522 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4-os-release\") pod \"multus-lnjtl\" (UID: \"e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4\") " pod="openshift-multus/multus-lnjtl" Jun 15 02:28:29.645873 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.644604 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/95473860-f2a8-4daa-b746-dac03c5462be-etc-modprobe-d\") pod \"tuned-wpq45\" (UID: \"95473860-f2a8-4daa-b746-dac03c5462be\") " pod="openshift-cluster-node-tuning-operator/tuned-wpq45" Jun 15 02:28:29.645873 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.644656 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4-host-run-netns\") pod \"multus-lnjtl\" (UID: \"e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4\") " pod="openshift-multus/multus-lnjtl" Jun 15 02:28:29.645873 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.644707 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4-host-var-lib-cni-multus\") pod \"multus-lnjtl\" (UID: \"e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4\") " pod="openshift-multus/multus-lnjtl" Jun 15 02:28:29.645873 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.644729 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/95473860-f2a8-4daa-b746-dac03c5462be-etc-sysctl-conf\") pod \"tuned-wpq45\" (UID: \"95473860-f2a8-4daa-b746-dac03c5462be\") " pod="openshift-cluster-node-tuning-operator/tuned-wpq45" Jun 15 02:28:29.645873 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.644793 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/95473860-f2a8-4daa-b746-dac03c5462be-etc-sysconfig\") pod \"tuned-wpq45\" (UID: \"95473860-f2a8-4daa-b746-dac03c5462be\") " pod="openshift-cluster-node-tuning-operator/tuned-wpq45" Jun 15 02:28:29.645873 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.644830 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4-multus-socket-dir-parent\") pod \"multus-lnjtl\" (UID: \"e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4\") " pod="openshift-multus/multus-lnjtl" Jun 15 02:28:29.645873 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.644952 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4-system-cni-dir\") pod \"multus-lnjtl\" (UID: \"e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4\") " pod="openshift-multus/multus-lnjtl" Jun 15 02:28:29.645873 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.644519 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fc66bce4-865f-4b30-aca2-93cedd05ffe3-konnectivity-ca\") pod \"konnectivity-agent-tsd5w\" (UID: \"fc66bce4-865f-4b30-aca2-93cedd05ffe3\") " pod="kube-system/konnectivity-agent-tsd5w" Jun 15 02:28:29.646658 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.644976 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4-multus-daemon-config\") pod \"multus-lnjtl\" (UID: \"e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4\") " pod="openshift-multus/multus-lnjtl" Jun 15 02:28:29.646658 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.645052 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f96eb18e-3121-4b4e-a6fb-a0c8bd807285-hosts-file\") pod \"node-resolver-wthd7\" (UID: \"f96eb18e-3121-4b4e-a6fb-a0c8bd807285\") " pod="openshift-dns/node-resolver-wthd7" Jun 15 02:28:29.646658 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.645196 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4-cnibin\") pod \"multus-lnjtl\" (UID: \"e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4\") " pod="openshift-multus/multus-lnjtl" Jun 15 02:28:29.646658 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.645227 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/54db0e1c-0d25-46d2-a3e3-aebf9339e272-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5hrsv\" (UID: \"54db0e1c-0d25-46d2-a3e3-aebf9339e272\") " pod="openshift-multus/multus-additional-cni-plugins-5hrsv" Jun 15 02:28:29.646658 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.645238 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4-hostroot\") pod \"multus-lnjtl\" (UID: \"e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4\") " pod="openshift-multus/multus-lnjtl" Jun 15 02:28:29.646658 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.645245 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4-host-var-lib-cni-bin\") pod \"multus-lnjtl\" (UID: \"e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4\") " pod="openshift-multus/multus-lnjtl" Jun 15 02:28:29.646658 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.645277 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4-host-run-k8s-cni-cncf-io\") pod \"multus-lnjtl\" (UID: \"e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4\") " pod="openshift-multus/multus-lnjtl" Jun 15 02:28:29.646658 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.645302 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d436483f-cbb4-4ce9-b4d5-a21564cf6e97-dbus\") pod \"global-pull-secret-syncer-qnxs7\" (UID: \"d436483f-cbb4-4ce9-b4d5-a21564cf6e97\") " pod="kube-system/global-pull-secret-syncer-qnxs7" Jun 15 02:28:29.646658 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.645323 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/95473860-f2a8-4daa-b746-dac03c5462be-run\") pod \"tuned-wpq45\" (UID: \"95473860-f2a8-4daa-b746-dac03c5462be\") " pod="openshift-cluster-node-tuning-operator/tuned-wpq45" Jun 15 02:28:29.646658 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.645409 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4-cni-binary-copy\") pod \"multus-lnjtl\" (UID: \"e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4\") " pod="openshift-multus/multus-lnjtl" Jun 15 02:28:29.646658 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.645466 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4-etc-kubernetes\") pod \"multus-lnjtl\" (UID: \"e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4\") " pod="openshift-multus/multus-lnjtl" Jun 15 02:28:29.646658 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.645554 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/95473860-f2a8-4daa-b746-dac03c5462be-etc-sysctl-d\") pod \"tuned-wpq45\" (UID: \"95473860-f2a8-4daa-b746-dac03c5462be\") " pod="openshift-cluster-node-tuning-operator/tuned-wpq45" Jun 15 02:28:29.646658 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.645555 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f96eb18e-3121-4b4e-a6fb-a0c8bd807285-tmp-dir\") pod \"node-resolver-wthd7\" (UID: \"f96eb18e-3121-4b4e-a6fb-a0c8bd807285\") " pod="openshift-dns/node-resolver-wthd7" Jun 15 02:28:29.646658 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.644740 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/54db0e1c-0d25-46d2-a3e3-aebf9339e272-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5hrsv\" (UID: \"54db0e1c-0d25-46d2-a3e3-aebf9339e272\") " pod="openshift-multus/multus-additional-cni-plugins-5hrsv" Jun 15 02:28:29.646658 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.645618 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/95473860-f2a8-4daa-b746-dac03c5462be-etc-systemd\") pod \"tuned-wpq45\" (UID: \"95473860-f2a8-4daa-b746-dac03c5462be\") " pod="openshift-cluster-node-tuning-operator/tuned-wpq45" Jun 15 02:28:29.646658 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.645895 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/54db0e1c-0d25-46d2-a3e3-aebf9339e272-cni-binary-copy\") pod \"multus-additional-cni-plugins-5hrsv\" (UID: \"54db0e1c-0d25-46d2-a3e3-aebf9339e272\") " pod="openshift-multus/multus-additional-cni-plugins-5hrsv" Jun 15 02:28:29.646658 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.646659 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/95473860-f2a8-4daa-b746-dac03c5462be-etc-tuned\") pod \"tuned-wpq45\" (UID: \"95473860-f2a8-4daa-b746-dac03c5462be\") " pod="openshift-cluster-node-tuning-operator/tuned-wpq45" Jun 15 02:28:29.647175 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.646717 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fc66bce4-865f-4b30-aca2-93cedd05ffe3-agent-certs\") pod \"konnectivity-agent-tsd5w\" (UID: \"fc66bce4-865f-4b30-aca2-93cedd05ffe3\") " pod="kube-system/konnectivity-agent-tsd5w" Jun 15 02:28:29.647737 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.647713 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/95473860-f2a8-4daa-b746-dac03c5462be-tmp\") pod \"tuned-wpq45\" (UID: \"95473860-f2a8-4daa-b746-dac03c5462be\") " pod="openshift-cluster-node-tuning-operator/tuned-wpq45" Jun 15 02:28:29.654332 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.654020 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9m26g\" (UniqueName: \"kubernetes.io/projected/e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4-kube-api-access-9m26g\") pod \"multus-lnjtl\" (UID: \"e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4\") " pod="openshift-multus/multus-lnjtl" Jun 15 02:28:29.654332 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.654157 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fc48h\" (UniqueName: \"kubernetes.io/projected/f96eb18e-3121-4b4e-a6fb-a0c8bd807285-kube-api-access-fc48h\") pod \"node-resolver-wthd7\" (UID: \"f96eb18e-3121-4b4e-a6fb-a0c8bd807285\") " pod="openshift-dns/node-resolver-wthd7" Jun 15 02:28:29.654332 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.654295 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4v8dm\" (UniqueName: \"kubernetes.io/projected/95473860-f2a8-4daa-b746-dac03c5462be-kube-api-access-4v8dm\") pod \"tuned-wpq45\" (UID: \"95473860-f2a8-4daa-b746-dac03c5462be\") " pod="openshift-cluster-node-tuning-operator/tuned-wpq45" Jun 15 02:28:29.654932 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.654915 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdnx7\" (UniqueName: \"kubernetes.io/projected/6798a601-4421-44c1-8ef7-3b2527cad953-kube-api-access-sdnx7\") pod \"node-ca-9zrhr\" (UID: \"6798a601-4421-44c1-8ef7-3b2527cad953\") " pod="openshift-image-registry/node-ca-9zrhr" Jun 15 02:28:29.655014 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.654990 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bfpv\" (UniqueName: \"kubernetes.io/projected/54db0e1c-0d25-46d2-a3e3-aebf9339e272-kube-api-access-2bfpv\") pod \"multus-additional-cni-plugins-5hrsv\" (UID: \"54db0e1c-0d25-46d2-a3e3-aebf9339e272\") " pod="openshift-multus/multus-additional-cni-plugins-5hrsv" Jun 15 02:28:29.725889 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.725834 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v956g" Jun 15 02:28:29.735064 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.735040 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-p76zd" Jun 15 02:28:29.742675 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.742656 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:29.749332 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.749312 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-tsd5w" Jun 15 02:28:29.754850 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.754832 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9zrhr" Jun 15 02:28:29.757926 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.757910 2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 15 02:28:29.760611 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.760596 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wthd7" Jun 15 02:28:29.766139 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.766122 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lnjtl" Jun 15 02:28:29.771730 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.771710 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5hrsv" Jun 15 02:28:29.776320 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:29.776303 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wpq45" Jun 15 02:28:30.047663 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:30.047570 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0c770be9-5ffa-4da5-bc85-58fd4c82b027-metrics-certs\") pod \"network-metrics-daemon-954mx\" (UID: \"0c770be9-5ffa-4da5-bc85-58fd4c82b027\") " pod="openshift-multus/network-metrics-daemon-954mx" Jun 15 02:28:30.047663 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:30.047656 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l75z9\" (UniqueName: \"kubernetes.io/projected/91285de0-6ca5-4b23-b554-0c06718dc1df-kube-api-access-l75z9\") pod \"network-check-target-5qfdl\" (UID: \"91285de0-6ca5-4b23-b554-0c06718dc1df\") " pod="openshift-network-diagnostics/network-check-target-5qfdl" Jun 15 02:28:30.047848 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:30.047794 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 02:28:30.047848 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:30.047806 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 15 02:28:30.047848 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:30.047820 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 15 02:28:30.047848 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:30.047830 2579 projected.go:196] Error preparing data for projected volume kube-api-access-l75z9 for pod openshift-network-diagnostics/network-check-target-5qfdl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 02:28:30.048000 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:30.047866 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c770be9-5ffa-4da5-bc85-58fd4c82b027-metrics-certs podName:0c770be9-5ffa-4da5-bc85-58fd4c82b027 nodeName:}" failed. No retries permitted until 2026-06-15 02:28:31.047849576 +0000 UTC m=+3.915634300 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0c770be9-5ffa-4da5-bc85-58fd4c82b027-metrics-certs") pod "network-metrics-daemon-954mx" (UID: "0c770be9-5ffa-4da5-bc85-58fd4c82b027") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 02:28:30.048000 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:30.047889 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/91285de0-6ca5-4b23-b554-0c06718dc1df-kube-api-access-l75z9 podName:91285de0-6ca5-4b23-b554-0c06718dc1df nodeName:}" failed. No retries permitted until 2026-06-15 02:28:31.047878157 +0000 UTC m=+3.915662887 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-l75z9" (UniqueName: "kubernetes.io/projected/91285de0-6ca5-4b23-b554-0c06718dc1df-kube-api-access-l75z9") pod "network-check-target-5qfdl" (UID: "91285de0-6ca5-4b23-b554-0c06718dc1df") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 02:28:30.148528 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:30.148502 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d436483f-cbb4-4ce9-b4d5-a21564cf6e97-original-pull-secret\") pod \"global-pull-secret-syncer-qnxs7\" (UID: \"d436483f-cbb4-4ce9-b4d5-a21564cf6e97\") " pod="kube-system/global-pull-secret-syncer-qnxs7" Jun 15 02:28:30.148701 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:30.148627 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 15 02:28:30.148701 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:30.148690 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d436483f-cbb4-4ce9-b4d5-a21564cf6e97-original-pull-secret podName:d436483f-cbb4-4ce9-b4d5-a21564cf6e97 nodeName:}" failed. No retries permitted until 2026-06-15 02:28:31.148677123 +0000 UTC m=+4.016461845 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d436483f-cbb4-4ce9-b4d5-a21564cf6e97-original-pull-secret") pod "global-pull-secret-syncer-qnxs7" (UID: "d436483f-cbb4-4ce9-b4d5-a21564cf6e97") : object "kube-system"/"original-pull-secret" not registered Jun 15 02:28:30.463585 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:30.463502 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-14 02:23:29 +0000 UTC" deadline="2028-01-31 02:28:58.713936342 +0000 UTC" Jun 15 02:28:30.463585 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:30.463530 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14280h0m28.250409389s" Jun 15 02:28:30.552255 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:30.552229 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf96eb18e_3121_4b4e_a6fb_a0c8bd807285.slice/crio-fff917c238bdb5b6e36dc6159608a202f10fb8a01c1b4bdd3d78024ddc123eed WatchSource:0}: Error finding container fff917c238bdb5b6e36dc6159608a202f10fb8a01c1b4bdd3d78024ddc123eed: Status 404 returned error can't find the container with id fff917c238bdb5b6e36dc6159608a202f10fb8a01c1b4bdd3d78024ddc123eed Jun 15 02:28:30.553710 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:30.553649 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95473860_f2a8_4daa_b746_dac03c5462be.slice/crio-1e3c4ac0309f5a4e095000ea3ecef6dfab6de682e724d2b53da06fdf743b246b WatchSource:0}: Error finding container 1e3c4ac0309f5a4e095000ea3ecef6dfab6de682e724d2b53da06fdf743b246b: Status 404 returned error can't find the container with id 1e3c4ac0309f5a4e095000ea3ecef6dfab6de682e724d2b53da06fdf743b246b Jun 15 02:28:30.554924 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:30.554499 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6798a601_4421_44c1_8ef7_3b2527cad953.slice/crio-fabe2864ad79a1dcff9cd68f7cc046bd4f3f873a8a85025b22982533d620515d WatchSource:0}: Error finding container fabe2864ad79a1dcff9cd68f7cc046bd4f3f873a8a85025b22982533d620515d: Status 404 returned error can't find the container with id fabe2864ad79a1dcff9cd68f7cc046bd4f3f873a8a85025b22982533d620515d Jun 15 02:28:30.557499 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:30.557468 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc66bce4_865f_4b30_aca2_93cedd05ffe3.slice/crio-e223bd1598a1a804403ed81ef7462977de97e3ac1915d2ab95c031be08c9bc3c WatchSource:0}: Error finding container e223bd1598a1a804403ed81ef7462977de97e3ac1915d2ab95c031be08c9bc3c: Status 404 returned error can't find the container with id e223bd1598a1a804403ed81ef7462977de97e3ac1915d2ab95c031be08c9bc3c Jun 15 02:28:30.558566 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:30.558541 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54db0e1c_0d25_46d2_a3e3_aebf9339e272.slice/crio-0fa4f8c0a351e0c1a4ceee0c29080ededdb694bd4e866dc397c0d1dff6feeac9 WatchSource:0}: Error finding container 0fa4f8c0a351e0c1a4ceee0c29080ededdb694bd4e866dc397c0d1dff6feeac9: Status 404 returned error can't find the container with id 0fa4f8c0a351e0c1a4ceee0c29080ededdb694bd4e866dc397c0d1dff6feeac9 Jun 15 02:28:30.559757 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:28:30.559574 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea5abfee_bd02_41ce_a0f9_189e5c238bec.slice/crio-47eff10b96cf8832195d7c740cc60e4993fbb71f3fb71e3d7e843af6383b9fef WatchSource:0}: Error finding container 47eff10b96cf8832195d7c740cc60e4993fbb71f3fb71e3d7e843af6383b9fef: Status 404 returned error can't find the container with id 47eff10b96cf8832195d7c740cc60e4993fbb71f3fb71e3d7e843af6383b9fef Jun 15 02:28:31.056728 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:31.055738 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0c770be9-5ffa-4da5-bc85-58fd4c82b027-metrics-certs\") pod \"network-metrics-daemon-954mx\" (UID: \"0c770be9-5ffa-4da5-bc85-58fd4c82b027\") " pod="openshift-multus/network-metrics-daemon-954mx" Jun 15 02:28:31.056728 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:31.055819 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l75z9\" (UniqueName: \"kubernetes.io/projected/91285de0-6ca5-4b23-b554-0c06718dc1df-kube-api-access-l75z9\") pod \"network-check-target-5qfdl\" (UID: \"91285de0-6ca5-4b23-b554-0c06718dc1df\") " pod="openshift-network-diagnostics/network-check-target-5qfdl" Jun 15 02:28:31.056728 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:31.055978 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 15 02:28:31.056728 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:31.055996 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 15 02:28:31.056728 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:31.056010 2579 projected.go:196] Error preparing data for projected volume kube-api-access-l75z9 for pod openshift-network-diagnostics/network-check-target-5qfdl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 02:28:31.056728 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:31.056065 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/91285de0-6ca5-4b23-b554-0c06718dc1df-kube-api-access-l75z9 podName:91285de0-6ca5-4b23-b554-0c06718dc1df nodeName:}" failed. No retries permitted until 2026-06-15 02:28:33.056046236 +0000 UTC m=+5.923830981 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-l75z9" (UniqueName: "kubernetes.io/projected/91285de0-6ca5-4b23-b554-0c06718dc1df-kube-api-access-l75z9") pod "network-check-target-5qfdl" (UID: "91285de0-6ca5-4b23-b554-0c06718dc1df") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 02:28:31.056728 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:31.056435 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 02:28:31.056728 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:31.056486 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c770be9-5ffa-4da5-bc85-58fd4c82b027-metrics-certs podName:0c770be9-5ffa-4da5-bc85-58fd4c82b027 nodeName:}" failed. No retries permitted until 2026-06-15 02:28:33.056468946 +0000 UTC m=+5.924253693 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0c770be9-5ffa-4da5-bc85-58fd4c82b027-metrics-certs") pod "network-metrics-daemon-954mx" (UID: "0c770be9-5ffa-4da5-bc85-58fd4c82b027") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 02:28:31.156285 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:31.156160 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d436483f-cbb4-4ce9-b4d5-a21564cf6e97-original-pull-secret\") pod \"global-pull-secret-syncer-qnxs7\" (UID: \"d436483f-cbb4-4ce9-b4d5-a21564cf6e97\") " pod="kube-system/global-pull-secret-syncer-qnxs7" Jun 15 02:28:31.156454 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:31.156319 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 15 02:28:31.156454 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:31.156388 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d436483f-cbb4-4ce9-b4d5-a21564cf6e97-original-pull-secret podName:d436483f-cbb4-4ce9-b4d5-a21564cf6e97 nodeName:}" failed. No retries permitted until 2026-06-15 02:28:33.156369239 +0000 UTC m=+6.024153975 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d436483f-cbb4-4ce9-b4d5-a21564cf6e97-original-pull-secret") pod "global-pull-secret-syncer-qnxs7" (UID: "d436483f-cbb4-4ce9-b4d5-a21564cf6e97") : object "kube-system"/"original-pull-secret" not registered Jun 15 02:28:31.464897 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:31.464778 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-14 02:23:29 +0000 UTC" deadline="2028-02-29 07:21:56.582082846 +0000 UTC" Jun 15 02:28:31.464897 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:31.464871 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14980h53m25.117246381s" Jun 15 02:28:31.487352 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:31.487323 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5qfdl" Jun 15 02:28:31.487492 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:31.487451 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5qfdl" podUID="91285de0-6ca5-4b23-b554-0c06718dc1df" Jun 15 02:28:31.487884 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:31.487864 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qnxs7" Jun 15 02:28:31.488005 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:31.487986 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qnxs7" podUID="d436483f-cbb4-4ce9-b4d5-a21564cf6e97" Jun 15 02:28:31.488151 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:31.488134 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-954mx" Jun 15 02:28:31.488247 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:31.488226 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-954mx" podUID="0c770be9-5ffa-4da5-bc85-58fd4c82b027" Jun 15 02:28:31.511603 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:31.511380 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wthd7" event={"ID":"f96eb18e-3121-4b4e-a6fb-a0c8bd807285","Type":"ContainerStarted","Data":"fff917c238bdb5b6e36dc6159608a202f10fb8a01c1b4bdd3d78024ddc123eed"} Jun 15 02:28:31.530651 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:31.530379 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-51.ec2.internal" event={"ID":"66c462553606b0900fb46b9deb100df6","Type":"ContainerStarted","Data":"d806809acef4b9021a4f27509b116a35d9f2f88d5d23fcc99ae5655438addf9c"} Jun 15 02:28:31.535882 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:31.535787 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" event={"ID":"0b0dd3a3-9aef-4f34-8499-c293d4d81a93","Type":"ContainerStarted","Data":"5a8817e8d0884f0dc04cd4a15314f0119f3261871f88c263e88aab7a71dfd61d"} Jun 15 02:28:31.544761 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:31.544694 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lnjtl" event={"ID":"e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4","Type":"ContainerStarted","Data":"42b798f47331a7e656088948693431029347b3cb216db57032821bdc97287338"} Jun 15 02:28:31.552120 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:31.552099 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v956g" event={"ID":"ea5abfee-bd02-41ce-a0f9-189e5c238bec","Type":"ContainerStarted","Data":"47eff10b96cf8832195d7c740cc60e4993fbb71f3fb71e3d7e843af6383b9fef"} Jun 15 02:28:31.556937 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:31.556907 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-tsd5w" event={"ID":"fc66bce4-865f-4b30-aca2-93cedd05ffe3","Type":"ContainerStarted","Data":"e223bd1598a1a804403ed81ef7462977de97e3ac1915d2ab95c031be08c9bc3c"} Jun 15 02:28:31.562365 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:31.562301 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9zrhr" event={"ID":"6798a601-4421-44c1-8ef7-3b2527cad953","Type":"ContainerStarted","Data":"fabe2864ad79a1dcff9cd68f7cc046bd4f3f873a8a85025b22982533d620515d"} Jun 15 02:28:31.566893 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:31.566866 2579 generic.go:356] "Generic (PLEG): container finished" podID="f8074a2ff640a6c74fbf4752467e2525" containerID="872e6c14e0901a6f85d06fcb76e1a6bc6e33fdee693a9398c7e5ddbf6dce424f" exitCode=0 Jun 15 02:28:31.566984 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:31.566947 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-51.ec2.internal" event={"ID":"f8074a2ff640a6c74fbf4752467e2525","Type":"ContainerDied","Data":"872e6c14e0901a6f85d06fcb76e1a6bc6e33fdee693a9398c7e5ddbf6dce424f"} Jun 15 02:28:31.572560 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:31.572537 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-p76zd" event={"ID":"f2a4bcb6-c233-4ccc-9fc1-88c268ae2fb6","Type":"ContainerStarted","Data":"55e301719409f4b7bcdda4493e5b862587173cc73713f8622109e91fdb022e07"} Jun 15 02:28:31.583214 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:31.583167 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-51.ec2.internal" podStartSLOduration=3.583143383 podStartE2EDuration="3.583143383s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 02:28:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 02:28:31.547609772 +0000 UTC m=+4.415394519" watchObservedRunningTime="2026-06-15 02:28:31.583143383 +0000 UTC m=+4.450928130" Jun 15 02:28:31.586972 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:31.586945 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5hrsv" event={"ID":"54db0e1c-0d25-46d2-a3e3-aebf9339e272","Type":"ContainerStarted","Data":"0fa4f8c0a351e0c1a4ceee0c29080ededdb694bd4e866dc397c0d1dff6feeac9"} Jun 15 02:28:31.594578 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:31.594558 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wpq45" event={"ID":"95473860-f2a8-4daa-b746-dac03c5462be","Type":"ContainerStarted","Data":"1e3c4ac0309f5a4e095000ea3ecef6dfab6de682e724d2b53da06fdf743b246b"} Jun 15 02:28:32.608214 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:32.608178 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-51.ec2.internal" event={"ID":"f8074a2ff640a6c74fbf4752467e2525","Type":"ContainerStarted","Data":"01264c9f0327722d595d495a0aaf6c804187afa53e608b88ebca12a6dfaff3ce"} Jun 15 02:28:33.072324 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:33.072244 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0c770be9-5ffa-4da5-bc85-58fd4c82b027-metrics-certs\") pod \"network-metrics-daemon-954mx\" (UID: \"0c770be9-5ffa-4da5-bc85-58fd4c82b027\") " pod="openshift-multus/network-metrics-daemon-954mx" Jun 15 02:28:33.072476 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:33.072328 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l75z9\" (UniqueName: \"kubernetes.io/projected/91285de0-6ca5-4b23-b554-0c06718dc1df-kube-api-access-l75z9\") pod \"network-check-target-5qfdl\" (UID: \"91285de0-6ca5-4b23-b554-0c06718dc1df\") " pod="openshift-network-diagnostics/network-check-target-5qfdl" Jun 15 02:28:33.072536 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:33.072495 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 15 02:28:33.072536 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:33.072515 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 15 02:28:33.072536 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:33.072527 2579 projected.go:196] Error preparing data for projected volume kube-api-access-l75z9 for pod openshift-network-diagnostics/network-check-target-5qfdl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 02:28:33.072696 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:33.072588 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/91285de0-6ca5-4b23-b554-0c06718dc1df-kube-api-access-l75z9 podName:91285de0-6ca5-4b23-b554-0c06718dc1df nodeName:}" failed. No retries permitted until 2026-06-15 02:28:37.07256698 +0000 UTC m=+9.940351705 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-l75z9" (UniqueName: "kubernetes.io/projected/91285de0-6ca5-4b23-b554-0c06718dc1df-kube-api-access-l75z9") pod "network-check-target-5qfdl" (UID: "91285de0-6ca5-4b23-b554-0c06718dc1df") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 02:28:33.072696 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:33.072662 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 02:28:33.072800 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:33.072728 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c770be9-5ffa-4da5-bc85-58fd4c82b027-metrics-certs podName:0c770be9-5ffa-4da5-bc85-58fd4c82b027 nodeName:}" failed. No retries permitted until 2026-06-15 02:28:37.072710664 +0000 UTC m=+9.940495397 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0c770be9-5ffa-4da5-bc85-58fd4c82b027-metrics-certs") pod "network-metrics-daemon-954mx" (UID: "0c770be9-5ffa-4da5-bc85-58fd4c82b027") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 02:28:33.173502 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:33.173460 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d436483f-cbb4-4ce9-b4d5-a21564cf6e97-original-pull-secret\") pod \"global-pull-secret-syncer-qnxs7\" (UID: \"d436483f-cbb4-4ce9-b4d5-a21564cf6e97\") " pod="kube-system/global-pull-secret-syncer-qnxs7" Jun 15 02:28:33.173696 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:33.173610 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 15 02:28:33.173696 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:33.173677 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d436483f-cbb4-4ce9-b4d5-a21564cf6e97-original-pull-secret podName:d436483f-cbb4-4ce9-b4d5-a21564cf6e97 nodeName:}" failed. No retries permitted until 2026-06-15 02:28:37.173659643 +0000 UTC m=+10.041444381 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d436483f-cbb4-4ce9-b4d5-a21564cf6e97-original-pull-secret") pod "global-pull-secret-syncer-qnxs7" (UID: "d436483f-cbb4-4ce9-b4d5-a21564cf6e97") : object "kube-system"/"original-pull-secret" not registered Jun 15 02:28:33.494708 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:33.494617 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5qfdl" Jun 15 02:28:33.494861 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:33.494803 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5qfdl" podUID="91285de0-6ca5-4b23-b554-0c06718dc1df" Jun 15 02:28:33.494923 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:33.494906 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qnxs7" Jun 15 02:28:33.495068 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:33.495048 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qnxs7" podUID="d436483f-cbb4-4ce9-b4d5-a21564cf6e97" Jun 15 02:28:33.495590 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:33.495570 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-954mx" Jun 15 02:28:33.495771 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:33.495749 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-954mx" podUID="0c770be9-5ffa-4da5-bc85-58fd4c82b027" Jun 15 02:28:35.486901 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:35.486867 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-954mx" Jun 15 02:28:35.487331 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:35.487004 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-954mx" podUID="0c770be9-5ffa-4da5-bc85-58fd4c82b027" Jun 15 02:28:35.487331 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:35.487172 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5qfdl" Jun 15 02:28:35.487331 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:35.487205 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qnxs7" Jun 15 02:28:35.487331 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:35.487280 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5qfdl" podUID="91285de0-6ca5-4b23-b554-0c06718dc1df" Jun 15 02:28:35.487574 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:35.487362 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qnxs7" podUID="d436483f-cbb4-4ce9-b4d5-a21564cf6e97" Jun 15 02:28:37.103333 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:37.103299 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l75z9\" (UniqueName: \"kubernetes.io/projected/91285de0-6ca5-4b23-b554-0c06718dc1df-kube-api-access-l75z9\") pod \"network-check-target-5qfdl\" (UID: \"91285de0-6ca5-4b23-b554-0c06718dc1df\") " pod="openshift-network-diagnostics/network-check-target-5qfdl" Jun 15 02:28:37.103804 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:37.103377 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0c770be9-5ffa-4da5-bc85-58fd4c82b027-metrics-certs\") pod \"network-metrics-daemon-954mx\" (UID: \"0c770be9-5ffa-4da5-bc85-58fd4c82b027\") " pod="openshift-multus/network-metrics-daemon-954mx" Jun 15 02:28:37.103804 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:37.103480 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 15 02:28:37.103804 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:37.103490 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 02:28:37.103804 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:37.103504 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 15 02:28:37.103804 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:37.103519 2579 projected.go:196] Error preparing data for projected volume kube-api-access-l75z9 for pod openshift-network-diagnostics/network-check-target-5qfdl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 02:28:37.103804 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:37.103545 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c770be9-5ffa-4da5-bc85-58fd4c82b027-metrics-certs podName:0c770be9-5ffa-4da5-bc85-58fd4c82b027 nodeName:}" failed. No retries permitted until 2026-06-15 02:28:45.103527035 +0000 UTC m=+17.971311760 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0c770be9-5ffa-4da5-bc85-58fd4c82b027-metrics-certs") pod "network-metrics-daemon-954mx" (UID: "0c770be9-5ffa-4da5-bc85-58fd4c82b027") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 02:28:37.103804 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:37.103574 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/91285de0-6ca5-4b23-b554-0c06718dc1df-kube-api-access-l75z9 podName:91285de0-6ca5-4b23-b554-0c06718dc1df nodeName:}" failed. No retries permitted until 2026-06-15 02:28:45.103555619 +0000 UTC m=+17.971340365 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-l75z9" (UniqueName: "kubernetes.io/projected/91285de0-6ca5-4b23-b554-0c06718dc1df-kube-api-access-l75z9") pod "network-check-target-5qfdl" (UID: "91285de0-6ca5-4b23-b554-0c06718dc1df") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 02:28:37.203977 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:37.203943 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d436483f-cbb4-4ce9-b4d5-a21564cf6e97-original-pull-secret\") pod \"global-pull-secret-syncer-qnxs7\" (UID: \"d436483f-cbb4-4ce9-b4d5-a21564cf6e97\") " pod="kube-system/global-pull-secret-syncer-qnxs7" Jun 15 02:28:37.204165 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:37.204108 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 15 02:28:37.204238 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:37.204167 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d436483f-cbb4-4ce9-b4d5-a21564cf6e97-original-pull-secret podName:d436483f-cbb4-4ce9-b4d5-a21564cf6e97 nodeName:}" failed. No retries permitted until 2026-06-15 02:28:45.204147686 +0000 UTC m=+18.071932410 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d436483f-cbb4-4ce9-b4d5-a21564cf6e97-original-pull-secret") pod "global-pull-secret-syncer-qnxs7" (UID: "d436483f-cbb4-4ce9-b4d5-a21564cf6e97") : object "kube-system"/"original-pull-secret" not registered Jun 15 02:28:37.487971 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:37.487896 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-954mx" Jun 15 02:28:37.488131 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:37.488018 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-954mx" podUID="0c770be9-5ffa-4da5-bc85-58fd4c82b027" Jun 15 02:28:37.488131 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:37.488060 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5qfdl" Jun 15 02:28:37.488242 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:37.488156 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5qfdl" podUID="91285de0-6ca5-4b23-b554-0c06718dc1df" Jun 15 02:28:37.488242 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:37.488219 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qnxs7" Jun 15 02:28:37.488367 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:37.488340 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qnxs7" podUID="d436483f-cbb4-4ce9-b4d5-a21564cf6e97" Jun 15 02:28:39.487908 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:39.487085 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qnxs7" Jun 15 02:28:39.487908 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:39.487212 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qnxs7" podUID="d436483f-cbb4-4ce9-b4d5-a21564cf6e97" Jun 15 02:28:39.487908 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:39.487598 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-954mx" Jun 15 02:28:39.487908 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:39.487722 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-954mx" podUID="0c770be9-5ffa-4da5-bc85-58fd4c82b027" Jun 15 02:28:39.487908 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:39.487797 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5qfdl" Jun 15 02:28:39.487908 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:39.487876 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5qfdl" podUID="91285de0-6ca5-4b23-b554-0c06718dc1df" Jun 15 02:28:41.486771 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:41.486709 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5qfdl" Jun 15 02:28:41.486771 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:41.486726 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qnxs7" Jun 15 02:28:41.487734 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:41.486833 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5qfdl" podUID="91285de0-6ca5-4b23-b554-0c06718dc1df" Jun 15 02:28:41.487734 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:41.486893 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-954mx" Jun 15 02:28:41.487734 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:41.487021 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qnxs7" podUID="d436483f-cbb4-4ce9-b4d5-a21564cf6e97" Jun 15 02:28:41.487734 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:41.487101 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-954mx" podUID="0c770be9-5ffa-4da5-bc85-58fd4c82b027" Jun 15 02:28:41.631268 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:41.631234 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v956g" event={"ID":"ea5abfee-bd02-41ce-a0f9-189e5c238bec","Type":"ContainerStarted","Data":"3425bcc9b9ae8e013f00f1e01b1fd1411901ddd3ce8d1be782bc929256c1eb89"} Jun 15 02:28:41.632741 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:41.632706 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-tsd5w" event={"ID":"fc66bce4-865f-4b30-aca2-93cedd05ffe3","Type":"ContainerStarted","Data":"62b6cecce3cfd6e9c89b1df83c10aaa0cf75f61a48374ff602d6e782a4086e69"} Jun 15 02:28:41.634231 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:41.634208 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9zrhr" event={"ID":"6798a601-4421-44c1-8ef7-3b2527cad953","Type":"ContainerStarted","Data":"cd59c5f12d50dff8eeeae91932d8b1d7370af9ae3809aee9e545e6acf0ee42c1"} Jun 15 02:28:41.635777 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:41.635751 2579 generic.go:356] "Generic (PLEG): container finished" podID="54db0e1c-0d25-46d2-a3e3-aebf9339e272" containerID="e8321176d0ae10870c74cd36b98529b5d9f08f0a7d4f598f3e4b3fb17ed76fc5" exitCode=0 Jun 15 02:28:41.635888 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:41.635827 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5hrsv" event={"ID":"54db0e1c-0d25-46d2-a3e3-aebf9339e272","Type":"ContainerDied","Data":"e8321176d0ae10870c74cd36b98529b5d9f08f0a7d4f598f3e4b3fb17ed76fc5"} Jun 15 02:28:41.637355 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:41.637332 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wpq45" event={"ID":"95473860-f2a8-4daa-b746-dac03c5462be","Type":"ContainerStarted","Data":"4321e80acb90ea06c4f29fa7f333180c8375b58737deb7f80176b691d590c549"} Jun 15 02:28:41.638925 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:41.638902 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wthd7" event={"ID":"f96eb18e-3121-4b4e-a6fb-a0c8bd807285","Type":"ContainerStarted","Data":"d9a51141819bd9c2cd052b7a8d92c80f6beffae8b84ec258dd7a21ce27280a34"} Jun 15 02:28:41.646910 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:41.646864 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-tsd5w" podStartSLOduration=4.677977486 podStartE2EDuration="14.646846723s" totalImagesPullingTime="9.968869237s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 02:28:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 02:28:41.646558423 +0000 UTC m=+14.514343169" watchObservedRunningTime="2026-06-15 02:28:41.646846723 +0000 UTC m=+14.514631479" Jun 15 02:28:41.647187 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:41.647155 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-51.ec2.internal" podStartSLOduration=13.647145768 podStartE2EDuration="13.647145768s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 02:28:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 02:28:32.622500197 +0000 UTC m=+5.490284943" watchObservedRunningTime="2026-06-15 02:28:41.647145768 +0000 UTC m=+14.514930514" Jun 15 02:28:41.659098 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:41.659052 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-9zrhr" podStartSLOduration=4.687073745 podStartE2EDuration="14.65904151s" totalImagesPullingTime="9.971967765s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 02:28:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 02:28:41.658415866 +0000 UTC m=+14.526200612" watchObservedRunningTime="2026-06-15 02:28:41.65904151 +0000 UTC m=+14.526826256" Jun 15 02:28:41.670925 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:41.670863 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-wthd7" podStartSLOduration=4.697369561 podStartE2EDuration="14.670847329s" totalImagesPullingTime="9.973477768s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 02:28:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 02:28:41.670813591 +0000 UTC m=+14.538598357" watchObservedRunningTime="2026-06-15 02:28:41.670847329 +0000 UTC m=+14.538632075" Jun 15 02:28:41.708645 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:41.708589 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-wpq45" podStartSLOduration=3.7308212530000002 podStartE2EDuration="13.708571885s" totalImagesPullingTime="9.977750632s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 02:28:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 02:28:41.708178742 +0000 UTC m=+14.575963487" watchObservedRunningTime="2026-06-15 02:28:41.708571885 +0000 UTC m=+14.576356632" Jun 15 02:28:42.642623 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:42.642559 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-p76zd" event={"ID":"f2a4bcb6-c233-4ccc-9fc1-88c268ae2fb6","Type":"ContainerStarted","Data":"b30789ec92a5f910358eac473b94a6aa700aea7e1baa484268b385cc1915ae7e"} Jun 15 02:28:42.656995 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:42.656951 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-p76zd" podStartSLOduration=5.686864537 podStartE2EDuration="15.656937404s" totalImagesPullingTime="9.970072867s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 02:28:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 02:28:42.656772546 +0000 UTC m=+15.524557293" watchObservedRunningTime="2026-06-15 02:28:42.656937404 +0000 UTC m=+15.524722149" Jun 15 02:28:43.487649 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:43.487418 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-954mx" Jun 15 02:28:43.487816 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:43.487418 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qnxs7" Jun 15 02:28:43.487816 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:43.487752 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-954mx" podUID="0c770be9-5ffa-4da5-bc85-58fd4c82b027" Jun 15 02:28:43.487816 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:43.487479 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5qfdl" Jun 15 02:28:43.487993 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:43.487829 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qnxs7" podUID="d436483f-cbb4-4ce9-b4d5-a21564cf6e97" Jun 15 02:28:43.487993 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:43.487892 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5qfdl" podUID="91285de0-6ca5-4b23-b554-0c06718dc1df" Jun 15 02:28:45.163246 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:45.163216 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0c770be9-5ffa-4da5-bc85-58fd4c82b027-metrics-certs\") pod \"network-metrics-daemon-954mx\" (UID: \"0c770be9-5ffa-4da5-bc85-58fd4c82b027\") " pod="openshift-multus/network-metrics-daemon-954mx" Jun 15 02:28:45.163627 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:45.163267 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l75z9\" (UniqueName: \"kubernetes.io/projected/91285de0-6ca5-4b23-b554-0c06718dc1df-kube-api-access-l75z9\") pod \"network-check-target-5qfdl\" (UID: \"91285de0-6ca5-4b23-b554-0c06718dc1df\") " pod="openshift-network-diagnostics/network-check-target-5qfdl" Jun 15 02:28:45.163627 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:45.163375 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 02:28:45.163627 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:45.163384 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 15 02:28:45.163627 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:45.163399 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 15 02:28:45.163627 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:45.163412 2579 projected.go:196] Error preparing data for projected volume kube-api-access-l75z9 for pod openshift-network-diagnostics/network-check-target-5qfdl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 02:28:45.163627 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:45.163455 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/91285de0-6ca5-4b23-b554-0c06718dc1df-kube-api-access-l75z9 podName:91285de0-6ca5-4b23-b554-0c06718dc1df nodeName:}" failed. No retries permitted until 2026-06-15 02:29:01.163438307 +0000 UTC m=+34.031223030 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-l75z9" (UniqueName: "kubernetes.io/projected/91285de0-6ca5-4b23-b554-0c06718dc1df-kube-api-access-l75z9") pod "network-check-target-5qfdl" (UID: "91285de0-6ca5-4b23-b554-0c06718dc1df") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 02:28:45.163627 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:45.163473 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c770be9-5ffa-4da5-bc85-58fd4c82b027-metrics-certs podName:0c770be9-5ffa-4da5-bc85-58fd4c82b027 nodeName:}" failed. No retries permitted until 2026-06-15 02:29:01.163463917 +0000 UTC m=+34.031248640 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0c770be9-5ffa-4da5-bc85-58fd4c82b027-metrics-certs") pod "network-metrics-daemon-954mx" (UID: "0c770be9-5ffa-4da5-bc85-58fd4c82b027") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 02:28:45.264627 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:45.264583 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d436483f-cbb4-4ce9-b4d5-a21564cf6e97-original-pull-secret\") pod \"global-pull-secret-syncer-qnxs7\" (UID: \"d436483f-cbb4-4ce9-b4d5-a21564cf6e97\") " pod="kube-system/global-pull-secret-syncer-qnxs7" Jun 15 02:28:45.264785 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:45.264762 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 15 02:28:45.264854 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:45.264843 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d436483f-cbb4-4ce9-b4d5-a21564cf6e97-original-pull-secret podName:d436483f-cbb4-4ce9-b4d5-a21564cf6e97 nodeName:}" failed. No retries permitted until 2026-06-15 02:29:01.264822741 +0000 UTC m=+34.132607479 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d436483f-cbb4-4ce9-b4d5-a21564cf6e97-original-pull-secret") pod "global-pull-secret-syncer-qnxs7" (UID: "d436483f-cbb4-4ce9-b4d5-a21564cf6e97") : object "kube-system"/"original-pull-secret" not registered Jun 15 02:28:45.486714 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:45.486676 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qnxs7" Jun 15 02:28:45.486883 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:45.486680 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-954mx" Jun 15 02:28:45.486883 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:45.486802 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qnxs7" podUID="d436483f-cbb4-4ce9-b4d5-a21564cf6e97" Jun 15 02:28:45.486883 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:45.486829 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5qfdl" Jun 15 02:28:45.487013 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:45.486934 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-954mx" podUID="0c770be9-5ffa-4da5-bc85-58fd4c82b027" Jun 15 02:28:45.487046 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:45.487009 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5qfdl" podUID="91285de0-6ca5-4b23-b554-0c06718dc1df" Jun 15 02:28:46.206123 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:46.206050 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-tsd5w" Jun 15 02:28:46.207008 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:46.206984 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-tsd5w" Jun 15 02:28:47.487518 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:47.487476 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5qfdl" Jun 15 02:28:47.487998 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:47.487557 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5qfdl" podUID="91285de0-6ca5-4b23-b554-0c06718dc1df" Jun 15 02:28:47.487998 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:47.487687 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qnxs7" Jun 15 02:28:47.487998 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:47.487810 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qnxs7" podUID="d436483f-cbb4-4ce9-b4d5-a21564cf6e97" Jun 15 02:28:47.487998 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:47.487829 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-954mx" Jun 15 02:28:47.487998 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:47.487959 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-954mx" podUID="0c770be9-5ffa-4da5-bc85-58fd4c82b027" Jun 15 02:28:49.487285 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:49.487252 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qnxs7" Jun 15 02:28:49.487770 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:49.487292 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-954mx" Jun 15 02:28:49.487770 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:49.487259 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5qfdl" Jun 15 02:28:49.487770 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:49.487380 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qnxs7" podUID="d436483f-cbb4-4ce9-b4d5-a21564cf6e97" Jun 15 02:28:49.487770 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:49.487495 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-954mx" podUID="0c770be9-5ffa-4da5-bc85-58fd4c82b027" Jun 15 02:28:49.487770 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:49.487581 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5qfdl" podUID="91285de0-6ca5-4b23-b554-0c06718dc1df" Jun 15 02:28:51.487484 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:51.487453 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qnxs7" Jun 15 02:28:51.488070 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:51.487450 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-954mx" Jun 15 02:28:51.488070 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:51.487588 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qnxs7" podUID="d436483f-cbb4-4ce9-b4d5-a21564cf6e97" Jun 15 02:28:51.488070 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:51.487452 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5qfdl" Jun 15 02:28:51.488070 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:51.487699 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-954mx" podUID="0c770be9-5ffa-4da5-bc85-58fd4c82b027" Jun 15 02:28:51.488070 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:51.487749 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5qfdl" podUID="91285de0-6ca5-4b23-b554-0c06718dc1df" Jun 15 02:28:51.490662 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:51.490626 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-tsd5w" Jun 15 02:28:51.490785 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:51.490709 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-tsd5w" Jun 15 02:28:51.850535 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:51.850511 2579 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 15 02:28:52.499396 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:52.499128 2579 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-15T02:28:51.850528782Z","UUID":"1ed9c069-6c93-4750-8a8c-d1cf620f3a99","Handler":null,"Name":"","Endpoint":""} Jun 15 02:28:52.500608 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:52.500594 2579 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 15 02:28:52.500706 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:52.500625 2579 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 15 02:28:52.662708 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:52.662678 2579 generic.go:356] "Generic (PLEG): container finished" podID="54db0e1c-0d25-46d2-a3e3-aebf9339e272" containerID="4f458f90ef84034262e7a0765b0b05cf361e5d1465ad7102184c98dbe3386a0e" exitCode=0 Jun 15 02:28:52.662790 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:52.662758 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5hrsv" event={"ID":"54db0e1c-0d25-46d2-a3e3-aebf9339e272","Type":"ContainerDied","Data":"4f458f90ef84034262e7a0765b0b05cf361e5d1465ad7102184c98dbe3386a0e"} Jun 15 02:28:52.667388 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:52.667353 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" event={"ID":"0b0dd3a3-9aef-4f34-8499-c293d4d81a93","Type":"ContainerStarted","Data":"d69edf4afa752013948818a3fd5ca446b5312d29eb02033de48d1a49f97eb827"} Jun 15 02:28:52.667501 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:52.667442 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" event={"ID":"0b0dd3a3-9aef-4f34-8499-c293d4d81a93","Type":"ContainerStarted","Data":"4f19c6f6909a537766884589c2da636b7c478bd5e03ba33006c382d37732689d"} Jun 15 02:28:52.667501 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:52.667497 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" event={"ID":"0b0dd3a3-9aef-4f34-8499-c293d4d81a93","Type":"ContainerStarted","Data":"5a6e944cf7462279972f345ef8ef3dadbd3caba38ba15632fb3bd07aaea2ef4f"} Jun 15 02:28:52.667611 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:52.667546 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" event={"ID":"0b0dd3a3-9aef-4f34-8499-c293d4d81a93","Type":"ContainerStarted","Data":"f5663949716a8f73c911471dcac125fe7b7ec1dc27ca04236ab50def1846a90b"} Jun 15 02:28:52.667952 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:52.667924 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" event={"ID":"0b0dd3a3-9aef-4f34-8499-c293d4d81a93","Type":"ContainerStarted","Data":"b9f9953cc72373f079cea0fb710a65f9e520752e7c1d10f11f1fba1bcb977ee6"} Jun 15 02:28:52.668088 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:52.668072 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" event={"ID":"0b0dd3a3-9aef-4f34-8499-c293d4d81a93","Type":"ContainerStarted","Data":"c07e6e9959db6e05295ac473a9722512d87a41aa1369f3ea2106fcd5d04b04b6"} Jun 15 02:28:52.672166 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:52.672141 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lnjtl" event={"ID":"e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4","Type":"ContainerStarted","Data":"2898f1de554c7719469fbc9d0feb475a37d33b6d2c1f00866eff2ff8e905db1c"} Jun 15 02:28:52.674095 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:52.674061 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v956g" event={"ID":"ea5abfee-bd02-41ce-a0f9-189e5c238bec","Type":"ContainerStarted","Data":"bf0f37f893d66b4fd4f1e7d7b3ec2b6929bbeb9409d159adfee7ddc7df6ac9f5"} Jun 15 02:28:52.697936 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:52.697891 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-lnjtl" podStartSLOduration=4.56413418 podStartE2EDuration="25.697876728s" totalImagesPullingTime="21.133742548s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 02:28:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 02:28:52.697834237 +0000 UTC m=+25.565618982" watchObservedRunningTime="2026-06-15 02:28:52.697876728 +0000 UTC m=+25.565661472" Jun 15 02:28:53.486946 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:53.486755 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qnxs7" Jun 15 02:28:53.487050 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:53.486758 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-954mx" Jun 15 02:28:53.487106 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:53.486774 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5qfdl" Jun 15 02:28:53.487145 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:53.487097 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-954mx" podUID="0c770be9-5ffa-4da5-bc85-58fd4c82b027" Jun 15 02:28:53.487185 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:53.487170 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5qfdl" podUID="91285de0-6ca5-4b23-b554-0c06718dc1df" Jun 15 02:28:53.487228 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:53.486981 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qnxs7" podUID="d436483f-cbb4-4ce9-b4d5-a21564cf6e97" Jun 15 02:28:53.678535 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:53.678506 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v956g" event={"ID":"ea5abfee-bd02-41ce-a0f9-189e5c238bec","Type":"ContainerStarted","Data":"5421f68e1901e703f545a2153ef5bb4e999d516b379b46f89cc6110deb92ac16"} Jun 15 02:28:53.696434 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:53.696392 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v956g" podStartSLOduration=4.544313288 podStartE2EDuration="26.696379783s" totalImagesPullingTime="22.152066495s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 02:28:27 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 02:28:53.695993596 +0000 UTC m=+26.563778340" watchObservedRunningTime="2026-06-15 02:28:53.696379783 +0000 UTC m=+26.564164528" Jun 15 02:28:54.682675 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:54.682595 2579 generic.go:356] "Generic (PLEG): container finished" podID="54db0e1c-0d25-46d2-a3e3-aebf9339e272" containerID="87df9e7167d40cadcacb71a4c40867e7d0ffbb0019698b5afb13b13e5f197e78" exitCode=0 Jun 15 02:28:54.682959 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:54.682673 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5hrsv" event={"ID":"54db0e1c-0d25-46d2-a3e3-aebf9339e272","Type":"ContainerDied","Data":"87df9e7167d40cadcacb71a4c40867e7d0ffbb0019698b5afb13b13e5f197e78"} Jun 15 02:28:55.487017 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:55.486996 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-954mx" Jun 15 02:28:55.487117 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:55.487004 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qnxs7" Jun 15 02:28:55.487174 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:55.487119 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-954mx" podUID="0c770be9-5ffa-4da5-bc85-58fd4c82b027" Jun 15 02:28:55.487228 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:55.487004 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5qfdl" Jun 15 02:28:55.487281 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:55.487230 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qnxs7" podUID="d436483f-cbb4-4ce9-b4d5-a21564cf6e97" Jun 15 02:28:55.487375 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:55.487354 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5qfdl" podUID="91285de0-6ca5-4b23-b554-0c06718dc1df" Jun 15 02:28:55.687987 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:55.687937 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" event={"ID":"0b0dd3a3-9aef-4f34-8499-c293d4d81a93","Type":"ContainerStarted","Data":"d642a2901f2c567f592ab414ebaec26b706d69a6a545ca01040ce245b113ac0a"} Jun 15 02:28:55.690107 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:55.690086 2579 generic.go:356] "Generic (PLEG): container finished" podID="54db0e1c-0d25-46d2-a3e3-aebf9339e272" containerID="e988fdbbb366a351cf238be6ab92fb0e3e86de0058505b342837d02b7f1aa2dc" exitCode=0 Jun 15 02:28:55.690202 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:55.690115 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5hrsv" event={"ID":"54db0e1c-0d25-46d2-a3e3-aebf9339e272","Type":"ContainerDied","Data":"e988fdbbb366a351cf238be6ab92fb0e3e86de0058505b342837d02b7f1aa2dc"} Jun 15 02:28:57.487608 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:57.487582 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-954mx" Jun 15 02:28:57.488069 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:57.487712 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-954mx" podUID="0c770be9-5ffa-4da5-bc85-58fd4c82b027" Jun 15 02:28:57.488069 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:57.487773 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qnxs7" Jun 15 02:28:57.488069 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:57.487863 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5qfdl" Jun 15 02:28:57.488069 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:57.487884 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qnxs7" podUID="d436483f-cbb4-4ce9-b4d5-a21564cf6e97" Jun 15 02:28:57.488069 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:57.487927 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5qfdl" podUID="91285de0-6ca5-4b23-b554-0c06718dc1df" Jun 15 02:28:57.698447 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:57.698409 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" event={"ID":"0b0dd3a3-9aef-4f34-8499-c293d4d81a93","Type":"ContainerStarted","Data":"e395efade83ac8e440639c10dc82d14c9fde544233656896836703c6fd189133"} Jun 15 02:28:57.698660 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:57.698624 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:57.698745 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:57.698716 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:57.719564 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:57.719541 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:57.722192 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:57.722146 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" podStartSLOduration=9.277414113 podStartE2EDuration="30.722133519s" totalImagesPullingTime="21.444719406s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 02:28:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 02:28:57.721084969 +0000 UTC m=+30.588869711" watchObservedRunningTime="2026-06-15 02:28:57.722133519 +0000 UTC m=+30.589918248" Jun 15 02:28:58.701845 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:58.701602 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:58.719289 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:58.719263 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:28:58.764689 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:58.764661 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5qfdl"] Jun 15 02:28:58.764816 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:58.764777 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5qfdl" Jun 15 02:28:58.764895 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:58.764879 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5qfdl" podUID="91285de0-6ca5-4b23-b554-0c06718dc1df" Jun 15 02:28:58.765254 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:58.765237 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-954mx"] Jun 15 02:28:58.765342 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:58.765332 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-954mx" Jun 15 02:28:58.765422 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:58.765408 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-954mx" podUID="0c770be9-5ffa-4da5-bc85-58fd4c82b027" Jun 15 02:28:58.774442 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:58.774420 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qnxs7"] Jun 15 02:28:58.774566 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:28:58.774525 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qnxs7" Jun 15 02:28:58.774655 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:28:58.774601 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qnxs7" podUID="d436483f-cbb4-4ce9-b4d5-a21564cf6e97" Jun 15 02:29:00.487085 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:00.487051 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-954mx" Jun 15 02:29:00.487085 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:00.487077 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qnxs7" Jun 15 02:29:00.487580 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:00.487067 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5qfdl" Jun 15 02:29:00.487580 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:29:00.487191 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-954mx" podUID="0c770be9-5ffa-4da5-bc85-58fd4c82b027" Jun 15 02:29:00.487580 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:29:00.487280 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5qfdl" podUID="91285de0-6ca5-4b23-b554-0c06718dc1df" Jun 15 02:29:00.487580 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:29:00.487356 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qnxs7" podUID="d436483f-cbb4-4ce9-b4d5-a21564cf6e97" Jun 15 02:29:01.187716 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:01.187628 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0c770be9-5ffa-4da5-bc85-58fd4c82b027-metrics-certs\") pod \"network-metrics-daemon-954mx\" (UID: \"0c770be9-5ffa-4da5-bc85-58fd4c82b027\") " pod="openshift-multus/network-metrics-daemon-954mx" Jun 15 02:29:01.187716 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:01.187716 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l75z9\" (UniqueName: \"kubernetes.io/projected/91285de0-6ca5-4b23-b554-0c06718dc1df-kube-api-access-l75z9\") pod \"network-check-target-5qfdl\" (UID: \"91285de0-6ca5-4b23-b554-0c06718dc1df\") " pod="openshift-network-diagnostics/network-check-target-5qfdl" Jun 15 02:29:01.187901 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:29:01.187779 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 02:29:01.187901 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:29:01.187834 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 15 02:29:01.187901 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:29:01.187839 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c770be9-5ffa-4da5-bc85-58fd4c82b027-metrics-certs podName:0c770be9-5ffa-4da5-bc85-58fd4c82b027 nodeName:}" failed. No retries permitted until 2026-06-15 02:29:33.187820053 +0000 UTC m=+66.055604791 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0c770be9-5ffa-4da5-bc85-58fd4c82b027-metrics-certs") pod "network-metrics-daemon-954mx" (UID: "0c770be9-5ffa-4da5-bc85-58fd4c82b027") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 02:29:01.187901 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:29:01.187848 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 15 02:29:01.187901 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:29:01.187860 2579 projected.go:196] Error preparing data for projected volume kube-api-access-l75z9 for pod openshift-network-diagnostics/network-check-target-5qfdl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 02:29:01.187901 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:29:01.187894 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/91285de0-6ca5-4b23-b554-0c06718dc1df-kube-api-access-l75z9 podName:91285de0-6ca5-4b23-b554-0c06718dc1df nodeName:}" failed. No retries permitted until 2026-06-15 02:29:33.187883815 +0000 UTC m=+66.055668555 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-l75z9" (UniqueName: "kubernetes.io/projected/91285de0-6ca5-4b23-b554-0c06718dc1df-kube-api-access-l75z9") pod "network-check-target-5qfdl" (UID: "91285de0-6ca5-4b23-b554-0c06718dc1df") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 02:29:01.288878 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:01.288848 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d436483f-cbb4-4ce9-b4d5-a21564cf6e97-original-pull-secret\") pod \"global-pull-secret-syncer-qnxs7\" (UID: \"d436483f-cbb4-4ce9-b4d5-a21564cf6e97\") " pod="kube-system/global-pull-secret-syncer-qnxs7" Jun 15 02:29:01.289030 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:29:01.289001 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 15 02:29:01.289097 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:29:01.289073 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d436483f-cbb4-4ce9-b4d5-a21564cf6e97-original-pull-secret podName:d436483f-cbb4-4ce9-b4d5-a21564cf6e97 nodeName:}" failed. No retries permitted until 2026-06-15 02:29:33.289054852 +0000 UTC m=+66.156839592 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d436483f-cbb4-4ce9-b4d5-a21564cf6e97-original-pull-secret") pod "global-pull-secret-syncer-qnxs7" (UID: "d436483f-cbb4-4ce9-b4d5-a21564cf6e97") : object "kube-system"/"original-pull-secret" not registered Jun 15 02:29:02.487166 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:02.487141 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5qfdl" Jun 15 02:29:02.487166 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:02.487165 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-954mx" Jun 15 02:29:02.487731 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:02.487142 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qnxs7" Jun 15 02:29:02.487731 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:29:02.487236 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5qfdl" podUID="91285de0-6ca5-4b23-b554-0c06718dc1df" Jun 15 02:29:02.487731 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:29:02.487313 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qnxs7" podUID="d436483f-cbb4-4ce9-b4d5-a21564cf6e97" Jun 15 02:29:02.487731 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:29:02.487378 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-954mx" podUID="0c770be9-5ffa-4da5-bc85-58fd4c82b027" Jun 15 02:29:02.714683 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:02.714657 2579 generic.go:356] "Generic (PLEG): container finished" podID="54db0e1c-0d25-46d2-a3e3-aebf9339e272" containerID="85d557a1d3edd6539af530279cab9401335dacd22a000dbcb99dfaa61cb0d95e" exitCode=0 Jun 15 02:29:02.714778 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:02.714707 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5hrsv" event={"ID":"54db0e1c-0d25-46d2-a3e3-aebf9339e272","Type":"ContainerDied","Data":"85d557a1d3edd6539af530279cab9401335dacd22a000dbcb99dfaa61cb0d95e"} Jun 15 02:29:03.719564 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:03.719533 2579 generic.go:356] "Generic (PLEG): container finished" podID="54db0e1c-0d25-46d2-a3e3-aebf9339e272" containerID="ffd31665937ce36dd95a6225b2c5f69f4a993cadb1d87879e0fc3f20625c37c8" exitCode=0 Jun 15 02:29:03.720074 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:03.719589 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5hrsv" event={"ID":"54db0e1c-0d25-46d2-a3e3-aebf9339e272","Type":"ContainerDied","Data":"ffd31665937ce36dd95a6225b2c5f69f4a993cadb1d87879e0fc3f20625c37c8"} Jun 15 02:29:04.487349 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:04.487323 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qnxs7" Jun 15 02:29:04.487469 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:04.487323 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5qfdl" Jun 15 02:29:04.487469 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:04.487364 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-954mx" Jun 15 02:29:04.487469 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:29:04.487449 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qnxs7" podUID="d436483f-cbb4-4ce9-b4d5-a21564cf6e97" Jun 15 02:29:04.487591 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:29:04.487556 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5qfdl" podUID="91285de0-6ca5-4b23-b554-0c06718dc1df" Jun 15 02:29:04.487658 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:29:04.487619 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-954mx" podUID="0c770be9-5ffa-4da5-bc85-58fd4c82b027" Jun 15 02:29:04.491958 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:04.491939 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-51.ec2.internal" event="NodeReady" Jun 15 02:29:04.492043 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:04.492030 2579 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jun 15 02:29:04.552797 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:04.552773 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-8wpj7"] Jun 15 02:29:04.576215 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:04.576193 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-hz7mx"] Jun 15 02:29:04.576317 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:04.576302 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8wpj7" Jun 15 02:29:04.578439 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:04.578422 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 15 02:29:04.578439 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:04.578438 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vrtgk\"" Jun 15 02:29:04.578591 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:04.578422 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 15 02:29:04.600253 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:04.600232 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8wpj7"] Jun 15 02:29:04.600333 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:04.600261 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hz7mx"] Jun 15 02:29:04.600333 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:04.600317 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hz7mx" Jun 15 02:29:04.603553 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:04.603027 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 15 02:29:04.603553 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:04.603105 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 15 02:29:04.603750 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:04.603028 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 15 02:29:04.603859 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:04.603840 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-fllmd\"" Jun 15 02:29:04.711333 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:04.711314 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c8c3e57a-ae17-4f87-ba38-acc0e14583c4-cert\") pod \"ingress-canary-hz7mx\" (UID: \"c8c3e57a-ae17-4f87-ba38-acc0e14583c4\") " pod="openshift-ingress-canary/ingress-canary-hz7mx" Jun 15 02:29:04.711433 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:04.711340 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1c70b33b-a4a3-4e1e-9693-41d77a5217fd-config-volume\") pod \"dns-default-8wpj7\" (UID: \"1c70b33b-a4a3-4e1e-9693-41d77a5217fd\") " pod="openshift-dns/dns-default-8wpj7" Jun 15 02:29:04.711433 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:04.711355 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1c70b33b-a4a3-4e1e-9693-41d77a5217fd-tmp-dir\") pod \"dns-default-8wpj7\" (UID: \"1c70b33b-a4a3-4e1e-9693-41d77a5217fd\") " pod="openshift-dns/dns-default-8wpj7" Jun 15 02:29:04.711433 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:04.711378 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qpc6\" (UniqueName: \"kubernetes.io/projected/1c70b33b-a4a3-4e1e-9693-41d77a5217fd-kube-api-access-9qpc6\") pod \"dns-default-8wpj7\" (UID: \"1c70b33b-a4a3-4e1e-9693-41d77a5217fd\") " pod="openshift-dns/dns-default-8wpj7" Jun 15 02:29:04.711541 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:04.711447 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1c70b33b-a4a3-4e1e-9693-41d77a5217fd-metrics-tls\") pod \"dns-default-8wpj7\" (UID: \"1c70b33b-a4a3-4e1e-9693-41d77a5217fd\") " pod="openshift-dns/dns-default-8wpj7" Jun 15 02:29:04.711541 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:04.711479 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dtw8\" (UniqueName: \"kubernetes.io/projected/c8c3e57a-ae17-4f87-ba38-acc0e14583c4-kube-api-access-6dtw8\") pod \"ingress-canary-hz7mx\" (UID: \"c8c3e57a-ae17-4f87-ba38-acc0e14583c4\") " pod="openshift-ingress-canary/ingress-canary-hz7mx" Jun 15 02:29:04.725210 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:04.725184 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5hrsv" event={"ID":"54db0e1c-0d25-46d2-a3e3-aebf9339e272","Type":"ContainerStarted","Data":"3be8e23f44a2829abaa3c6938a906f97462a47f3c06632dfbed735770e8f2299"} Jun 15 02:29:04.745949 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:04.745882 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-5hrsv" podStartSLOduration=8.657959595 podStartE2EDuration="37.745866643s" totalImagesPullingTime="29.087907048s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 02:28:27 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 02:29:04.74487638 +0000 UTC m=+37.612661125" watchObservedRunningTime="2026-06-15 02:29:04.745866643 +0000 UTC m=+37.613651389" Jun 15 02:29:04.812552 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:04.812529 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c8c3e57a-ae17-4f87-ba38-acc0e14583c4-cert\") pod \"ingress-canary-hz7mx\" (UID: \"c8c3e57a-ae17-4f87-ba38-acc0e14583c4\") " pod="openshift-ingress-canary/ingress-canary-hz7mx" Jun 15 02:29:04.812677 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:04.812558 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1c70b33b-a4a3-4e1e-9693-41d77a5217fd-config-volume\") pod \"dns-default-8wpj7\" (UID: \"1c70b33b-a4a3-4e1e-9693-41d77a5217fd\") " pod="openshift-dns/dns-default-8wpj7" Jun 15 02:29:04.812677 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:04.812574 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1c70b33b-a4a3-4e1e-9693-41d77a5217fd-tmp-dir\") pod \"dns-default-8wpj7\" (UID: \"1c70b33b-a4a3-4e1e-9693-41d77a5217fd\") " pod="openshift-dns/dns-default-8wpj7" Jun 15 02:29:04.812677 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:04.812605 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qpc6\" (UniqueName: \"kubernetes.io/projected/1c70b33b-a4a3-4e1e-9693-41d77a5217fd-kube-api-access-9qpc6\") pod \"dns-default-8wpj7\" (UID: \"1c70b33b-a4a3-4e1e-9693-41d77a5217fd\") " pod="openshift-dns/dns-default-8wpj7" Jun 15 02:29:04.812852 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:04.812690 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1c70b33b-a4a3-4e1e-9693-41d77a5217fd-metrics-tls\") pod \"dns-default-8wpj7\" (UID: \"1c70b33b-a4a3-4e1e-9693-41d77a5217fd\") " pod="openshift-dns/dns-default-8wpj7" Jun 15 02:29:04.812852 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:04.812723 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dtw8\" (UniqueName: \"kubernetes.io/projected/c8c3e57a-ae17-4f87-ba38-acc0e14583c4-kube-api-access-6dtw8\") pod \"ingress-canary-hz7mx\" (UID: \"c8c3e57a-ae17-4f87-ba38-acc0e14583c4\") " pod="openshift-ingress-canary/ingress-canary-hz7mx" Jun 15 02:29:04.812852 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:29:04.812840 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 15 02:29:04.812996 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:29:04.812911 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c70b33b-a4a3-4e1e-9693-41d77a5217fd-metrics-tls podName:1c70b33b-a4a3-4e1e-9693-41d77a5217fd nodeName:}" failed. No retries permitted until 2026-06-15 02:29:05.31289181 +0000 UTC m=+38.180676547 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1c70b33b-a4a3-4e1e-9693-41d77a5217fd-metrics-tls") pod "dns-default-8wpj7" (UID: "1c70b33b-a4a3-4e1e-9693-41d77a5217fd") : secret "dns-default-metrics-tls" not found Jun 15 02:29:04.813156 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:04.813133 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1c70b33b-a4a3-4e1e-9693-41d77a5217fd-tmp-dir\") pod \"dns-default-8wpj7\" (UID: \"1c70b33b-a4a3-4e1e-9693-41d77a5217fd\") " pod="openshift-dns/dns-default-8wpj7" Jun 15 02:29:04.813300 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:04.813280 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1c70b33b-a4a3-4e1e-9693-41d77a5217fd-config-volume\") pod \"dns-default-8wpj7\" (UID: \"1c70b33b-a4a3-4e1e-9693-41d77a5217fd\") " pod="openshift-dns/dns-default-8wpj7" Jun 15 02:29:04.816783 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:04.816760 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c8c3e57a-ae17-4f87-ba38-acc0e14583c4-cert\") pod \"ingress-canary-hz7mx\" (UID: \"c8c3e57a-ae17-4f87-ba38-acc0e14583c4\") " pod="openshift-ingress-canary/ingress-canary-hz7mx" Jun 15 02:29:04.822449 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:04.822430 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dtw8\" (UniqueName: \"kubernetes.io/projected/c8c3e57a-ae17-4f87-ba38-acc0e14583c4-kube-api-access-6dtw8\") pod \"ingress-canary-hz7mx\" (UID: \"c8c3e57a-ae17-4f87-ba38-acc0e14583c4\") " pod="openshift-ingress-canary/ingress-canary-hz7mx" Jun 15 02:29:04.825509 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:04.825486 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qpc6\" (UniqueName: \"kubernetes.io/projected/1c70b33b-a4a3-4e1e-9693-41d77a5217fd-kube-api-access-9qpc6\") pod \"dns-default-8wpj7\" (UID: \"1c70b33b-a4a3-4e1e-9693-41d77a5217fd\") " pod="openshift-dns/dns-default-8wpj7" Jun 15 02:29:04.910823 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:04.910801 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hz7mx" Jun 15 02:29:05.126203 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:05.126175 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hz7mx"] Jun 15 02:29:05.129612 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:29:05.129585 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8c3e57a_ae17_4f87_ba38_acc0e14583c4.slice/crio-fc7d7c3ab8d9d4ca6413afe2cca2972cebb70fff4caa89f515a729cde602b100 WatchSource:0}: Error finding container fc7d7c3ab8d9d4ca6413afe2cca2972cebb70fff4caa89f515a729cde602b100: Status 404 returned error can't find the container with id fc7d7c3ab8d9d4ca6413afe2cca2972cebb70fff4caa89f515a729cde602b100 Jun 15 02:29:05.316750 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:05.316690 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1c70b33b-a4a3-4e1e-9693-41d77a5217fd-metrics-tls\") pod \"dns-default-8wpj7\" (UID: \"1c70b33b-a4a3-4e1e-9693-41d77a5217fd\") " pod="openshift-dns/dns-default-8wpj7" Jun 15 02:29:05.316831 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:29:05.316800 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 15 02:29:05.316870 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:29:05.316843 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c70b33b-a4a3-4e1e-9693-41d77a5217fd-metrics-tls podName:1c70b33b-a4a3-4e1e-9693-41d77a5217fd nodeName:}" failed. No retries permitted until 2026-06-15 02:29:06.316830818 +0000 UTC m=+39.184615540 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1c70b33b-a4a3-4e1e-9693-41d77a5217fd-metrics-tls") pod "dns-default-8wpj7" (UID: "1c70b33b-a4a3-4e1e-9693-41d77a5217fd") : secret "dns-default-metrics-tls" not found Jun 15 02:29:05.728207 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:05.728176 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hz7mx" event={"ID":"c8c3e57a-ae17-4f87-ba38-acc0e14583c4","Type":"ContainerStarted","Data":"fc7d7c3ab8d9d4ca6413afe2cca2972cebb70fff4caa89f515a729cde602b100"} Jun 15 02:29:06.322284 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:06.322253 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1c70b33b-a4a3-4e1e-9693-41d77a5217fd-metrics-tls\") pod \"dns-default-8wpj7\" (UID: \"1c70b33b-a4a3-4e1e-9693-41d77a5217fd\") " pod="openshift-dns/dns-default-8wpj7" Jun 15 02:29:06.322468 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:29:06.322419 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 15 02:29:06.322526 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:29:06.322488 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c70b33b-a4a3-4e1e-9693-41d77a5217fd-metrics-tls podName:1c70b33b-a4a3-4e1e-9693-41d77a5217fd nodeName:}" failed. No retries permitted until 2026-06-15 02:29:08.322471201 +0000 UTC m=+41.190255927 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1c70b33b-a4a3-4e1e-9693-41d77a5217fd-metrics-tls") pod "dns-default-8wpj7" (UID: "1c70b33b-a4a3-4e1e-9693-41d77a5217fd") : secret "dns-default-metrics-tls" not found Jun 15 02:29:06.486769 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:06.486739 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-954mx" Jun 15 02:29:06.486919 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:06.486853 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5qfdl" Jun 15 02:29:06.486919 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:06.486879 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qnxs7" Jun 15 02:29:06.489140 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:06.489122 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 15 02:29:06.489933 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:06.489915 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 15 02:29:06.490186 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:06.490164 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-j9gzq\"" Jun 15 02:29:06.490295 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:06.490196 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 15 02:29:06.490295 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:06.490228 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cch6q\"" Jun 15 02:29:06.490295 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:06.490175 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 15 02:29:08.338076 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:08.338043 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1c70b33b-a4a3-4e1e-9693-41d77a5217fd-metrics-tls\") pod \"dns-default-8wpj7\" (UID: \"1c70b33b-a4a3-4e1e-9693-41d77a5217fd\") " pod="openshift-dns/dns-default-8wpj7" Jun 15 02:29:08.338434 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:29:08.338235 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 15 02:29:08.338434 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:29:08.338309 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c70b33b-a4a3-4e1e-9693-41d77a5217fd-metrics-tls podName:1c70b33b-a4a3-4e1e-9693-41d77a5217fd nodeName:}" failed. No retries permitted until 2026-06-15 02:29:12.338289495 +0000 UTC m=+45.206074241 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1c70b33b-a4a3-4e1e-9693-41d77a5217fd-metrics-tls") pod "dns-default-8wpj7" (UID: "1c70b33b-a4a3-4e1e-9693-41d77a5217fd") : secret "dns-default-metrics-tls" not found Jun 15 02:29:08.736260 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:08.736223 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hz7mx" event={"ID":"c8c3e57a-ae17-4f87-ba38-acc0e14583c4","Type":"ContainerStarted","Data":"f0b3b5162c0daec2a10a6466078f7c5dd472e4f4a81c1ed56de33dfa57133e4b"} Jun 15 02:29:08.751365 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:08.751321 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-hz7mx" podStartSLOduration=1.784912507 podStartE2EDuration="4.751307215s" totalImagesPullingTime="2.966394708s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 02:29:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 02:29:08.750364518 +0000 UTC m=+41.618149260" watchObservedRunningTime="2026-06-15 02:29:08.751307215 +0000 UTC m=+41.619091959" Jun 15 02:29:12.365034 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:12.365001 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1c70b33b-a4a3-4e1e-9693-41d77a5217fd-metrics-tls\") pod \"dns-default-8wpj7\" (UID: \"1c70b33b-a4a3-4e1e-9693-41d77a5217fd\") " pod="openshift-dns/dns-default-8wpj7" Jun 15 02:29:12.365595 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:29:12.365137 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 15 02:29:12.365595 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:29:12.365189 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c70b33b-a4a3-4e1e-9693-41d77a5217fd-metrics-tls podName:1c70b33b-a4a3-4e1e-9693-41d77a5217fd nodeName:}" failed. No retries permitted until 2026-06-15 02:29:20.365176248 +0000 UTC m=+53.232960974 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1c70b33b-a4a3-4e1e-9693-41d77a5217fd-metrics-tls") pod "dns-default-8wpj7" (UID: "1c70b33b-a4a3-4e1e-9693-41d77a5217fd") : secret "dns-default-metrics-tls" not found Jun 15 02:29:20.423389 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:20.423349 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1c70b33b-a4a3-4e1e-9693-41d77a5217fd-metrics-tls\") pod \"dns-default-8wpj7\" (UID: \"1c70b33b-a4a3-4e1e-9693-41d77a5217fd\") " pod="openshift-dns/dns-default-8wpj7" Jun 15 02:29:20.423836 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:29:20.423508 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 15 02:29:20.423836 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:29:20.423577 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c70b33b-a4a3-4e1e-9693-41d77a5217fd-metrics-tls podName:1c70b33b-a4a3-4e1e-9693-41d77a5217fd nodeName:}" failed. No retries permitted until 2026-06-15 02:29:36.423557333 +0000 UTC m=+69.291342080 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1c70b33b-a4a3-4e1e-9693-41d77a5217fd-metrics-tls") pod "dns-default-8wpj7" (UID: "1c70b33b-a4a3-4e1e-9693-41d77a5217fd") : secret "dns-default-metrics-tls" not found Jun 15 02:29:30.716911 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:30.716880 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qg4vc" Jun 15 02:29:33.210340 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:33.210302 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0c770be9-5ffa-4da5-bc85-58fd4c82b027-metrics-certs\") pod \"network-metrics-daemon-954mx\" (UID: \"0c770be9-5ffa-4da5-bc85-58fd4c82b027\") " pod="openshift-multus/network-metrics-daemon-954mx" Jun 15 02:29:33.210740 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:33.210355 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l75z9\" (UniqueName: \"kubernetes.io/projected/91285de0-6ca5-4b23-b554-0c06718dc1df-kube-api-access-l75z9\") pod \"network-check-target-5qfdl\" (UID: \"91285de0-6ca5-4b23-b554-0c06718dc1df\") " pod="openshift-network-diagnostics/network-check-target-5qfdl" Jun 15 02:29:33.212805 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:33.212782 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 15 02:29:33.212851 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:33.212811 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 15 02:29:33.220757 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:29:33.220735 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 15 02:29:33.220813 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:29:33.220806 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c770be9-5ffa-4da5-bc85-58fd4c82b027-metrics-certs podName:0c770be9-5ffa-4da5-bc85-58fd4c82b027 nodeName:}" failed. No retries permitted until 2026-06-15 02:30:37.220790174 +0000 UTC m=+130.088574900 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0c770be9-5ffa-4da5-bc85-58fd4c82b027-metrics-certs") pod "network-metrics-daemon-954mx" (UID: "0c770be9-5ffa-4da5-bc85-58fd4c82b027") : secret "metrics-daemon-secret" not found Jun 15 02:29:33.222616 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:33.222597 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 15 02:29:33.235073 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:33.235053 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l75z9\" (UniqueName: \"kubernetes.io/projected/91285de0-6ca5-4b23-b554-0c06718dc1df-kube-api-access-l75z9\") pod \"network-check-target-5qfdl\" (UID: \"91285de0-6ca5-4b23-b554-0c06718dc1df\") " pod="openshift-network-diagnostics/network-check-target-5qfdl" Jun 15 02:29:33.311067 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:33.311046 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d436483f-cbb4-4ce9-b4d5-a21564cf6e97-original-pull-secret\") pod \"global-pull-secret-syncer-qnxs7\" (UID: \"d436483f-cbb4-4ce9-b4d5-a21564cf6e97\") " pod="kube-system/global-pull-secret-syncer-qnxs7" Jun 15 02:29:33.313563 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:33.313548 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 15 02:29:33.323704 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:33.323681 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d436483f-cbb4-4ce9-b4d5-a21564cf6e97-original-pull-secret\") pod \"global-pull-secret-syncer-qnxs7\" (UID: \"d436483f-cbb4-4ce9-b4d5-a21564cf6e97\") " pod="kube-system/global-pull-secret-syncer-qnxs7" Jun 15 02:29:33.504802 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:33.504783 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cch6q\"" Jun 15 02:29:33.506423 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:33.506402 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qnxs7" Jun 15 02:29:33.512391 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:33.512376 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5qfdl" Jun 15 02:29:33.666855 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:33.666824 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qnxs7"] Jun 15 02:29:33.670082 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:29:33.670061 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd436483f_cbb4_4ce9_b4d5_a21564cf6e97.slice/crio-5b7ed0634a0484b438cd67915500cf7007014e74d60ebe2376907f3f96bb499c WatchSource:0}: Error finding container 5b7ed0634a0484b438cd67915500cf7007014e74d60ebe2376907f3f96bb499c: Status 404 returned error can't find the container with id 5b7ed0634a0484b438cd67915500cf7007014e74d60ebe2376907f3f96bb499c Jun 15 02:29:33.676611 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:33.676585 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5qfdl"] Jun 15 02:29:33.680182 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:29:33.680155 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91285de0_6ca5_4b23_b554_0c06718dc1df.slice/crio-602ae1e6e9a599c3efb0c23495dd226acfaa708d7e2660a0dc2e8d7b26ac45bf WatchSource:0}: Error finding container 602ae1e6e9a599c3efb0c23495dd226acfaa708d7e2660a0dc2e8d7b26ac45bf: Status 404 returned error can't find the container with id 602ae1e6e9a599c3efb0c23495dd226acfaa708d7e2660a0dc2e8d7b26ac45bf Jun 15 02:29:33.793347 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:33.793288 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5qfdl" event={"ID":"91285de0-6ca5-4b23-b554-0c06718dc1df","Type":"ContainerStarted","Data":"602ae1e6e9a599c3efb0c23495dd226acfaa708d7e2660a0dc2e8d7b26ac45bf"} Jun 15 02:29:33.794386 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:33.794366 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qnxs7" event={"ID":"d436483f-cbb4-4ce9-b4d5-a21564cf6e97","Type":"ContainerStarted","Data":"5b7ed0634a0484b438cd67915500cf7007014e74d60ebe2376907f3f96bb499c"} Jun 15 02:29:36.436680 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:36.436600 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1c70b33b-a4a3-4e1e-9693-41d77a5217fd-metrics-tls\") pod \"dns-default-8wpj7\" (UID: \"1c70b33b-a4a3-4e1e-9693-41d77a5217fd\") " pod="openshift-dns/dns-default-8wpj7" Jun 15 02:29:36.437099 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:29:36.436766 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 15 02:29:36.437099 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:29:36.436861 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c70b33b-a4a3-4e1e-9693-41d77a5217fd-metrics-tls podName:1c70b33b-a4a3-4e1e-9693-41d77a5217fd nodeName:}" failed. No retries permitted until 2026-06-15 02:30:08.436839907 +0000 UTC m=+101.304624641 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1c70b33b-a4a3-4e1e-9693-41d77a5217fd-metrics-tls") pod "dns-default-8wpj7" (UID: "1c70b33b-a4a3-4e1e-9693-41d77a5217fd") : secret "dns-default-metrics-tls" not found Jun 15 02:29:38.809471 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:38.809384 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5qfdl" event={"ID":"91285de0-6ca5-4b23-b554-0c06718dc1df","Type":"ContainerStarted","Data":"c14d8e2c3a2818439edd9f13aa373c689e64fac5bf5d38625276e41062cba38a"} Jun 15 02:29:38.809471 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:38.809457 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-5qfdl" Jun 15 02:29:38.810941 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:38.810912 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qnxs7" event={"ID":"d436483f-cbb4-4ce9-b4d5-a21564cf6e97","Type":"ContainerStarted","Data":"10fb64353e09f1166cc80766108d086641cbe410f6e6f4491f2489af2664fe7f"} Jun 15 02:29:38.823902 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:38.823860 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-5qfdl" podStartSLOduration=66.967173299 podStartE2EDuration="1m11.823845596s" totalImagesPullingTime="4.856672297s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 02:28:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 02:29:38.823342595 +0000 UTC m=+71.691127340" watchObservedRunningTime="2026-06-15 02:29:38.823845596 +0000 UTC m=+71.691630344" Jun 15 02:29:38.837158 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:38.837115 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-qnxs7" podStartSLOduration=66.965800401 podStartE2EDuration="1m11.837102891s" totalImagesPullingTime="4.87130249s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 02:28:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 02:29:38.836809974 +0000 UTC m=+71.704594720" watchObservedRunningTime="2026-06-15 02:29:38.837102891 +0000 UTC m=+71.704887638" Jun 15 02:29:55.426466 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:55.426415 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76c5ddf76-tnt45"] Jun 15 02:29:55.431835 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:55.431809 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76c5ddf76-tnt45" Jun 15 02:29:55.434081 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:55.434057 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jun 15 02:29:55.434314 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:55.434295 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jun 15 02:29:55.434894 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:55.434871 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jun 15 02:29:55.435169 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:55.435149 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jun 15 02:29:55.435316 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:55.435179 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-fwlh5\"" Jun 15 02:29:55.436795 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:55.436758 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f45c88d54-xkgkn"] Jun 15 02:29:55.440411 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:55.440384 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76c5ddf76-tnt45"] Jun 15 02:29:55.440536 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:55.440491 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f45c88d54-xkgkn" Jun 15 02:29:55.442830 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:55.442807 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jun 15 02:29:55.448960 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:55.448937 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f45c88d54-xkgkn"] Jun 15 02:29:55.456119 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:55.456096 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/92d2ca8e-7ada-4792-98a3-cf2ca24ead9f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-76c5ddf76-tnt45\" (UID: \"92d2ca8e-7ada-4792-98a3-cf2ca24ead9f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76c5ddf76-tnt45" Jun 15 02:29:55.456211 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:55.456132 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzbkh\" (UniqueName: \"kubernetes.io/projected/92d2ca8e-7ada-4792-98a3-cf2ca24ead9f-kube-api-access-bzbkh\") pod \"managed-serviceaccount-addon-agent-76c5ddf76-tnt45\" (UID: \"92d2ca8e-7ada-4792-98a3-cf2ca24ead9f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76c5ddf76-tnt45" Jun 15 02:29:55.456211 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:55.456190 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7de0ddc2-b6d9-4500-bdf6-93f8db2272fa-tmp\") pod \"klusterlet-addon-workmgr-7f45c88d54-xkgkn\" (UID: \"7de0ddc2-b6d9-4500-bdf6-93f8db2272fa\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f45c88d54-xkgkn" Jun 15 02:29:55.456277 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:55.456222 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/7de0ddc2-b6d9-4500-bdf6-93f8db2272fa-klusterlet-config\") pod \"klusterlet-addon-workmgr-7f45c88d54-xkgkn\" (UID: \"7de0ddc2-b6d9-4500-bdf6-93f8db2272fa\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f45c88d54-xkgkn" Jun 15 02:29:55.456317 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:55.456276 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8c27\" (UniqueName: \"kubernetes.io/projected/7de0ddc2-b6d9-4500-bdf6-93f8db2272fa-kube-api-access-t8c27\") pod \"klusterlet-addon-workmgr-7f45c88d54-xkgkn\" (UID: \"7de0ddc2-b6d9-4500-bdf6-93f8db2272fa\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f45c88d54-xkgkn" Jun 15 02:29:55.557043 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:55.557023 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/92d2ca8e-7ada-4792-98a3-cf2ca24ead9f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-76c5ddf76-tnt45\" (UID: \"92d2ca8e-7ada-4792-98a3-cf2ca24ead9f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76c5ddf76-tnt45" Jun 15 02:29:55.557158 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:55.557061 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzbkh\" (UniqueName: \"kubernetes.io/projected/92d2ca8e-7ada-4792-98a3-cf2ca24ead9f-kube-api-access-bzbkh\") pod \"managed-serviceaccount-addon-agent-76c5ddf76-tnt45\" (UID: \"92d2ca8e-7ada-4792-98a3-cf2ca24ead9f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76c5ddf76-tnt45" Jun 15 02:29:55.557158 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:55.557140 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7de0ddc2-b6d9-4500-bdf6-93f8db2272fa-tmp\") pod \"klusterlet-addon-workmgr-7f45c88d54-xkgkn\" (UID: \"7de0ddc2-b6d9-4500-bdf6-93f8db2272fa\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f45c88d54-xkgkn" Jun 15 02:29:55.557272 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:55.557186 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/7de0ddc2-b6d9-4500-bdf6-93f8db2272fa-klusterlet-config\") pod \"klusterlet-addon-workmgr-7f45c88d54-xkgkn\" (UID: \"7de0ddc2-b6d9-4500-bdf6-93f8db2272fa\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f45c88d54-xkgkn" Jun 15 02:29:55.557272 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:55.557218 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8c27\" (UniqueName: \"kubernetes.io/projected/7de0ddc2-b6d9-4500-bdf6-93f8db2272fa-kube-api-access-t8c27\") pod \"klusterlet-addon-workmgr-7f45c88d54-xkgkn\" (UID: \"7de0ddc2-b6d9-4500-bdf6-93f8db2272fa\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f45c88d54-xkgkn" Jun 15 02:29:55.557590 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:55.557564 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7de0ddc2-b6d9-4500-bdf6-93f8db2272fa-tmp\") pod \"klusterlet-addon-workmgr-7f45c88d54-xkgkn\" (UID: \"7de0ddc2-b6d9-4500-bdf6-93f8db2272fa\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f45c88d54-xkgkn" Jun 15 02:29:55.559687 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:55.559662 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/92d2ca8e-7ada-4792-98a3-cf2ca24ead9f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-76c5ddf76-tnt45\" (UID: \"92d2ca8e-7ada-4792-98a3-cf2ca24ead9f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76c5ddf76-tnt45" Jun 15 02:29:55.559766 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:55.559697 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/7de0ddc2-b6d9-4500-bdf6-93f8db2272fa-klusterlet-config\") pod \"klusterlet-addon-workmgr-7f45c88d54-xkgkn\" (UID: \"7de0ddc2-b6d9-4500-bdf6-93f8db2272fa\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f45c88d54-xkgkn" Jun 15 02:29:55.564761 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:55.564736 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzbkh\" (UniqueName: \"kubernetes.io/projected/92d2ca8e-7ada-4792-98a3-cf2ca24ead9f-kube-api-access-bzbkh\") pod \"managed-serviceaccount-addon-agent-76c5ddf76-tnt45\" (UID: \"92d2ca8e-7ada-4792-98a3-cf2ca24ead9f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76c5ddf76-tnt45" Jun 15 02:29:55.564875 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:55.564852 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8c27\" (UniqueName: \"kubernetes.io/projected/7de0ddc2-b6d9-4500-bdf6-93f8db2272fa-kube-api-access-t8c27\") pod \"klusterlet-addon-workmgr-7f45c88d54-xkgkn\" (UID: \"7de0ddc2-b6d9-4500-bdf6-93f8db2272fa\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f45c88d54-xkgkn" Jun 15 02:29:55.751147 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:55.751118 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76c5ddf76-tnt45" Jun 15 02:29:55.758668 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:55.758647 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f45c88d54-xkgkn" Jun 15 02:29:55.884179 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:55.884140 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76c5ddf76-tnt45"] Jun 15 02:29:55.887504 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:29:55.887481 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod92d2ca8e_7ada_4792_98a3_cf2ca24ead9f.slice/crio-768764b46c85cd4d8796a5e3c15a690ebf7b54fa96c87905c4c7115a8e2ff7a9 WatchSource:0}: Error finding container 768764b46c85cd4d8796a5e3c15a690ebf7b54fa96c87905c4c7115a8e2ff7a9: Status 404 returned error can't find the container with id 768764b46c85cd4d8796a5e3c15a690ebf7b54fa96c87905c4c7115a8e2ff7a9 Jun 15 02:29:55.898521 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:55.898499 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f45c88d54-xkgkn"] Jun 15 02:29:55.901921 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:29:55.901897 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7de0ddc2_b6d9_4500_bdf6_93f8db2272fa.slice/crio-fae17e026f73b61577c43238018ea988628b756bcc4369a7468fd1351f93fc9e WatchSource:0}: Error finding container fae17e026f73b61577c43238018ea988628b756bcc4369a7468fd1351f93fc9e: Status 404 returned error can't find the container with id fae17e026f73b61577c43238018ea988628b756bcc4369a7468fd1351f93fc9e Jun 15 02:29:56.855397 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:56.855334 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76c5ddf76-tnt45" event={"ID":"92d2ca8e-7ada-4792-98a3-cf2ca24ead9f","Type":"ContainerStarted","Data":"768764b46c85cd4d8796a5e3c15a690ebf7b54fa96c87905c4c7115a8e2ff7a9"} Jun 15 02:29:56.856714 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:29:56.856685 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f45c88d54-xkgkn" event={"ID":"7de0ddc2-b6d9-4500-bdf6-93f8db2272fa","Type":"ContainerStarted","Data":"fae17e026f73b61577c43238018ea988628b756bcc4369a7468fd1351f93fc9e"} Jun 15 02:30:00.867903 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:30:00.867870 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f45c88d54-xkgkn" event={"ID":"7de0ddc2-b6d9-4500-bdf6-93f8db2272fa","Type":"ContainerStarted","Data":"9300247a9595476c832ffe7246954d0e329711913ba7a5d7475ffea6df9e4ff6"} Jun 15 02:30:00.868302 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:30:00.868067 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f45c88d54-xkgkn" Jun 15 02:30:00.869473 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:30:00.869452 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f45c88d54-xkgkn" Jun 15 02:30:00.869583 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:30:00.869484 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76c5ddf76-tnt45" event={"ID":"92d2ca8e-7ada-4792-98a3-cf2ca24ead9f","Type":"ContainerStarted","Data":"ad2301b6ddfeb0014e304f539b2cb8a593a62044d3de1eec430062a630715349"} Jun 15 02:30:00.883294 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:30:00.883213 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f45c88d54-xkgkn" podStartSLOduration=1.152077052 podStartE2EDuration="5.883200464s" totalImagesPullingTime="4.731123412s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 02:29:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 02:30:00.882699557 +0000 UTC m=+93.750484302" watchObservedRunningTime="2026-06-15 02:30:00.883200464 +0000 UTC m=+93.750985209" Jun 15 02:30:00.910787 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:30:00.910749 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76c5ddf76-tnt45" podStartSLOduration=1.177593765 podStartE2EDuration="5.910738177s" totalImagesPullingTime="4.733144412s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 02:29:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 02:30:00.910315703 +0000 UTC m=+93.778100447" watchObservedRunningTime="2026-06-15 02:30:00.910738177 +0000 UTC m=+93.778522922" Jun 15 02:30:08.440344 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:30:08.440312 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1c70b33b-a4a3-4e1e-9693-41d77a5217fd-metrics-tls\") pod \"dns-default-8wpj7\" (UID: \"1c70b33b-a4a3-4e1e-9693-41d77a5217fd\") " pod="openshift-dns/dns-default-8wpj7" Jun 15 02:30:08.440754 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:30:08.440433 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 15 02:30:08.440754 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:30:08.440496 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c70b33b-a4a3-4e1e-9693-41d77a5217fd-metrics-tls podName:1c70b33b-a4a3-4e1e-9693-41d77a5217fd nodeName:}" failed. No retries permitted until 2026-06-15 02:31:12.440482093 +0000 UTC m=+165.308266816 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1c70b33b-a4a3-4e1e-9693-41d77a5217fd-metrics-tls") pod "dns-default-8wpj7" (UID: "1c70b33b-a4a3-4e1e-9693-41d77a5217fd") : secret "dns-default-metrics-tls" not found Jun 15 02:30:09.815625 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:30:09.815590 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-5qfdl" Jun 15 02:30:37.321287 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:30:37.321251 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0c770be9-5ffa-4da5-bc85-58fd4c82b027-metrics-certs\") pod \"network-metrics-daemon-954mx\" (UID: \"0c770be9-5ffa-4da5-bc85-58fd4c82b027\") " pod="openshift-multus/network-metrics-daemon-954mx" Jun 15 02:30:37.321752 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:30:37.321372 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 15 02:30:37.321752 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:30:37.321434 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c770be9-5ffa-4da5-bc85-58fd4c82b027-metrics-certs podName:0c770be9-5ffa-4da5-bc85-58fd4c82b027 nodeName:}" failed. No retries permitted until 2026-06-15 02:32:39.321417648 +0000 UTC m=+252.189202370 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0c770be9-5ffa-4da5-bc85-58fd4c82b027-metrics-certs") pod "network-metrics-daemon-954mx" (UID: "0c770be9-5ffa-4da5-bc85-58fd4c82b027") : secret "metrics-daemon-secret" not found Jun 15 02:31:07.584599 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:31:07.584560 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-8wpj7" podUID="1c70b33b-a4a3-4e1e-9693-41d77a5217fd" Jun 15 02:31:08.024125 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:31:08.024098 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8wpj7" Jun 15 02:31:09.496500 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:31:09.496465 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-954mx" podUID="0c770be9-5ffa-4da5-bc85-58fd4c82b027" Jun 15 02:31:12.456173 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:31:12.456090 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1c70b33b-a4a3-4e1e-9693-41d77a5217fd-metrics-tls\") pod \"dns-default-8wpj7\" (UID: \"1c70b33b-a4a3-4e1e-9693-41d77a5217fd\") " pod="openshift-dns/dns-default-8wpj7" Jun 15 02:31:12.456517 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:31:12.456249 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 15 02:31:12.456517 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:31:12.456314 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c70b33b-a4a3-4e1e-9693-41d77a5217fd-metrics-tls podName:1c70b33b-a4a3-4e1e-9693-41d77a5217fd nodeName:}" failed. No retries permitted until 2026-06-15 02:33:14.456295633 +0000 UTC m=+287.324080373 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1c70b33b-a4a3-4e1e-9693-41d77a5217fd-metrics-tls") pod "dns-default-8wpj7" (UID: "1c70b33b-a4a3-4e1e-9693-41d77a5217fd") : secret "dns-default-metrics-tls" not found Jun 15 02:31:21.487053 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:31:21.487017 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-954mx" Jun 15 02:32:00.868842 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:00.868798 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f45c88d54-xkgkn" podUID="7de0ddc2-b6d9-4500-bdf6-93f8db2272fa" containerName="acm-agent" probeResult="failure" output="Get \"http://10.132.0.9:8000/readyz\": dial tcp 10.132.0.9:8000: connect: connection refused" Jun 15 02:32:01.152898 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:01.152872 2579 generic.go:356] "Generic (PLEG): container finished" podID="92d2ca8e-7ada-4792-98a3-cf2ca24ead9f" containerID="ad2301b6ddfeb0014e304f539b2cb8a593a62044d3de1eec430062a630715349" exitCode=255 Jun 15 02:32:01.153022 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:01.152951 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76c5ddf76-tnt45" event={"ID":"92d2ca8e-7ada-4792-98a3-cf2ca24ead9f","Type":"ContainerDied","Data":"ad2301b6ddfeb0014e304f539b2cb8a593a62044d3de1eec430062a630715349"} Jun 15 02:32:01.154331 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:01.154307 2579 generic.go:356] "Generic (PLEG): container finished" podID="7de0ddc2-b6d9-4500-bdf6-93f8db2272fa" containerID="9300247a9595476c832ffe7246954d0e329711913ba7a5d7475ffea6df9e4ff6" exitCode=1 Jun 15 02:32:01.154425 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:01.154347 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f45c88d54-xkgkn" event={"ID":"7de0ddc2-b6d9-4500-bdf6-93f8db2272fa","Type":"ContainerDied","Data":"9300247a9595476c832ffe7246954d0e329711913ba7a5d7475ffea6df9e4ff6"} Jun 15 02:32:01.154622 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:01.154609 2579 scope.go:117] "RemoveContainer" containerID="9300247a9595476c832ffe7246954d0e329711913ba7a5d7475ffea6df9e4ff6" Jun 15 02:32:01.159607 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:01.159592 2579 scope.go:117] "RemoveContainer" containerID="ad2301b6ddfeb0014e304f539b2cb8a593a62044d3de1eec430062a630715349" Jun 15 02:32:02.159486 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:02.159455 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f45c88d54-xkgkn" event={"ID":"7de0ddc2-b6d9-4500-bdf6-93f8db2272fa","Type":"ContainerStarted","Data":"21ef9bb3b05c0a992739336e897a7c6fee6dd4e324d0c2a77c0eb0b0d15f3227"} Jun 15 02:32:02.159932 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:02.159873 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f45c88d54-xkgkn" Jun 15 02:32:02.160376 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:02.160356 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f45c88d54-xkgkn" Jun 15 02:32:02.161405 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:02.161386 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76c5ddf76-tnt45" event={"ID":"92d2ca8e-7ada-4792-98a3-cf2ca24ead9f","Type":"ContainerStarted","Data":"3a957db3b5f138a9728466eeb1190499524a6dedaf4b38848819ad79392395d1"} Jun 15 02:32:17.505714 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:17.505673 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-5dbbb5d744-w4m86"] Jun 15 02:32:17.508677 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:17.508659 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-69d957bb74-n8jmq"] Jun 15 02:32:17.508740 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:17.508717 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-5dbbb5d744-w4m86" Jun 15 02:32:17.511322 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:17.511299 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69d957bb74-n8jmq" Jun 15 02:32:17.511723 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:17.511703 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jun 15 02:32:17.511723 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:17.511705 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-gf4dz\"" Jun 15 02:32:17.511881 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:17.511795 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jun 15 02:32:17.512469 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:17.512447 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jun 15 02:32:17.512534 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:17.512474 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jun 15 02:32:17.514382 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:17.514360 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 15 02:32:17.514598 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:17.514583 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 15 02:32:17.514716 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:17.514605 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-8fqxc\"" Jun 15 02:32:17.516074 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:17.515942 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 15 02:32:17.527249 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:17.527228 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jun 15 02:32:17.527896 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:17.527871 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-5dbbb5d744-w4m86"] Jun 15 02:32:17.529777 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:17.529752 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-69d957bb74-n8jmq"] Jun 15 02:32:17.530599 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:17.530580 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 15 02:32:17.661478 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:17.661449 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fb6051d8-9291-4309-9047-9e8d0b5d2e48-serving-cert\") pod \"console-operator-5dbbb5d744-w4m86\" (UID: \"fb6051d8-9291-4309-9047-9e8d0b5d2e48\") " pod="openshift-console-operator/console-operator-5dbbb5d744-w4m86" Jun 15 02:32:17.661593 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:17.661492 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-installation-pull-secrets\") pod \"image-registry-69d957bb74-n8jmq\" (UID: \"786694ac-916e-4cbd-a6fb-b2fa97ce7bb9\") " pod="openshift-image-registry/image-registry-69d957bb74-n8jmq" Jun 15 02:32:17.661593 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:17.661512 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fb6051d8-9291-4309-9047-9e8d0b5d2e48-trusted-ca\") pod \"console-operator-5dbbb5d744-w4m86\" (UID: \"fb6051d8-9291-4309-9047-9e8d0b5d2e48\") " pod="openshift-console-operator/console-operator-5dbbb5d744-w4m86" Jun 15 02:32:17.661593 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:17.661529 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-image-registry-private-configuration\") pod \"image-registry-69d957bb74-n8jmq\" (UID: \"786694ac-916e-4cbd-a6fb-b2fa97ce7bb9\") " pod="openshift-image-registry/image-registry-69d957bb74-n8jmq" Jun 15 02:32:17.661766 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:17.661603 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-registry-tls\") pod \"image-registry-69d957bb74-n8jmq\" (UID: \"786694ac-916e-4cbd-a6fb-b2fa97ce7bb9\") " pod="openshift-image-registry/image-registry-69d957bb74-n8jmq" Jun 15 02:32:17.661766 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:17.661629 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-registry-certificates\") pod \"image-registry-69d957bb74-n8jmq\" (UID: \"786694ac-916e-4cbd-a6fb-b2fa97ce7bb9\") " pod="openshift-image-registry/image-registry-69d957bb74-n8jmq" Jun 15 02:32:17.661766 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:17.661663 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fb6051d8-9291-4309-9047-9e8d0b5d2e48-config\") pod \"console-operator-5dbbb5d744-w4m86\" (UID: \"fb6051d8-9291-4309-9047-9e8d0b5d2e48\") " pod="openshift-console-operator/console-operator-5dbbb5d744-w4m86" Jun 15 02:32:17.661766 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:17.661684 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rj55\" (UniqueName: \"kubernetes.io/projected/fb6051d8-9291-4309-9047-9e8d0b5d2e48-kube-api-access-2rj55\") pod \"console-operator-5dbbb5d744-w4m86\" (UID: \"fb6051d8-9291-4309-9047-9e8d0b5d2e48\") " pod="openshift-console-operator/console-operator-5dbbb5d744-w4m86" Jun 15 02:32:17.661766 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:17.661730 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrjf7\" (UniqueName: \"kubernetes.io/projected/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-kube-api-access-nrjf7\") pod \"image-registry-69d957bb74-n8jmq\" (UID: \"786694ac-916e-4cbd-a6fb-b2fa97ce7bb9\") " pod="openshift-image-registry/image-registry-69d957bb74-n8jmq" Jun 15 02:32:17.662011 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:17.661872 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-ca-trust-extracted\") pod \"image-registry-69d957bb74-n8jmq\" (UID: \"786694ac-916e-4cbd-a6fb-b2fa97ce7bb9\") " pod="openshift-image-registry/image-registry-69d957bb74-n8jmq" Jun 15 02:32:17.662011 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:17.661897 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-trusted-ca\") pod \"image-registry-69d957bb74-n8jmq\" (UID: \"786694ac-916e-4cbd-a6fb-b2fa97ce7bb9\") " pod="openshift-image-registry/image-registry-69d957bb74-n8jmq" Jun 15 02:32:17.662112 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:17.662092 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-bound-sa-token\") pod \"image-registry-69d957bb74-n8jmq\" (UID: \"786694ac-916e-4cbd-a6fb-b2fa97ce7bb9\") " pod="openshift-image-registry/image-registry-69d957bb74-n8jmq" Jun 15 02:32:17.762667 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:17.762578 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-bound-sa-token\") pod \"image-registry-69d957bb74-n8jmq\" (UID: \"786694ac-916e-4cbd-a6fb-b2fa97ce7bb9\") " pod="openshift-image-registry/image-registry-69d957bb74-n8jmq" Jun 15 02:32:17.762667 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:17.762625 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fb6051d8-9291-4309-9047-9e8d0b5d2e48-serving-cert\") pod \"console-operator-5dbbb5d744-w4m86\" (UID: \"fb6051d8-9291-4309-9047-9e8d0b5d2e48\") " pod="openshift-console-operator/console-operator-5dbbb5d744-w4m86" Jun 15 02:32:17.762667 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:17.762665 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-installation-pull-secrets\") pod \"image-registry-69d957bb74-n8jmq\" (UID: \"786694ac-916e-4cbd-a6fb-b2fa97ce7bb9\") " pod="openshift-image-registry/image-registry-69d957bb74-n8jmq" Jun 15 02:32:17.762913 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:17.762683 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fb6051d8-9291-4309-9047-9e8d0b5d2e48-trusted-ca\") pod \"console-operator-5dbbb5d744-w4m86\" (UID: \"fb6051d8-9291-4309-9047-9e8d0b5d2e48\") " pod="openshift-console-operator/console-operator-5dbbb5d744-w4m86" Jun 15 02:32:17.762913 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:17.762698 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-image-registry-private-configuration\") pod \"image-registry-69d957bb74-n8jmq\" (UID: \"786694ac-916e-4cbd-a6fb-b2fa97ce7bb9\") " pod="openshift-image-registry/image-registry-69d957bb74-n8jmq" Jun 15 02:32:17.762913 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:17.762736 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-registry-tls\") pod \"image-registry-69d957bb74-n8jmq\" (UID: \"786694ac-916e-4cbd-a6fb-b2fa97ce7bb9\") " pod="openshift-image-registry/image-registry-69d957bb74-n8jmq" Jun 15 02:32:17.762913 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:17.762799 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-registry-certificates\") pod \"image-registry-69d957bb74-n8jmq\" (UID: \"786694ac-916e-4cbd-a6fb-b2fa97ce7bb9\") " pod="openshift-image-registry/image-registry-69d957bb74-n8jmq" Jun 15 02:32:17.762913 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:17.762845 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fb6051d8-9291-4309-9047-9e8d0b5d2e48-config\") pod \"console-operator-5dbbb5d744-w4m86\" (UID: \"fb6051d8-9291-4309-9047-9e8d0b5d2e48\") " pod="openshift-console-operator/console-operator-5dbbb5d744-w4m86" Jun 15 02:32:17.762913 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:17.762870 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rj55\" (UniqueName: \"kubernetes.io/projected/fb6051d8-9291-4309-9047-9e8d0b5d2e48-kube-api-access-2rj55\") pod \"console-operator-5dbbb5d744-w4m86\" (UID: \"fb6051d8-9291-4309-9047-9e8d0b5d2e48\") " pod="openshift-console-operator/console-operator-5dbbb5d744-w4m86" Jun 15 02:32:17.762913 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:32:17.762874 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 15 02:32:17.762913 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:32:17.762894 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69d957bb74-n8jmq: secret "image-registry-tls" not found Jun 15 02:32:17.762913 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:17.762895 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrjf7\" (UniqueName: \"kubernetes.io/projected/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-kube-api-access-nrjf7\") pod \"image-registry-69d957bb74-n8jmq\" (UID: \"786694ac-916e-4cbd-a6fb-b2fa97ce7bb9\") " pod="openshift-image-registry/image-registry-69d957bb74-n8jmq" Jun 15 02:32:17.763379 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:17.762930 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-ca-trust-extracted\") pod \"image-registry-69d957bb74-n8jmq\" (UID: \"786694ac-916e-4cbd-a6fb-b2fa97ce7bb9\") " pod="openshift-image-registry/image-registry-69d957bb74-n8jmq" Jun 15 02:32:17.763379 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:32:17.762955 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-registry-tls podName:786694ac-916e-4cbd-a6fb-b2fa97ce7bb9 nodeName:}" failed. No retries permitted until 2026-06-15 02:32:18.262932475 +0000 UTC m=+231.130717214 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-registry-tls") pod "image-registry-69d957bb74-n8jmq" (UID: "786694ac-916e-4cbd-a6fb-b2fa97ce7bb9") : secret "image-registry-tls" not found Jun 15 02:32:17.763379 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:17.762985 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-trusted-ca\") pod \"image-registry-69d957bb74-n8jmq\" (UID: \"786694ac-916e-4cbd-a6fb-b2fa97ce7bb9\") " pod="openshift-image-registry/image-registry-69d957bb74-n8jmq" Jun 15 02:32:17.763553 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:17.763464 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-registry-certificates\") pod \"image-registry-69d957bb74-n8jmq\" (UID: \"786694ac-916e-4cbd-a6fb-b2fa97ce7bb9\") " pod="openshift-image-registry/image-registry-69d957bb74-n8jmq" Jun 15 02:32:17.763624 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:17.763578 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-ca-trust-extracted\") pod \"image-registry-69d957bb74-n8jmq\" (UID: \"786694ac-916e-4cbd-a6fb-b2fa97ce7bb9\") " pod="openshift-image-registry/image-registry-69d957bb74-n8jmq" Jun 15 02:32:17.763624 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:17.763589 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fb6051d8-9291-4309-9047-9e8d0b5d2e48-config\") pod \"console-operator-5dbbb5d744-w4m86\" (UID: \"fb6051d8-9291-4309-9047-9e8d0b5d2e48\") " pod="openshift-console-operator/console-operator-5dbbb5d744-w4m86" Jun 15 02:32:17.764019 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:17.764001 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-trusted-ca\") pod \"image-registry-69d957bb74-n8jmq\" (UID: \"786694ac-916e-4cbd-a6fb-b2fa97ce7bb9\") " pod="openshift-image-registry/image-registry-69d957bb74-n8jmq" Jun 15 02:32:17.764182 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:17.764156 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fb6051d8-9291-4309-9047-9e8d0b5d2e48-trusted-ca\") pod \"console-operator-5dbbb5d744-w4m86\" (UID: \"fb6051d8-9291-4309-9047-9e8d0b5d2e48\") " pod="openshift-console-operator/console-operator-5dbbb5d744-w4m86" Jun 15 02:32:17.765225 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:17.765182 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-installation-pull-secrets\") pod \"image-registry-69d957bb74-n8jmq\" (UID: \"786694ac-916e-4cbd-a6fb-b2fa97ce7bb9\") " pod="openshift-image-registry/image-registry-69d957bb74-n8jmq" Jun 15 02:32:17.765322 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:17.765268 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-image-registry-private-configuration\") pod \"image-registry-69d957bb74-n8jmq\" (UID: \"786694ac-916e-4cbd-a6fb-b2fa97ce7bb9\") " pod="openshift-image-registry/image-registry-69d957bb74-n8jmq" Jun 15 02:32:17.765384 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:17.765368 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fb6051d8-9291-4309-9047-9e8d0b5d2e48-serving-cert\") pod \"console-operator-5dbbb5d744-w4m86\" (UID: \"fb6051d8-9291-4309-9047-9e8d0b5d2e48\") " pod="openshift-console-operator/console-operator-5dbbb5d744-w4m86" Jun 15 02:32:17.769834 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:17.769814 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-bound-sa-token\") pod \"image-registry-69d957bb74-n8jmq\" (UID: \"786694ac-916e-4cbd-a6fb-b2fa97ce7bb9\") " pod="openshift-image-registry/image-registry-69d957bb74-n8jmq" Jun 15 02:32:17.771370 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:17.771347 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rj55\" (UniqueName: \"kubernetes.io/projected/fb6051d8-9291-4309-9047-9e8d0b5d2e48-kube-api-access-2rj55\") pod \"console-operator-5dbbb5d744-w4m86\" (UID: \"fb6051d8-9291-4309-9047-9e8d0b5d2e48\") " pod="openshift-console-operator/console-operator-5dbbb5d744-w4m86" Jun 15 02:32:17.772085 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:17.772065 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrjf7\" (UniqueName: \"kubernetes.io/projected/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-kube-api-access-nrjf7\") pod \"image-registry-69d957bb74-n8jmq\" (UID: \"786694ac-916e-4cbd-a6fb-b2fa97ce7bb9\") " pod="openshift-image-registry/image-registry-69d957bb74-n8jmq" Jun 15 02:32:17.818855 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:17.818833 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-5dbbb5d744-w4m86" Jun 15 02:32:17.937353 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:17.937324 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-5dbbb5d744-w4m86"] Jun 15 02:32:17.940531 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:32:17.940504 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb6051d8_9291_4309_9047_9e8d0b5d2e48.slice/crio-66830ccd3020765df4d075df4fddb7cdc3add520b8a1495b5d91e860cdef3837 WatchSource:0}: Error finding container 66830ccd3020765df4d075df4fddb7cdc3add520b8a1495b5d91e860cdef3837: Status 404 returned error can't find the container with id 66830ccd3020765df4d075df4fddb7cdc3add520b8a1495b5d91e860cdef3837 Jun 15 02:32:18.216986 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:18.216900 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5dbbb5d744-w4m86" event={"ID":"fb6051d8-9291-4309-9047-9e8d0b5d2e48","Type":"ContainerStarted","Data":"66830ccd3020765df4d075df4fddb7cdc3add520b8a1495b5d91e860cdef3837"} Jun 15 02:32:18.265679 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:18.265614 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-registry-tls\") pod \"image-registry-69d957bb74-n8jmq\" (UID: \"786694ac-916e-4cbd-a6fb-b2fa97ce7bb9\") " pod="openshift-image-registry/image-registry-69d957bb74-n8jmq" Jun 15 02:32:18.265790 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:32:18.265743 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 15 02:32:18.265790 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:32:18.265757 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69d957bb74-n8jmq: secret "image-registry-tls" not found Jun 15 02:32:18.265894 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:32:18.265800 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-registry-tls podName:786694ac-916e-4cbd-a6fb-b2fa97ce7bb9 nodeName:}" failed. No retries permitted until 2026-06-15 02:32:19.265787751 +0000 UTC m=+232.133572474 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-registry-tls") pod "image-registry-69d957bb74-n8jmq" (UID: "786694ac-916e-4cbd-a6fb-b2fa97ce7bb9") : secret "image-registry-tls" not found Jun 15 02:32:19.271712 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:19.271678 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-registry-tls\") pod \"image-registry-69d957bb74-n8jmq\" (UID: \"786694ac-916e-4cbd-a6fb-b2fa97ce7bb9\") " pod="openshift-image-registry/image-registry-69d957bb74-n8jmq" Jun 15 02:32:19.272139 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:32:19.271834 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 15 02:32:19.272139 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:32:19.271850 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69d957bb74-n8jmq: secret "image-registry-tls" not found Jun 15 02:32:19.272139 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:32:19.271903 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-registry-tls podName:786694ac-916e-4cbd-a6fb-b2fa97ce7bb9 nodeName:}" failed. No retries permitted until 2026-06-15 02:32:21.271884753 +0000 UTC m=+234.139669487 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-registry-tls") pod "image-registry-69d957bb74-n8jmq" (UID: "786694ac-916e-4cbd-a6fb-b2fa97ce7bb9") : secret "image-registry-tls" not found Jun 15 02:32:20.225266 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:20.225245 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-w4m86_fb6051d8-9291-4309-9047-9e8d0b5d2e48/console-operator/0.log" Jun 15 02:32:20.225360 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:20.225281 2579 generic.go:356] "Generic (PLEG): container finished" podID="fb6051d8-9291-4309-9047-9e8d0b5d2e48" containerID="692c7e4dfd1820e36fe1c4250d15f71ba81cea7e304527b0b65024a504f7f095" exitCode=255 Jun 15 02:32:20.225360 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:20.225351 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5dbbb5d744-w4m86" event={"ID":"fb6051d8-9291-4309-9047-9e8d0b5d2e48","Type":"ContainerDied","Data":"692c7e4dfd1820e36fe1c4250d15f71ba81cea7e304527b0b65024a504f7f095"} Jun 15 02:32:20.225581 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:20.225567 2579 scope.go:117] "RemoveContainer" containerID="692c7e4dfd1820e36fe1c4250d15f71ba81cea7e304527b0b65024a504f7f095" Jun 15 02:32:21.229759 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:21.229735 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-w4m86_fb6051d8-9291-4309-9047-9e8d0b5d2e48/console-operator/1.log" Jun 15 02:32:21.230144 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:21.230078 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-w4m86_fb6051d8-9291-4309-9047-9e8d0b5d2e48/console-operator/0.log" Jun 15 02:32:21.230144 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:21.230108 2579 generic.go:356] "Generic (PLEG): container finished" podID="fb6051d8-9291-4309-9047-9e8d0b5d2e48" containerID="2eb7ffca1cfc97fea2c6f276b652111b5829e5646ead8e5a8a07ee2ecda195a8" exitCode=255 Jun 15 02:32:21.230210 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:21.230144 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5dbbb5d744-w4m86" event={"ID":"fb6051d8-9291-4309-9047-9e8d0b5d2e48","Type":"ContainerDied","Data":"2eb7ffca1cfc97fea2c6f276b652111b5829e5646ead8e5a8a07ee2ecda195a8"} Jun 15 02:32:21.230210 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:21.230188 2579 scope.go:117] "RemoveContainer" containerID="692c7e4dfd1820e36fe1c4250d15f71ba81cea7e304527b0b65024a504f7f095" Jun 15 02:32:21.230355 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:21.230337 2579 scope.go:117] "RemoveContainer" containerID="2eb7ffca1cfc97fea2c6f276b652111b5829e5646ead8e5a8a07ee2ecda195a8" Jun 15 02:32:21.230548 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:32:21.230525 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5dbbb5d744-w4m86_openshift-console-operator(fb6051d8-9291-4309-9047-9e8d0b5d2e48)\"" pod="openshift-console-operator/console-operator-5dbbb5d744-w4m86" podUID="fb6051d8-9291-4309-9047-9e8d0b5d2e48" Jun 15 02:32:21.286708 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:21.286683 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-registry-tls\") pod \"image-registry-69d957bb74-n8jmq\" (UID: \"786694ac-916e-4cbd-a6fb-b2fa97ce7bb9\") " pod="openshift-image-registry/image-registry-69d957bb74-n8jmq" Jun 15 02:32:21.286836 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:32:21.286799 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 15 02:32:21.286836 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:32:21.286814 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69d957bb74-n8jmq: secret "image-registry-tls" not found Jun 15 02:32:21.286944 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:32:21.286871 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-registry-tls podName:786694ac-916e-4cbd-a6fb-b2fa97ce7bb9 nodeName:}" failed. No retries permitted until 2026-06-15 02:32:25.286852705 +0000 UTC m=+238.154637431 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-registry-tls") pod "image-registry-69d957bb74-n8jmq" (UID: "786694ac-916e-4cbd-a6fb-b2fa97ce7bb9") : secret "image-registry-tls" not found Jun 15 02:32:22.234808 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:22.234781 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-w4m86_fb6051d8-9291-4309-9047-9e8d0b5d2e48/console-operator/1.log" Jun 15 02:32:22.235190 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:22.235122 2579 scope.go:117] "RemoveContainer" containerID="2eb7ffca1cfc97fea2c6f276b652111b5829e5646ead8e5a8a07ee2ecda195a8" Jun 15 02:32:22.235314 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:32:22.235293 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5dbbb5d744-w4m86_openshift-console-operator(fb6051d8-9291-4309-9047-9e8d0b5d2e48)\"" pod="openshift-console-operator/console-operator-5dbbb5d744-w4m86" podUID="fb6051d8-9291-4309-9047-9e8d0b5d2e48" Jun 15 02:32:23.345886 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:23.345847 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-lfc67"] Jun 15 02:32:23.350201 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:23.350175 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lfc67" Jun 15 02:32:23.352338 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:23.352314 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 15 02:32:23.352451 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:23.352411 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 15 02:32:23.352513 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:23.352486 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6jzbf\"" Jun 15 02:32:23.353074 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:23.353057 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 15 02:32:23.353074 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:23.353066 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 15 02:32:23.358166 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:23.358139 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lfc67"] Jun 15 02:32:23.503841 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:23.503817 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7d8d\" (UniqueName: \"kubernetes.io/projected/a4c0951e-c8e4-496e-8761-b6856b0fefb3-kube-api-access-w7d8d\") pod \"insights-runtime-extractor-lfc67\" (UID: \"a4c0951e-c8e4-496e-8761-b6856b0fefb3\") " pod="openshift-insights/insights-runtime-extractor-lfc67" Jun 15 02:32:23.503993 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:23.503882 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a4c0951e-c8e4-496e-8761-b6856b0fefb3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lfc67\" (UID: \"a4c0951e-c8e4-496e-8761-b6856b0fefb3\") " pod="openshift-insights/insights-runtime-extractor-lfc67" Jun 15 02:32:23.503993 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:23.503926 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a4c0951e-c8e4-496e-8761-b6856b0fefb3-crio-socket\") pod \"insights-runtime-extractor-lfc67\" (UID: \"a4c0951e-c8e4-496e-8761-b6856b0fefb3\") " pod="openshift-insights/insights-runtime-extractor-lfc67" Jun 15 02:32:23.503993 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:23.503953 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a4c0951e-c8e4-496e-8761-b6856b0fefb3-data-volume\") pod \"insights-runtime-extractor-lfc67\" (UID: \"a4c0951e-c8e4-496e-8761-b6856b0fefb3\") " pod="openshift-insights/insights-runtime-extractor-lfc67" Jun 15 02:32:23.503993 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:23.503978 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a4c0951e-c8e4-496e-8761-b6856b0fefb3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lfc67\" (UID: \"a4c0951e-c8e4-496e-8761-b6856b0fefb3\") " pod="openshift-insights/insights-runtime-extractor-lfc67" Jun 15 02:32:23.604867 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:23.604811 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a4c0951e-c8e4-496e-8761-b6856b0fefb3-data-volume\") pod \"insights-runtime-extractor-lfc67\" (UID: \"a4c0951e-c8e4-496e-8761-b6856b0fefb3\") " pod="openshift-insights/insights-runtime-extractor-lfc67" Jun 15 02:32:23.604867 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:23.604842 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a4c0951e-c8e4-496e-8761-b6856b0fefb3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lfc67\" (UID: \"a4c0951e-c8e4-496e-8761-b6856b0fefb3\") " pod="openshift-insights/insights-runtime-extractor-lfc67" Jun 15 02:32:23.604966 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:23.604875 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7d8d\" (UniqueName: \"kubernetes.io/projected/a4c0951e-c8e4-496e-8761-b6856b0fefb3-kube-api-access-w7d8d\") pod \"insights-runtime-extractor-lfc67\" (UID: \"a4c0951e-c8e4-496e-8761-b6856b0fefb3\") " pod="openshift-insights/insights-runtime-extractor-lfc67" Jun 15 02:32:23.604966 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:23.604928 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a4c0951e-c8e4-496e-8761-b6856b0fefb3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lfc67\" (UID: \"a4c0951e-c8e4-496e-8761-b6856b0fefb3\") " pod="openshift-insights/insights-runtime-extractor-lfc67" Jun 15 02:32:23.604966 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:23.604945 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a4c0951e-c8e4-496e-8761-b6856b0fefb3-crio-socket\") pod \"insights-runtime-extractor-lfc67\" (UID: \"a4c0951e-c8e4-496e-8761-b6856b0fefb3\") " pod="openshift-insights/insights-runtime-extractor-lfc67" Jun 15 02:32:23.604966 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:32:23.604960 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 15 02:32:23.605122 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:32:23.605013 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4c0951e-c8e4-496e-8761-b6856b0fefb3-insights-runtime-extractor-tls podName:a4c0951e-c8e4-496e-8761-b6856b0fefb3 nodeName:}" failed. No retries permitted until 2026-06-15 02:32:24.104998707 +0000 UTC m=+236.972783429 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a4c0951e-c8e4-496e-8761-b6856b0fefb3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lfc67" (UID: "a4c0951e-c8e4-496e-8761-b6856b0fefb3") : secret "insights-runtime-extractor-tls" not found Jun 15 02:32:23.605122 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:32:23.605024 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 15 02:32:23.605122 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:32:23.605077 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a4c0951e-c8e4-496e-8761-b6856b0fefb3-kube-rbac-proxy-cm podName:a4c0951e-c8e4-496e-8761-b6856b0fefb3 nodeName:}" failed. No retries permitted until 2026-06-15 02:32:24.105062896 +0000 UTC m=+236.972847633 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a4c0951e-c8e4-496e-8761-b6856b0fefb3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lfc67" (UID: "a4c0951e-c8e4-496e-8761-b6856b0fefb3") : configmap "kube-rbac-proxy" not found Jun 15 02:32:23.605253 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:23.605123 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a4c0951e-c8e4-496e-8761-b6856b0fefb3-crio-socket\") pod \"insights-runtime-extractor-lfc67\" (UID: \"a4c0951e-c8e4-496e-8761-b6856b0fefb3\") " pod="openshift-insights/insights-runtime-extractor-lfc67" Jun 15 02:32:23.605253 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:23.605125 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a4c0951e-c8e4-496e-8761-b6856b0fefb3-data-volume\") pod \"insights-runtime-extractor-lfc67\" (UID: \"a4c0951e-c8e4-496e-8761-b6856b0fefb3\") " pod="openshift-insights/insights-runtime-extractor-lfc67" Jun 15 02:32:23.612629 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:23.612604 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7d8d\" (UniqueName: \"kubernetes.io/projected/a4c0951e-c8e4-496e-8761-b6856b0fefb3-kube-api-access-w7d8d\") pod \"insights-runtime-extractor-lfc67\" (UID: \"a4c0951e-c8e4-496e-8761-b6856b0fefb3\") " pod="openshift-insights/insights-runtime-extractor-lfc67" Jun 15 02:32:24.108396 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:24.108353 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a4c0951e-c8e4-496e-8761-b6856b0fefb3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lfc67\" (UID: \"a4c0951e-c8e4-496e-8761-b6856b0fefb3\") " pod="openshift-insights/insights-runtime-extractor-lfc67" Jun 15 02:32:24.108563 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:24.108409 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a4c0951e-c8e4-496e-8761-b6856b0fefb3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lfc67\" (UID: \"a4c0951e-c8e4-496e-8761-b6856b0fefb3\") " pod="openshift-insights/insights-runtime-extractor-lfc67" Jun 15 02:32:24.108563 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:32:24.108473 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 15 02:32:24.108563 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:32:24.108515 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 15 02:32:24.108563 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:32:24.108556 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a4c0951e-c8e4-496e-8761-b6856b0fefb3-kube-rbac-proxy-cm podName:a4c0951e-c8e4-496e-8761-b6856b0fefb3 nodeName:}" failed. No retries permitted until 2026-06-15 02:32:25.108539519 +0000 UTC m=+237.976324246 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a4c0951e-c8e4-496e-8761-b6856b0fefb3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lfc67" (UID: "a4c0951e-c8e4-496e-8761-b6856b0fefb3") : configmap "kube-rbac-proxy" not found Jun 15 02:32:24.108757 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:32:24.108572 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4c0951e-c8e4-496e-8761-b6856b0fefb3-insights-runtime-extractor-tls podName:a4c0951e-c8e4-496e-8761-b6856b0fefb3 nodeName:}" failed. No retries permitted until 2026-06-15 02:32:25.108566116 +0000 UTC m=+237.976350839 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a4c0951e-c8e4-496e-8761-b6856b0fefb3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lfc67" (UID: "a4c0951e-c8e4-496e-8761-b6856b0fefb3") : secret "insights-runtime-extractor-tls" not found Jun 15 02:32:25.115782 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:25.115744 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a4c0951e-c8e4-496e-8761-b6856b0fefb3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lfc67\" (UID: \"a4c0951e-c8e4-496e-8761-b6856b0fefb3\") " pod="openshift-insights/insights-runtime-extractor-lfc67" Jun 15 02:32:25.116192 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:25.115797 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a4c0951e-c8e4-496e-8761-b6856b0fefb3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lfc67\" (UID: \"a4c0951e-c8e4-496e-8761-b6856b0fefb3\") " pod="openshift-insights/insights-runtime-extractor-lfc67" Jun 15 02:32:25.116192 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:32:25.115863 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 15 02:32:25.116192 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:32:25.115925 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a4c0951e-c8e4-496e-8761-b6856b0fefb3-kube-rbac-proxy-cm podName:a4c0951e-c8e4-496e-8761-b6856b0fefb3 nodeName:}" failed. No retries permitted until 2026-06-15 02:32:27.115912367 +0000 UTC m=+239.983697089 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a4c0951e-c8e4-496e-8761-b6856b0fefb3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lfc67" (UID: "a4c0951e-c8e4-496e-8761-b6856b0fefb3") : configmap "kube-rbac-proxy" not found Jun 15 02:32:25.116192 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:32:25.115940 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 15 02:32:25.116192 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:32:25.116000 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4c0951e-c8e4-496e-8761-b6856b0fefb3-insights-runtime-extractor-tls podName:a4c0951e-c8e4-496e-8761-b6856b0fefb3 nodeName:}" failed. No retries permitted until 2026-06-15 02:32:27.115985707 +0000 UTC m=+239.983770436 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a4c0951e-c8e4-496e-8761-b6856b0fefb3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lfc67" (UID: "a4c0951e-c8e4-496e-8761-b6856b0fefb3") : secret "insights-runtime-extractor-tls" not found Jun 15 02:32:25.316627 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:25.316592 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-registry-tls\") pod \"image-registry-69d957bb74-n8jmq\" (UID: \"786694ac-916e-4cbd-a6fb-b2fa97ce7bb9\") " pod="openshift-image-registry/image-registry-69d957bb74-n8jmq" Jun 15 02:32:25.316820 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:32:25.316766 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 15 02:32:25.316820 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:32:25.316785 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69d957bb74-n8jmq: secret "image-registry-tls" not found Jun 15 02:32:25.316935 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:32:25.316859 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-registry-tls podName:786694ac-916e-4cbd-a6fb-b2fa97ce7bb9 nodeName:}" failed. No retries permitted until 2026-06-15 02:32:33.316839792 +0000 UTC m=+246.184624514 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-registry-tls") pod "image-registry-69d957bb74-n8jmq" (UID: "786694ac-916e-4cbd-a6fb-b2fa97ce7bb9") : secret "image-registry-tls" not found Jun 15 02:32:26.342527 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:26.342498 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wthd7_f96eb18e-3121-4b4e-a6fb-a0c8bd807285/dns-node-resolver/0.log" Jun 15 02:32:27.127208 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:27.127166 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a4c0951e-c8e4-496e-8761-b6856b0fefb3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lfc67\" (UID: \"a4c0951e-c8e4-496e-8761-b6856b0fefb3\") " pod="openshift-insights/insights-runtime-extractor-lfc67" Jun 15 02:32:27.127208 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:27.127214 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a4c0951e-c8e4-496e-8761-b6856b0fefb3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lfc67\" (UID: \"a4c0951e-c8e4-496e-8761-b6856b0fefb3\") " pod="openshift-insights/insights-runtime-extractor-lfc67" Jun 15 02:32:27.127474 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:32:27.127287 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 15 02:32:27.127474 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:32:27.127310 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 15 02:32:27.127474 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:32:27.127351 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a4c0951e-c8e4-496e-8761-b6856b0fefb3-kube-rbac-proxy-cm podName:a4c0951e-c8e4-496e-8761-b6856b0fefb3 nodeName:}" failed. No retries permitted until 2026-06-15 02:32:31.127335594 +0000 UTC m=+243.995120316 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a4c0951e-c8e4-496e-8761-b6856b0fefb3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lfc67" (UID: "a4c0951e-c8e4-496e-8761-b6856b0fefb3") : configmap "kube-rbac-proxy" not found Jun 15 02:32:27.127474 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:32:27.127387 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4c0951e-c8e4-496e-8761-b6856b0fefb3-insights-runtime-extractor-tls podName:a4c0951e-c8e4-496e-8761-b6856b0fefb3 nodeName:}" failed. No retries permitted until 2026-06-15 02:32:31.127380814 +0000 UTC m=+243.995165537 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a4c0951e-c8e4-496e-8761-b6856b0fefb3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lfc67" (UID: "a4c0951e-c8e4-496e-8761-b6856b0fefb3") : secret "insights-runtime-extractor-tls" not found Jun 15 02:32:27.143109 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:27.143086 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9zrhr_6798a601-4421-44c1-8ef7-3b2527cad953/node-ca/0.log" Jun 15 02:32:27.819812 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:27.819737 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-5dbbb5d744-w4m86" Jun 15 02:32:27.819812 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:27.819788 2579 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-5dbbb5d744-w4m86" Jun 15 02:32:27.820280 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:27.820186 2579 scope.go:117] "RemoveContainer" containerID="2eb7ffca1cfc97fea2c6f276b652111b5829e5646ead8e5a8a07ee2ecda195a8" Jun 15 02:32:27.820376 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:32:27.820355 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5dbbb5d744-w4m86_openshift-console-operator(fb6051d8-9291-4309-9047-9e8d0b5d2e48)\"" pod="openshift-console-operator/console-operator-5dbbb5d744-w4m86" podUID="fb6051d8-9291-4309-9047-9e8d0b5d2e48" Jun 15 02:32:27.943294 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:27.943270 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hz7mx_c8c3e57a-ae17-4f87-ba38-acc0e14583c4/serve-healthcheck-canary/0.log" Jun 15 02:32:31.156835 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:31.156794 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a4c0951e-c8e4-496e-8761-b6856b0fefb3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lfc67\" (UID: \"a4c0951e-c8e4-496e-8761-b6856b0fefb3\") " pod="openshift-insights/insights-runtime-extractor-lfc67" Jun 15 02:32:31.157304 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:31.156846 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a4c0951e-c8e4-496e-8761-b6856b0fefb3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lfc67\" (UID: \"a4c0951e-c8e4-496e-8761-b6856b0fefb3\") " pod="openshift-insights/insights-runtime-extractor-lfc67" Jun 15 02:32:31.157304 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:32:31.156912 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 15 02:32:31.157304 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:32:31.156957 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 15 02:32:31.157304 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:32:31.156972 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a4c0951e-c8e4-496e-8761-b6856b0fefb3-kube-rbac-proxy-cm podName:a4c0951e-c8e4-496e-8761-b6856b0fefb3 nodeName:}" failed. No retries permitted until 2026-06-15 02:32:39.156957734 +0000 UTC m=+252.024742456 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a4c0951e-c8e4-496e-8761-b6856b0fefb3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lfc67" (UID: "a4c0951e-c8e4-496e-8761-b6856b0fefb3") : configmap "kube-rbac-proxy" not found Jun 15 02:32:31.157304 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:32:31.157012 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4c0951e-c8e4-496e-8761-b6856b0fefb3-insights-runtime-extractor-tls podName:a4c0951e-c8e4-496e-8761-b6856b0fefb3 nodeName:}" failed. No retries permitted until 2026-06-15 02:32:39.156997013 +0000 UTC m=+252.024781759 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a4c0951e-c8e4-496e-8761-b6856b0fefb3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lfc67" (UID: "a4c0951e-c8e4-496e-8761-b6856b0fefb3") : secret "insights-runtime-extractor-tls" not found Jun 15 02:32:33.372918 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:33.372887 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-registry-tls\") pod \"image-registry-69d957bb74-n8jmq\" (UID: \"786694ac-916e-4cbd-a6fb-b2fa97ce7bb9\") " pod="openshift-image-registry/image-registry-69d957bb74-n8jmq" Jun 15 02:32:33.375358 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:33.375333 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-registry-tls\") pod \"image-registry-69d957bb74-n8jmq\" (UID: \"786694ac-916e-4cbd-a6fb-b2fa97ce7bb9\") " pod="openshift-image-registry/image-registry-69d957bb74-n8jmq" Jun 15 02:32:33.429545 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:33.429520 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-8fqxc\"" Jun 15 02:32:33.437692 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:33.437670 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69d957bb74-n8jmq" Jun 15 02:32:33.560984 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:33.560957 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-69d957bb74-n8jmq"] Jun 15 02:32:33.564495 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:32:33.564472 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod786694ac_916e_4cbd_a6fb_b2fa97ce7bb9.slice/crio-67bb87ff9a4697ceab17e7dafd2a36ff2159d396f201a1a4aa6e7baf8cfd6a38 WatchSource:0}: Error finding container 67bb87ff9a4697ceab17e7dafd2a36ff2159d396f201a1a4aa6e7baf8cfd6a38: Status 404 returned error can't find the container with id 67bb87ff9a4697ceab17e7dafd2a36ff2159d396f201a1a4aa6e7baf8cfd6a38 Jun 15 02:32:34.272771 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:34.272724 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69d957bb74-n8jmq" event={"ID":"786694ac-916e-4cbd-a6fb-b2fa97ce7bb9","Type":"ContainerStarted","Data":"382146b576089217cca838079767866a1d46127fb0b846d1481b14939082f1e8"} Jun 15 02:32:34.272953 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:34.272781 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69d957bb74-n8jmq" event={"ID":"786694ac-916e-4cbd-a6fb-b2fa97ce7bb9","Type":"ContainerStarted","Data":"67bb87ff9a4697ceab17e7dafd2a36ff2159d396f201a1a4aa6e7baf8cfd6a38"} Jun 15 02:32:34.272953 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:34.272815 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-69d957bb74-n8jmq" Jun 15 02:32:34.291075 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:34.291022 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-69d957bb74-n8jmq" podStartSLOduration=17.291006193 podStartE2EDuration="17.291006193s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 02:32:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 02:32:34.290604116 +0000 UTC m=+247.158388863" watchObservedRunningTime="2026-06-15 02:32:34.291006193 +0000 UTC m=+247.158790937" Jun 15 02:32:39.214794 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:39.214751 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a4c0951e-c8e4-496e-8761-b6856b0fefb3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lfc67\" (UID: \"a4c0951e-c8e4-496e-8761-b6856b0fefb3\") " pod="openshift-insights/insights-runtime-extractor-lfc67" Jun 15 02:32:39.214794 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:39.214811 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a4c0951e-c8e4-496e-8761-b6856b0fefb3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lfc67\" (UID: \"a4c0951e-c8e4-496e-8761-b6856b0fefb3\") " pod="openshift-insights/insights-runtime-extractor-lfc67" Jun 15 02:32:39.215320 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:39.215231 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a4c0951e-c8e4-496e-8761-b6856b0fefb3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lfc67\" (UID: \"a4c0951e-c8e4-496e-8761-b6856b0fefb3\") " pod="openshift-insights/insights-runtime-extractor-lfc67" Jun 15 02:32:39.217054 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:39.217030 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a4c0951e-c8e4-496e-8761-b6856b0fefb3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lfc67\" (UID: \"a4c0951e-c8e4-496e-8761-b6856b0fefb3\") " pod="openshift-insights/insights-runtime-extractor-lfc67" Jun 15 02:32:39.261449 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:39.261427 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6jzbf\"" Jun 15 02:32:39.270051 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:39.270033 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lfc67" Jun 15 02:32:39.393176 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:39.393145 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lfc67"] Jun 15 02:32:39.396677 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:32:39.396654 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4c0951e_c8e4_496e_8761_b6856b0fefb3.slice/crio-b749fc29a28b57ba18b07ffae3f96ae68f898a37d382fd1475c75954e2c86312 WatchSource:0}: Error finding container b749fc29a28b57ba18b07ffae3f96ae68f898a37d382fd1475c75954e2c86312: Status 404 returned error can't find the container with id b749fc29a28b57ba18b07ffae3f96ae68f898a37d382fd1475c75954e2c86312 Jun 15 02:32:39.416584 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:39.416563 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0c770be9-5ffa-4da5-bc85-58fd4c82b027-metrics-certs\") pod \"network-metrics-daemon-954mx\" (UID: \"0c770be9-5ffa-4da5-bc85-58fd4c82b027\") " pod="openshift-multus/network-metrics-daemon-954mx" Jun 15 02:32:39.418688 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:39.418663 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0c770be9-5ffa-4da5-bc85-58fd4c82b027-metrics-certs\") pod \"network-metrics-daemon-954mx\" (UID: \"0c770be9-5ffa-4da5-bc85-58fd4c82b027\") " pod="openshift-multus/network-metrics-daemon-954mx" Jun 15 02:32:39.489466 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:39.489447 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-j9gzq\"" Jun 15 02:32:39.498300 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:39.498281 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-954mx" Jun 15 02:32:39.614686 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:39.614624 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-954mx"] Jun 15 02:32:39.617229 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:32:39.617204 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c770be9_5ffa_4da5_bc85_58fd4c82b027.slice/crio-ace86828bb75ca6fb94e8caf3868f03ed2fcc162b134458cec36471b22ba7fad WatchSource:0}: Error finding container ace86828bb75ca6fb94e8caf3868f03ed2fcc162b134458cec36471b22ba7fad: Status 404 returned error can't find the container with id ace86828bb75ca6fb94e8caf3868f03ed2fcc162b134458cec36471b22ba7fad Jun 15 02:32:40.294921 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:40.294813 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lfc67" event={"ID":"a4c0951e-c8e4-496e-8761-b6856b0fefb3","Type":"ContainerStarted","Data":"b749fc29a28b57ba18b07ffae3f96ae68f898a37d382fd1475c75954e2c86312"} Jun 15 02:32:40.296296 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:40.296262 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-954mx" event={"ID":"0c770be9-5ffa-4da5-bc85-58fd4c82b027","Type":"ContainerStarted","Data":"ace86828bb75ca6fb94e8caf3868f03ed2fcc162b134458cec36471b22ba7fad"} Jun 15 02:32:40.487219 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:40.487188 2579 scope.go:117] "RemoveContainer" containerID="2eb7ffca1cfc97fea2c6f276b652111b5829e5646ead8e5a8a07ee2ecda195a8" Jun 15 02:32:41.306770 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:41.306694 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-w4m86_fb6051d8-9291-4309-9047-9e8d0b5d2e48/console-operator/1.log" Jun 15 02:32:41.307164 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:41.306846 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5dbbb5d744-w4m86" event={"ID":"fb6051d8-9291-4309-9047-9e8d0b5d2e48","Type":"ContainerStarted","Data":"61934d8cb2301786aa7d399fd4ea8322f48381cebbac1158882c39018072c526"} Jun 15 02:32:41.307423 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:41.307333 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-5dbbb5d744-w4m86" Jun 15 02:32:41.310048 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:41.310019 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-954mx" event={"ID":"0c770be9-5ffa-4da5-bc85-58fd4c82b027","Type":"ContainerStarted","Data":"a8b5dcd2fe4b83df0208db38c81e20550863742026e7b0ed46cfec3697c35811"} Jun 15 02:32:41.310154 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:41.310070 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-954mx" event={"ID":"0c770be9-5ffa-4da5-bc85-58fd4c82b027","Type":"ContainerStarted","Data":"73572e955e63e815c59bb0d8b9f831b647027ea91809e71ac893716f08cb8f36"} Jun 15 02:32:41.313834 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:41.313810 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-5dbbb5d744-w4m86" Jun 15 02:32:41.323960 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:41.323911 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-5dbbb5d744-w4m86" podStartSLOduration=22.152789584 podStartE2EDuration="24.323892849s" totalImagesPullingTime="2.171103265s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 02:32:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 02:32:41.322045961 +0000 UTC m=+254.189830708" watchObservedRunningTime="2026-06-15 02:32:41.323892849 +0000 UTC m=+254.191677597" Jun 15 02:32:41.335900 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:41.335861 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-954mx" podStartSLOduration=253.167684562 podStartE2EDuration="4m14.335847472s" totalImagesPullingTime="1.16816291s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 02:28:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 02:32:41.335154141 +0000 UTC m=+254.202938885" watchObservedRunningTime="2026-06-15 02:32:41.335847472 +0000 UTC m=+254.203632217" Jun 15 02:32:43.318393 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:43.318318 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lfc67" event={"ID":"a4c0951e-c8e4-496e-8761-b6856b0fefb3","Type":"ContainerStarted","Data":"e278c82b849e6ccc78efb5bbf6bf053f324f78b202cce54c97b819c372151bb2"} Jun 15 02:32:44.323851 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:44.323819 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lfc67" event={"ID":"a4c0951e-c8e4-496e-8761-b6856b0fefb3","Type":"ContainerStarted","Data":"696a6af3a3290b535df1446fb1083699c683fe22ed68ac510a36cc68165078ba"} Jun 15 02:32:45.328696 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:45.328664 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lfc67" event={"ID":"a4c0951e-c8e4-496e-8761-b6856b0fefb3","Type":"ContainerStarted","Data":"816a7eee0647ddfd2c57712c645cbf2dc33f781851ad7cf22507922e5ea8a6db"} Jun 15 02:32:45.344312 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:45.344263 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-lfc67" podStartSLOduration=16.686899516 podStartE2EDuration="22.34424967s" totalImagesPullingTime="5.657350154s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 02:32:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 02:32:45.343690448 +0000 UTC m=+258.211475190" watchObservedRunningTime="2026-06-15 02:32:45.34424967 +0000 UTC m=+258.212034415" Jun 15 02:32:49.169622 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:49.169586 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-69d957bb74-n8jmq"] Jun 15 02:32:49.173728 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:49.173683 2579 patch_prober.go:28] interesting pod/image-registry-69d957bb74-n8jmq container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 15 02:32:49.173857 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:49.173748 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-69d957bb74-n8jmq" podUID="786694ac-916e-4cbd-a6fb-b2fa97ce7bb9" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 15 02:32:51.168445 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:51.168415 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-b897df459-svdtj"] Jun 15 02:32:51.171482 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:51.171464 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b897df459-svdtj" Jun 15 02:32:51.173580 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:51.173556 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jun 15 02:32:51.173580 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:51.173577 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jun 15 02:32:51.173780 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:51.173589 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jun 15 02:32:51.174328 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:51.174297 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jun 15 02:32:51.174328 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:51.174300 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jun 15 02:32:51.174495 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:51.174347 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jun 15 02:32:51.174495 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:51.174390 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-h8zx5\"" Jun 15 02:32:51.174495 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:51.174351 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jun 15 02:32:51.181475 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:51.181456 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b897df459-svdtj"] Jun 15 02:32:51.192736 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:51.192714 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0580edc3-3dea-43e9-9839-d5600831c982-service-ca\") pod \"console-b897df459-svdtj\" (UID: \"0580edc3-3dea-43e9-9839-d5600831c982\") " pod="openshift-console/console-b897df459-svdtj" Jun 15 02:32:51.192832 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:51.192750 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0580edc3-3dea-43e9-9839-d5600831c982-console-serving-cert\") pod \"console-b897df459-svdtj\" (UID: \"0580edc3-3dea-43e9-9839-d5600831c982\") " pod="openshift-console/console-b897df459-svdtj" Jun 15 02:32:51.192832 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:51.192780 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0580edc3-3dea-43e9-9839-d5600831c982-console-oauth-config\") pod \"console-b897df459-svdtj\" (UID: \"0580edc3-3dea-43e9-9839-d5600831c982\") " pod="openshift-console/console-b897df459-svdtj" Jun 15 02:32:51.192832 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:51.192813 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2rlg\" (UniqueName: \"kubernetes.io/projected/0580edc3-3dea-43e9-9839-d5600831c982-kube-api-access-r2rlg\") pod \"console-b897df459-svdtj\" (UID: \"0580edc3-3dea-43e9-9839-d5600831c982\") " pod="openshift-console/console-b897df459-svdtj" Jun 15 02:32:51.192935 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:51.192862 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0580edc3-3dea-43e9-9839-d5600831c982-console-config\") pod \"console-b897df459-svdtj\" (UID: \"0580edc3-3dea-43e9-9839-d5600831c982\") " pod="openshift-console/console-b897df459-svdtj" Jun 15 02:32:51.192935 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:51.192880 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0580edc3-3dea-43e9-9839-d5600831c982-oauth-serving-cert\") pod \"console-b897df459-svdtj\" (UID: \"0580edc3-3dea-43e9-9839-d5600831c982\") " pod="openshift-console/console-b897df459-svdtj" Jun 15 02:32:51.293879 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:51.293855 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0580edc3-3dea-43e9-9839-d5600831c982-console-serving-cert\") pod \"console-b897df459-svdtj\" (UID: \"0580edc3-3dea-43e9-9839-d5600831c982\") " pod="openshift-console/console-b897df459-svdtj" Jun 15 02:32:51.293981 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:51.293894 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0580edc3-3dea-43e9-9839-d5600831c982-console-oauth-config\") pod \"console-b897df459-svdtj\" (UID: \"0580edc3-3dea-43e9-9839-d5600831c982\") " pod="openshift-console/console-b897df459-svdtj" Jun 15 02:32:51.293981 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:51.293913 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2rlg\" (UniqueName: \"kubernetes.io/projected/0580edc3-3dea-43e9-9839-d5600831c982-kube-api-access-r2rlg\") pod \"console-b897df459-svdtj\" (UID: \"0580edc3-3dea-43e9-9839-d5600831c982\") " pod="openshift-console/console-b897df459-svdtj" Jun 15 02:32:51.293981 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:51.293947 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0580edc3-3dea-43e9-9839-d5600831c982-console-config\") pod \"console-b897df459-svdtj\" (UID: \"0580edc3-3dea-43e9-9839-d5600831c982\") " pod="openshift-console/console-b897df459-svdtj" Jun 15 02:32:51.293981 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:51.293970 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0580edc3-3dea-43e9-9839-d5600831c982-oauth-serving-cert\") pod \"console-b897df459-svdtj\" (UID: \"0580edc3-3dea-43e9-9839-d5600831c982\") " pod="openshift-console/console-b897df459-svdtj" Jun 15 02:32:51.294173 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:51.294012 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0580edc3-3dea-43e9-9839-d5600831c982-service-ca\") pod \"console-b897df459-svdtj\" (UID: \"0580edc3-3dea-43e9-9839-d5600831c982\") " pod="openshift-console/console-b897df459-svdtj" Jun 15 02:32:51.294656 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:51.294617 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0580edc3-3dea-43e9-9839-d5600831c982-service-ca\") pod \"console-b897df459-svdtj\" (UID: \"0580edc3-3dea-43e9-9839-d5600831c982\") " pod="openshift-console/console-b897df459-svdtj" Jun 15 02:32:51.294767 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:51.294751 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0580edc3-3dea-43e9-9839-d5600831c982-oauth-serving-cert\") pod \"console-b897df459-svdtj\" (UID: \"0580edc3-3dea-43e9-9839-d5600831c982\") " pod="openshift-console/console-b897df459-svdtj" Jun 15 02:32:51.294813 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:51.294782 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0580edc3-3dea-43e9-9839-d5600831c982-console-config\") pod \"console-b897df459-svdtj\" (UID: \"0580edc3-3dea-43e9-9839-d5600831c982\") " pod="openshift-console/console-b897df459-svdtj" Jun 15 02:32:51.296156 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:51.296138 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0580edc3-3dea-43e9-9839-d5600831c982-console-serving-cert\") pod \"console-b897df459-svdtj\" (UID: \"0580edc3-3dea-43e9-9839-d5600831c982\") " pod="openshift-console/console-b897df459-svdtj" Jun 15 02:32:51.296272 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:51.296255 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0580edc3-3dea-43e9-9839-d5600831c982-console-oauth-config\") pod \"console-b897df459-svdtj\" (UID: \"0580edc3-3dea-43e9-9839-d5600831c982\") " pod="openshift-console/console-b897df459-svdtj" Jun 15 02:32:51.300909 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:51.300891 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2rlg\" (UniqueName: \"kubernetes.io/projected/0580edc3-3dea-43e9-9839-d5600831c982-kube-api-access-r2rlg\") pod \"console-b897df459-svdtj\" (UID: \"0580edc3-3dea-43e9-9839-d5600831c982\") " pod="openshift-console/console-b897df459-svdtj" Jun 15 02:32:51.480475 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:51.480447 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b897df459-svdtj" Jun 15 02:32:51.601424 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:51.601393 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b897df459-svdtj"] Jun 15 02:32:51.605007 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:32:51.604982 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0580edc3_3dea_43e9_9839_d5600831c982.slice/crio-e763bc498109f5fb730c2c63e5d795a200e546621faa245b06de3ec767b04d6a WatchSource:0}: Error finding container e763bc498109f5fb730c2c63e5d795a200e546621faa245b06de3ec767b04d6a: Status 404 returned error can't find the container with id e763bc498109f5fb730c2c63e5d795a200e546621faa245b06de3ec767b04d6a Jun 15 02:32:52.264997 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:52.264966 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-bml6h"] Jun 15 02:32:52.270782 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:52.270755 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-bml6h" Jun 15 02:32:52.273888 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:52.273794 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jun 15 02:32:52.274145 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:52.274096 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-4pfvz\"" Jun 15 02:32:52.277306 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:52.277285 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-bml6h"] Jun 15 02:32:52.302196 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:52.302170 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/44e88673-e402-4647-81e6-52be405164ef-tls-certificates\") pod \"prometheus-operator-admission-webhook-5cf667749c-bml6h\" (UID: \"44e88673-e402-4647-81e6-52be405164ef\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-bml6h" Jun 15 02:32:52.352960 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:52.352923 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b897df459-svdtj" event={"ID":"0580edc3-3dea-43e9-9839-d5600831c982","Type":"ContainerStarted","Data":"e763bc498109f5fb730c2c63e5d795a200e546621faa245b06de3ec767b04d6a"} Jun 15 02:32:52.403298 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:52.403276 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/44e88673-e402-4647-81e6-52be405164ef-tls-certificates\") pod \"prometheus-operator-admission-webhook-5cf667749c-bml6h\" (UID: \"44e88673-e402-4647-81e6-52be405164ef\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-bml6h" Jun 15 02:32:52.405493 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:52.405478 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/44e88673-e402-4647-81e6-52be405164ef-tls-certificates\") pod \"prometheus-operator-admission-webhook-5cf667749c-bml6h\" (UID: \"44e88673-e402-4647-81e6-52be405164ef\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-bml6h" Jun 15 02:32:52.581320 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:52.581238 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-bml6h" Jun 15 02:32:52.718690 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:52.718590 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-bml6h"] Jun 15 02:32:52.723595 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:32:52.723566 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44e88673_e402_4647_81e6_52be405164ef.slice/crio-e5ee89a8fb9d1625b90bffa02c0cdf2548a65248aa4c433e36326f530cdd4e22 WatchSource:0}: Error finding container e5ee89a8fb9d1625b90bffa02c0cdf2548a65248aa4c433e36326f530cdd4e22: Status 404 returned error can't find the container with id e5ee89a8fb9d1625b90bffa02c0cdf2548a65248aa4c433e36326f530cdd4e22 Jun 15 02:32:53.358897 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:53.358861 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-bml6h" event={"ID":"44e88673-e402-4647-81e6-52be405164ef","Type":"ContainerStarted","Data":"e5ee89a8fb9d1625b90bffa02c0cdf2548a65248aa4c433e36326f530cdd4e22"} Jun 15 02:32:55.367875 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:55.367843 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b897df459-svdtj" event={"ID":"0580edc3-3dea-43e9-9839-d5600831c982","Type":"ContainerStarted","Data":"90d60e1540e4b636003ced235a95e6f2803158d4d62076f4ddac7a9ec371a1e4"} Jun 15 02:32:55.369359 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:55.369337 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-bml6h" event={"ID":"44e88673-e402-4647-81e6-52be405164ef","Type":"ContainerStarted","Data":"4dd6fbb305599f069812f7c58a6106073c6cec8009caebcd8dadd0b163f755a4"} Jun 15 02:32:55.369484 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:55.369377 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-bml6h" Jun 15 02:32:55.373992 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:55.373972 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-bml6h" Jun 15 02:32:55.384668 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:55.384612 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-b897df459-svdtj" podStartSLOduration=1.06935956 podStartE2EDuration="4.384599485s" totalImagesPullingTime="3.315239925s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 02:32:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 02:32:55.383163782 +0000 UTC m=+268.250948521" watchObservedRunningTime="2026-06-15 02:32:55.384599485 +0000 UTC m=+268.252384230" Jun 15 02:32:55.397019 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:55.396976 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-bml6h" podStartSLOduration=1.207956436 podStartE2EDuration="3.396963824s" totalImagesPullingTime="2.189007388s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 02:32:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 02:32:55.396018514 +0000 UTC m=+268.263803261" watchObservedRunningTime="2026-06-15 02:32:55.396963824 +0000 UTC m=+268.264748930" Jun 15 02:32:56.316111 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:56.316071 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-864fcfc6f9-z7lj5"] Jun 15 02:32:56.319323 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:56.319306 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-864fcfc6f9-z7lj5" Jun 15 02:32:56.322515 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:56.322482 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 15 02:32:56.323412 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:56.323392 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jun 15 02:32:56.323508 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:56.323452 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-pq7v2\"" Jun 15 02:32:56.323508 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:56.323500 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jun 15 02:32:56.323625 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:56.323557 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 15 02:32:56.323822 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:56.323804 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 15 02:32:56.333054 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:56.333025 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/09014b39-704e-4344-a80e-535a5a9a0e81-metrics-client-ca\") pod \"prometheus-operator-864fcfc6f9-z7lj5\" (UID: \"09014b39-704e-4344-a80e-535a5a9a0e81\") " pod="openshift-monitoring/prometheus-operator-864fcfc6f9-z7lj5" Jun 15 02:32:56.333146 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:56.333066 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/09014b39-704e-4344-a80e-535a5a9a0e81-prometheus-operator-tls\") pod \"prometheus-operator-864fcfc6f9-z7lj5\" (UID: \"09014b39-704e-4344-a80e-535a5a9a0e81\") " pod="openshift-monitoring/prometheus-operator-864fcfc6f9-z7lj5" Jun 15 02:32:56.333146 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:56.333089 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/09014b39-704e-4344-a80e-535a5a9a0e81-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-864fcfc6f9-z7lj5\" (UID: \"09014b39-704e-4344-a80e-535a5a9a0e81\") " pod="openshift-monitoring/prometheus-operator-864fcfc6f9-z7lj5" Jun 15 02:32:56.333146 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:56.333137 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6tv4\" (UniqueName: \"kubernetes.io/projected/09014b39-704e-4344-a80e-535a5a9a0e81-kube-api-access-t6tv4\") pod \"prometheus-operator-864fcfc6f9-z7lj5\" (UID: \"09014b39-704e-4344-a80e-535a5a9a0e81\") " pod="openshift-monitoring/prometheus-operator-864fcfc6f9-z7lj5" Jun 15 02:32:56.342265 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:56.342246 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-864fcfc6f9-z7lj5"] Jun 15 02:32:56.433557 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:56.433525 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/09014b39-704e-4344-a80e-535a5a9a0e81-prometheus-operator-tls\") pod \"prometheus-operator-864fcfc6f9-z7lj5\" (UID: \"09014b39-704e-4344-a80e-535a5a9a0e81\") " pod="openshift-monitoring/prometheus-operator-864fcfc6f9-z7lj5" Jun 15 02:32:56.433886 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:56.433577 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/09014b39-704e-4344-a80e-535a5a9a0e81-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-864fcfc6f9-z7lj5\" (UID: \"09014b39-704e-4344-a80e-535a5a9a0e81\") " pod="openshift-monitoring/prometheus-operator-864fcfc6f9-z7lj5" Jun 15 02:32:56.433886 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:32:56.433694 2579 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jun 15 02:32:56.433886 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:56.433711 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6tv4\" (UniqueName: \"kubernetes.io/projected/09014b39-704e-4344-a80e-535a5a9a0e81-kube-api-access-t6tv4\") pod \"prometheus-operator-864fcfc6f9-z7lj5\" (UID: \"09014b39-704e-4344-a80e-535a5a9a0e81\") " pod="openshift-monitoring/prometheus-operator-864fcfc6f9-z7lj5" Jun 15 02:32:56.433886 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:32:56.433747 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09014b39-704e-4344-a80e-535a5a9a0e81-prometheus-operator-tls podName:09014b39-704e-4344-a80e-535a5a9a0e81 nodeName:}" failed. No retries permitted until 2026-06-15 02:32:56.933732854 +0000 UTC m=+269.801517577 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/09014b39-704e-4344-a80e-535a5a9a0e81-prometheus-operator-tls") pod "prometheus-operator-864fcfc6f9-z7lj5" (UID: "09014b39-704e-4344-a80e-535a5a9a0e81") : secret "prometheus-operator-tls" not found Jun 15 02:32:56.434096 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:56.433901 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/09014b39-704e-4344-a80e-535a5a9a0e81-metrics-client-ca\") pod \"prometheus-operator-864fcfc6f9-z7lj5\" (UID: \"09014b39-704e-4344-a80e-535a5a9a0e81\") " pod="openshift-monitoring/prometheus-operator-864fcfc6f9-z7lj5" Jun 15 02:32:56.434586 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:56.434568 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/09014b39-704e-4344-a80e-535a5a9a0e81-metrics-client-ca\") pod \"prometheus-operator-864fcfc6f9-z7lj5\" (UID: \"09014b39-704e-4344-a80e-535a5a9a0e81\") " pod="openshift-monitoring/prometheus-operator-864fcfc6f9-z7lj5" Jun 15 02:32:56.435945 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:56.435928 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/09014b39-704e-4344-a80e-535a5a9a0e81-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-864fcfc6f9-z7lj5\" (UID: \"09014b39-704e-4344-a80e-535a5a9a0e81\") " pod="openshift-monitoring/prometheus-operator-864fcfc6f9-z7lj5" Jun 15 02:32:56.441503 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:56.441484 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6tv4\" (UniqueName: \"kubernetes.io/projected/09014b39-704e-4344-a80e-535a5a9a0e81-kube-api-access-t6tv4\") pod \"prometheus-operator-864fcfc6f9-z7lj5\" (UID: \"09014b39-704e-4344-a80e-535a5a9a0e81\") " pod="openshift-monitoring/prometheus-operator-864fcfc6f9-z7lj5" Jun 15 02:32:56.936865 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:56.936829 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/09014b39-704e-4344-a80e-535a5a9a0e81-prometheus-operator-tls\") pod \"prometheus-operator-864fcfc6f9-z7lj5\" (UID: \"09014b39-704e-4344-a80e-535a5a9a0e81\") " pod="openshift-monitoring/prometheus-operator-864fcfc6f9-z7lj5" Jun 15 02:32:56.939106 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:56.939087 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/09014b39-704e-4344-a80e-535a5a9a0e81-prometheus-operator-tls\") pod \"prometheus-operator-864fcfc6f9-z7lj5\" (UID: \"09014b39-704e-4344-a80e-535a5a9a0e81\") " pod="openshift-monitoring/prometheus-operator-864fcfc6f9-z7lj5" Jun 15 02:32:57.228603 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:57.228582 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-864fcfc6f9-z7lj5" Jun 15 02:32:57.347143 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:57.347115 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-864fcfc6f9-z7lj5"] Jun 15 02:32:57.350046 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:32:57.350022 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09014b39_704e_4344_a80e_535a5a9a0e81.slice/crio-eba697678fe2803b29c1d95380a6cd66f7443bc2fbfa7f2763355f321ead7ef2 WatchSource:0}: Error finding container eba697678fe2803b29c1d95380a6cd66f7443bc2fbfa7f2763355f321ead7ef2: Status 404 returned error can't find the container with id eba697678fe2803b29c1d95380a6cd66f7443bc2fbfa7f2763355f321ead7ef2 Jun 15 02:32:57.377822 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:57.377793 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-864fcfc6f9-z7lj5" event={"ID":"09014b39-704e-4344-a80e-535a5a9a0e81","Type":"ContainerStarted","Data":"eba697678fe2803b29c1d95380a6cd66f7443bc2fbfa7f2763355f321ead7ef2"} Jun 15 02:32:59.174273 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:59.174236 2579 patch_prober.go:28] interesting pod/image-registry-69d957bb74-n8jmq container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 15 02:32:59.174653 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:59.174287 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-69d957bb74-n8jmq" podUID="786694ac-916e-4cbd-a6fb-b2fa97ce7bb9" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 15 02:32:59.387301 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:59.387271 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-864fcfc6f9-z7lj5" event={"ID":"09014b39-704e-4344-a80e-535a5a9a0e81","Type":"ContainerStarted","Data":"4dc992e587dc87460be6ca42c3273952d8c238d3e4d5d8b0aef8b90cf74ae2fd"} Jun 15 02:32:59.387430 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:59.387318 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-864fcfc6f9-z7lj5" event={"ID":"09014b39-704e-4344-a80e-535a5a9a0e81","Type":"ContainerStarted","Data":"1149db18bf063dd368d6be3618373fe74c840cb3fab5edd85899f19ca98ef81e"} Jun 15 02:32:59.403015 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:32:59.402967 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-864fcfc6f9-z7lj5" podStartSLOduration=2.164317 podStartE2EDuration="3.402950963s" totalImagesPullingTime="1.238633963s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 02:32:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 02:32:59.402722063 +0000 UTC m=+272.270506837" watchObservedRunningTime="2026-06-15 02:32:59.402950963 +0000 UTC m=+272.270735708" Jun 15 02:33:01.481528 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:01.481501 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-b897df459-svdtj" Jun 15 02:33:01.481938 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:01.481580 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-b897df459-svdtj" Jun 15 02:33:01.482746 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:01.482723 2579 patch_prober.go:28] interesting pod/console-b897df459-svdtj container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.132.0.13:8443/health\": dial tcp 10.132.0.13:8443: connect: connection refused" start-of-body= Jun 15 02:33:01.482862 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:01.482768 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-b897df459-svdtj" podUID="0580edc3-3dea-43e9-9839-d5600831c982" containerName="console" probeResult="failure" output="Get \"https://10.132.0.13:8443/health\": dial tcp 10.132.0.13:8443: connect: connection refused" Jun 15 02:33:01.645301 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:01.645265 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-6b8d965dd5-4m5dz"] Jun 15 02:33:01.651696 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:01.651673 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-4m5dz" Jun 15 02:33:01.653921 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:01.653902 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jun 15 02:33:01.654029 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:01.653900 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jun 15 02:33:01.654029 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:01.653954 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-4vfbq\"" Jun 15 02:33:01.659876 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:01.659851 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-6b8d965dd5-4m5dz"] Jun 15 02:33:01.668753 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:01.668729 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/cfbbc525-96a9-4c20-b125-742b83a9db41-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6b8d965dd5-4m5dz\" (UID: \"cfbbc525-96a9-4c20-b125-742b83a9db41\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-4m5dz" Jun 15 02:33:01.668853 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:01.668778 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lc4xq\" (UniqueName: \"kubernetes.io/projected/cfbbc525-96a9-4c20-b125-742b83a9db41-kube-api-access-lc4xq\") pod \"openshift-state-metrics-6b8d965dd5-4m5dz\" (UID: \"cfbbc525-96a9-4c20-b125-742b83a9db41\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-4m5dz" Jun 15 02:33:01.668853 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:01.668831 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cfbbc525-96a9-4c20-b125-742b83a9db41-metrics-client-ca\") pod \"openshift-state-metrics-6b8d965dd5-4m5dz\" (UID: \"cfbbc525-96a9-4c20-b125-742b83a9db41\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-4m5dz" Jun 15 02:33:01.668960 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:01.668895 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cfbbc525-96a9-4c20-b125-742b83a9db41-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-6b8d965dd5-4m5dz\" (UID: \"cfbbc525-96a9-4c20-b125-742b83a9db41\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-4m5dz" Jun 15 02:33:01.682833 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:01.682803 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-l6v8k"] Jun 15 02:33:01.685973 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:01.685956 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-l6v8k" Jun 15 02:33:01.688028 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:01.688010 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 15 02:33:01.688105 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:01.688055 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 15 02:33:01.688340 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:01.688323 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 15 02:33:01.688423 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:01.688403 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-s4z92\"" Jun 15 02:33:01.769956 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:01.769906 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qbcf\" (UniqueName: \"kubernetes.io/projected/4bf0467b-72a9-4ab5-a525-2385561f9da4-kube-api-access-2qbcf\") pod \"node-exporter-l6v8k\" (UID: \"4bf0467b-72a9-4ab5-a525-2385561f9da4\") " pod="openshift-monitoring/node-exporter-l6v8k" Jun 15 02:33:01.769956 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:01.769945 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4bf0467b-72a9-4ab5-a525-2385561f9da4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-l6v8k\" (UID: \"4bf0467b-72a9-4ab5-a525-2385561f9da4\") " pod="openshift-monitoring/node-exporter-l6v8k" Jun 15 02:33:01.770091 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:01.769972 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cfbbc525-96a9-4c20-b125-742b83a9db41-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-6b8d965dd5-4m5dz\" (UID: \"cfbbc525-96a9-4c20-b125-742b83a9db41\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-4m5dz" Jun 15 02:33:01.770091 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:01.770069 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4bf0467b-72a9-4ab5-a525-2385561f9da4-metrics-client-ca\") pod \"node-exporter-l6v8k\" (UID: \"4bf0467b-72a9-4ab5-a525-2385561f9da4\") " pod="openshift-monitoring/node-exporter-l6v8k" Jun 15 02:33:01.770174 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:01.770101 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4bf0467b-72a9-4ab5-a525-2385561f9da4-root\") pod \"node-exporter-l6v8k\" (UID: \"4bf0467b-72a9-4ab5-a525-2385561f9da4\") " pod="openshift-monitoring/node-exporter-l6v8k" Jun 15 02:33:01.770174 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:01.770116 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4bf0467b-72a9-4ab5-a525-2385561f9da4-node-exporter-textfile\") pod \"node-exporter-l6v8k\" (UID: \"4bf0467b-72a9-4ab5-a525-2385561f9da4\") " pod="openshift-monitoring/node-exporter-l6v8k" Jun 15 02:33:01.770174 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:01.770136 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4bf0467b-72a9-4ab5-a525-2385561f9da4-node-exporter-accelerators-collector-config\") pod \"node-exporter-l6v8k\" (UID: \"4bf0467b-72a9-4ab5-a525-2385561f9da4\") " pod="openshift-monitoring/node-exporter-l6v8k" Jun 15 02:33:01.770273 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:01.770177 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/cfbbc525-96a9-4c20-b125-742b83a9db41-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6b8d965dd5-4m5dz\" (UID: \"cfbbc525-96a9-4c20-b125-742b83a9db41\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-4m5dz" Jun 15 02:33:01.770273 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:01.770198 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4bf0467b-72a9-4ab5-a525-2385561f9da4-node-exporter-wtmp\") pod \"node-exporter-l6v8k\" (UID: \"4bf0467b-72a9-4ab5-a525-2385561f9da4\") " pod="openshift-monitoring/node-exporter-l6v8k" Jun 15 02:33:01.770273 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:01.770221 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4bf0467b-72a9-4ab5-a525-2385561f9da4-sys\") pod \"node-exporter-l6v8k\" (UID: \"4bf0467b-72a9-4ab5-a525-2385561f9da4\") " pod="openshift-monitoring/node-exporter-l6v8k" Jun 15 02:33:01.770273 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:01.770243 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4bf0467b-72a9-4ab5-a525-2385561f9da4-node-exporter-tls\") pod \"node-exporter-l6v8k\" (UID: \"4bf0467b-72a9-4ab5-a525-2385561f9da4\") " pod="openshift-monitoring/node-exporter-l6v8k" Jun 15 02:33:01.770444 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:33:01.770280 2579 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jun 15 02:33:01.770444 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:01.770304 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lc4xq\" (UniqueName: \"kubernetes.io/projected/cfbbc525-96a9-4c20-b125-742b83a9db41-kube-api-access-lc4xq\") pod \"openshift-state-metrics-6b8d965dd5-4m5dz\" (UID: \"cfbbc525-96a9-4c20-b125-742b83a9db41\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-4m5dz" Jun 15 02:33:01.770444 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:33:01.770330 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cfbbc525-96a9-4c20-b125-742b83a9db41-openshift-state-metrics-tls podName:cfbbc525-96a9-4c20-b125-742b83a9db41 nodeName:}" failed. No retries permitted until 2026-06-15 02:33:02.270313881 +0000 UTC m=+275.138098607 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/cfbbc525-96a9-4c20-b125-742b83a9db41-openshift-state-metrics-tls") pod "openshift-state-metrics-6b8d965dd5-4m5dz" (UID: "cfbbc525-96a9-4c20-b125-742b83a9db41") : secret "openshift-state-metrics-tls" not found Jun 15 02:33:01.770444 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:01.770364 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cfbbc525-96a9-4c20-b125-742b83a9db41-metrics-client-ca\") pod \"openshift-state-metrics-6b8d965dd5-4m5dz\" (UID: \"cfbbc525-96a9-4c20-b125-742b83a9db41\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-4m5dz" Jun 15 02:33:01.770969 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:01.770950 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cfbbc525-96a9-4c20-b125-742b83a9db41-metrics-client-ca\") pod \"openshift-state-metrics-6b8d965dd5-4m5dz\" (UID: \"cfbbc525-96a9-4c20-b125-742b83a9db41\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-4m5dz" Jun 15 02:33:01.772304 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:01.772285 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cfbbc525-96a9-4c20-b125-742b83a9db41-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-6b8d965dd5-4m5dz\" (UID: \"cfbbc525-96a9-4c20-b125-742b83a9db41\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-4m5dz" Jun 15 02:33:01.777883 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:01.777862 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lc4xq\" (UniqueName: \"kubernetes.io/projected/cfbbc525-96a9-4c20-b125-742b83a9db41-kube-api-access-lc4xq\") pod \"openshift-state-metrics-6b8d965dd5-4m5dz\" (UID: \"cfbbc525-96a9-4c20-b125-742b83a9db41\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-4m5dz" Jun 15 02:33:01.870852 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:01.870826 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4bf0467b-72a9-4ab5-a525-2385561f9da4-node-exporter-wtmp\") pod \"node-exporter-l6v8k\" (UID: \"4bf0467b-72a9-4ab5-a525-2385561f9da4\") " pod="openshift-monitoring/node-exporter-l6v8k" Jun 15 02:33:01.870852 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:01.870853 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4bf0467b-72a9-4ab5-a525-2385561f9da4-sys\") pod \"node-exporter-l6v8k\" (UID: \"4bf0467b-72a9-4ab5-a525-2385561f9da4\") " pod="openshift-monitoring/node-exporter-l6v8k" Jun 15 02:33:01.870977 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:01.870869 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4bf0467b-72a9-4ab5-a525-2385561f9da4-node-exporter-tls\") pod \"node-exporter-l6v8k\" (UID: \"4bf0467b-72a9-4ab5-a525-2385561f9da4\") " pod="openshift-monitoring/node-exporter-l6v8k" Jun 15 02:33:01.870977 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:01.870910 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qbcf\" (UniqueName: \"kubernetes.io/projected/4bf0467b-72a9-4ab5-a525-2385561f9da4-kube-api-access-2qbcf\") pod \"node-exporter-l6v8k\" (UID: \"4bf0467b-72a9-4ab5-a525-2385561f9da4\") " pod="openshift-monitoring/node-exporter-l6v8k" Jun 15 02:33:01.870977 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:01.870935 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4bf0467b-72a9-4ab5-a525-2385561f9da4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-l6v8k\" (UID: \"4bf0467b-72a9-4ab5-a525-2385561f9da4\") " pod="openshift-monitoring/node-exporter-l6v8k" Jun 15 02:33:01.870977 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:01.870941 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4bf0467b-72a9-4ab5-a525-2385561f9da4-sys\") pod \"node-exporter-l6v8k\" (UID: \"4bf0467b-72a9-4ab5-a525-2385561f9da4\") " pod="openshift-monitoring/node-exporter-l6v8k" Jun 15 02:33:01.870977 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:01.870964 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4bf0467b-72a9-4ab5-a525-2385561f9da4-metrics-client-ca\") pod \"node-exporter-l6v8k\" (UID: \"4bf0467b-72a9-4ab5-a525-2385561f9da4\") " pod="openshift-monitoring/node-exporter-l6v8k" Jun 15 02:33:01.871214 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:01.871049 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4bf0467b-72a9-4ab5-a525-2385561f9da4-node-exporter-wtmp\") pod \"node-exporter-l6v8k\" (UID: \"4bf0467b-72a9-4ab5-a525-2385561f9da4\") " pod="openshift-monitoring/node-exporter-l6v8k" Jun 15 02:33:01.871214 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:01.871071 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4bf0467b-72a9-4ab5-a525-2385561f9da4-root\") pod \"node-exporter-l6v8k\" (UID: \"4bf0467b-72a9-4ab5-a525-2385561f9da4\") " pod="openshift-monitoring/node-exporter-l6v8k" Jun 15 02:33:01.871214 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:01.871095 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4bf0467b-72a9-4ab5-a525-2385561f9da4-node-exporter-textfile\") pod \"node-exporter-l6v8k\" (UID: \"4bf0467b-72a9-4ab5-a525-2385561f9da4\") " pod="openshift-monitoring/node-exporter-l6v8k" Jun 15 02:33:01.871214 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:01.871121 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4bf0467b-72a9-4ab5-a525-2385561f9da4-node-exporter-accelerators-collector-config\") pod \"node-exporter-l6v8k\" (UID: \"4bf0467b-72a9-4ab5-a525-2385561f9da4\") " pod="openshift-monitoring/node-exporter-l6v8k" Jun 15 02:33:01.871214 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:01.871170 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4bf0467b-72a9-4ab5-a525-2385561f9da4-root\") pod \"node-exporter-l6v8k\" (UID: \"4bf0467b-72a9-4ab5-a525-2385561f9da4\") " pod="openshift-monitoring/node-exporter-l6v8k" Jun 15 02:33:01.871522 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:01.871503 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4bf0467b-72a9-4ab5-a525-2385561f9da4-node-exporter-textfile\") pod \"node-exporter-l6v8k\" (UID: \"4bf0467b-72a9-4ab5-a525-2385561f9da4\") " pod="openshift-monitoring/node-exporter-l6v8k" Jun 15 02:33:01.871609 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:01.871594 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4bf0467b-72a9-4ab5-a525-2385561f9da4-metrics-client-ca\") pod \"node-exporter-l6v8k\" (UID: \"4bf0467b-72a9-4ab5-a525-2385561f9da4\") " pod="openshift-monitoring/node-exporter-l6v8k" Jun 15 02:33:01.871721 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:01.871701 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4bf0467b-72a9-4ab5-a525-2385561f9da4-node-exporter-accelerators-collector-config\") pod \"node-exporter-l6v8k\" (UID: \"4bf0467b-72a9-4ab5-a525-2385561f9da4\") " pod="openshift-monitoring/node-exporter-l6v8k" Jun 15 02:33:01.873008 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:01.872988 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4bf0467b-72a9-4ab5-a525-2385561f9da4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-l6v8k\" (UID: \"4bf0467b-72a9-4ab5-a525-2385561f9da4\") " pod="openshift-monitoring/node-exporter-l6v8k" Jun 15 02:33:01.873141 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:01.873125 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4bf0467b-72a9-4ab5-a525-2385561f9da4-node-exporter-tls\") pod \"node-exporter-l6v8k\" (UID: \"4bf0467b-72a9-4ab5-a525-2385561f9da4\") " pod="openshift-monitoring/node-exporter-l6v8k" Jun 15 02:33:01.878541 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:01.878523 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qbcf\" (UniqueName: \"kubernetes.io/projected/4bf0467b-72a9-4ab5-a525-2385561f9da4-kube-api-access-2qbcf\") pod \"node-exporter-l6v8k\" (UID: \"4bf0467b-72a9-4ab5-a525-2385561f9da4\") " pod="openshift-monitoring/node-exporter-l6v8k" Jun 15 02:33:01.994412 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:01.994386 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-l6v8k" Jun 15 02:33:02.005279 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:33:02.005252 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bf0467b_72a9_4ab5_a525_2385561f9da4.slice/crio-a52ce9cdc990196e5616d779d26fc22cb5b4aac19bf014e45b2f6b8e1d85dc4f WatchSource:0}: Error finding container a52ce9cdc990196e5616d779d26fc22cb5b4aac19bf014e45b2f6b8e1d85dc4f: Status 404 returned error can't find the container with id a52ce9cdc990196e5616d779d26fc22cb5b4aac19bf014e45b2f6b8e1d85dc4f Jun 15 02:33:02.275235 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:02.275209 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/cfbbc525-96a9-4c20-b125-742b83a9db41-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6b8d965dd5-4m5dz\" (UID: \"cfbbc525-96a9-4c20-b125-742b83a9db41\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-4m5dz" Jun 15 02:33:02.277551 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:02.277531 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/cfbbc525-96a9-4c20-b125-742b83a9db41-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6b8d965dd5-4m5dz\" (UID: \"cfbbc525-96a9-4c20-b125-742b83a9db41\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-4m5dz" Jun 15 02:33:02.397959 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:02.397930 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l6v8k" event={"ID":"4bf0467b-72a9-4ab5-a525-2385561f9da4","Type":"ContainerStarted","Data":"a52ce9cdc990196e5616d779d26fc22cb5b4aac19bf014e45b2f6b8e1d85dc4f"} Jun 15 02:33:02.560739 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:02.560680 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-4m5dz" Jun 15 02:33:02.702194 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:02.702162 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-6b8d965dd5-4m5dz"] Jun 15 02:33:02.705263 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:33:02.705235 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfbbc525_96a9_4c20_b125_742b83a9db41.slice/crio-e648b60f6c8a009e3429c5dced36ed37c21626190c9e9ae9f056a58288d98357 WatchSource:0}: Error finding container e648b60f6c8a009e3429c5dced36ed37c21626190c9e9ae9f056a58288d98357: Status 404 returned error can't find the container with id e648b60f6c8a009e3429c5dced36ed37c21626190c9e9ae9f056a58288d98357 Jun 15 02:33:02.736194 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:02.736170 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 15 02:33:02.741212 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:02.740971 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 15 02:33:02.743252 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:02.743230 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jun 15 02:33:02.743336 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:02.743231 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jun 15 02:33:02.743589 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:02.743569 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-8t6dp\"" Jun 15 02:33:02.743740 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:02.743592 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jun 15 02:33:02.743740 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:02.743735 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jun 15 02:33:02.743877 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:02.743606 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jun 15 02:33:02.743930 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:02.743627 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jun 15 02:33:02.743980 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:02.743933 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jun 15 02:33:02.744054 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:02.744037 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jun 15 02:33:02.744154 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:02.744136 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jun 15 02:33:02.756435 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:02.756392 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 15 02:33:02.779362 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:02.778848 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/eb38ec6a-eee4-43fe-b3e1-51aee1ca310b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"eb38ec6a-eee4-43fe-b3e1-51aee1ca310b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 02:33:02.779362 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:02.778887 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tcrb\" (UniqueName: \"kubernetes.io/projected/eb38ec6a-eee4-43fe-b3e1-51aee1ca310b-kube-api-access-4tcrb\") pod \"alertmanager-main-0\" (UID: \"eb38ec6a-eee4-43fe-b3e1-51aee1ca310b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 02:33:02.779362 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:02.778922 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb38ec6a-eee4-43fe-b3e1-51aee1ca310b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"eb38ec6a-eee4-43fe-b3e1-51aee1ca310b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 02:33:02.779362 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:02.778955 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/eb38ec6a-eee4-43fe-b3e1-51aee1ca310b-config-volume\") pod \"alertmanager-main-0\" (UID: \"eb38ec6a-eee4-43fe-b3e1-51aee1ca310b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 02:33:02.779362 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:02.778978 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/eb38ec6a-eee4-43fe-b3e1-51aee1ca310b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"eb38ec6a-eee4-43fe-b3e1-51aee1ca310b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 02:33:02.779362 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:02.779001 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/eb38ec6a-eee4-43fe-b3e1-51aee1ca310b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"eb38ec6a-eee4-43fe-b3e1-51aee1ca310b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 02:33:02.779362 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:02.779065 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/eb38ec6a-eee4-43fe-b3e1-51aee1ca310b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"eb38ec6a-eee4-43fe-b3e1-51aee1ca310b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 02:33:02.779362 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:02.779097 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/eb38ec6a-eee4-43fe-b3e1-51aee1ca310b-config-out\") pod \"alertmanager-main-0\" (UID: \"eb38ec6a-eee4-43fe-b3e1-51aee1ca310b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 02:33:02.779362 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:02.779120 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/eb38ec6a-eee4-43fe-b3e1-51aee1ca310b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"eb38ec6a-eee4-43fe-b3e1-51aee1ca310b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 02:33:02.779362 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:02.779141 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/eb38ec6a-eee4-43fe-b3e1-51aee1ca310b-web-config\") pod \"alertmanager-main-0\" (UID: \"eb38ec6a-eee4-43fe-b3e1-51aee1ca310b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 02:33:02.779362 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:02.779172 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/eb38ec6a-eee4-43fe-b3e1-51aee1ca310b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"eb38ec6a-eee4-43fe-b3e1-51aee1ca310b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 02:33:02.779362 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:02.779206 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/eb38ec6a-eee4-43fe-b3e1-51aee1ca310b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"eb38ec6a-eee4-43fe-b3e1-51aee1ca310b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 02:33:02.779362 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:02.779228 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eb38ec6a-eee4-43fe-b3e1-51aee1ca310b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"eb38ec6a-eee4-43fe-b3e1-51aee1ca310b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 02:33:02.880074 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:02.880049 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/eb38ec6a-eee4-43fe-b3e1-51aee1ca310b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"eb38ec6a-eee4-43fe-b3e1-51aee1ca310b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 02:33:02.880185 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:02.880083 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/eb38ec6a-eee4-43fe-b3e1-51aee1ca310b-config-out\") pod \"alertmanager-main-0\" (UID: \"eb38ec6a-eee4-43fe-b3e1-51aee1ca310b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 02:33:02.880185 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:02.880109 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/eb38ec6a-eee4-43fe-b3e1-51aee1ca310b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"eb38ec6a-eee4-43fe-b3e1-51aee1ca310b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 02:33:02.880185 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:02.880131 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/eb38ec6a-eee4-43fe-b3e1-51aee1ca310b-web-config\") pod \"alertmanager-main-0\" (UID: \"eb38ec6a-eee4-43fe-b3e1-51aee1ca310b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 02:33:02.880185 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:02.880162 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/eb38ec6a-eee4-43fe-b3e1-51aee1ca310b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"eb38ec6a-eee4-43fe-b3e1-51aee1ca310b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 02:33:02.880386 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:02.880194 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/eb38ec6a-eee4-43fe-b3e1-51aee1ca310b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"eb38ec6a-eee4-43fe-b3e1-51aee1ca310b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 02:33:02.880386 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:02.880216 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eb38ec6a-eee4-43fe-b3e1-51aee1ca310b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"eb38ec6a-eee4-43fe-b3e1-51aee1ca310b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 02:33:02.880386 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:02.880251 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/eb38ec6a-eee4-43fe-b3e1-51aee1ca310b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"eb38ec6a-eee4-43fe-b3e1-51aee1ca310b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 02:33:02.880386 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:02.880273 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tcrb\" (UniqueName: \"kubernetes.io/projected/eb38ec6a-eee4-43fe-b3e1-51aee1ca310b-kube-api-access-4tcrb\") pod \"alertmanager-main-0\" (UID: \"eb38ec6a-eee4-43fe-b3e1-51aee1ca310b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 02:33:02.880386 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:33:02.880298 2579 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jun 15 02:33:02.880386 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:02.880306 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb38ec6a-eee4-43fe-b3e1-51aee1ca310b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"eb38ec6a-eee4-43fe-b3e1-51aee1ca310b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 02:33:02.880386 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:02.880343 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/eb38ec6a-eee4-43fe-b3e1-51aee1ca310b-config-volume\") pod \"alertmanager-main-0\" (UID: \"eb38ec6a-eee4-43fe-b3e1-51aee1ca310b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 02:33:02.880386 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:02.880372 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/eb38ec6a-eee4-43fe-b3e1-51aee1ca310b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"eb38ec6a-eee4-43fe-b3e1-51aee1ca310b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 02:33:02.880806 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:02.880397 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/eb38ec6a-eee4-43fe-b3e1-51aee1ca310b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"eb38ec6a-eee4-43fe-b3e1-51aee1ca310b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 02:33:02.880806 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:33:02.880583 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb38ec6a-eee4-43fe-b3e1-51aee1ca310b-secret-alertmanager-main-tls podName:eb38ec6a-eee4-43fe-b3e1-51aee1ca310b nodeName:}" failed. No retries permitted until 2026-06-15 02:33:03.380553714 +0000 UTC m=+276.248338657 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/eb38ec6a-eee4-43fe-b3e1-51aee1ca310b-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "eb38ec6a-eee4-43fe-b3e1-51aee1ca310b") : secret "alertmanager-main-tls" not found Jun 15 02:33:02.880806 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:02.880615 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/eb38ec6a-eee4-43fe-b3e1-51aee1ca310b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"eb38ec6a-eee4-43fe-b3e1-51aee1ca310b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 02:33:02.881906 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:02.881532 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eb38ec6a-eee4-43fe-b3e1-51aee1ca310b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"eb38ec6a-eee4-43fe-b3e1-51aee1ca310b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 02:33:02.883184 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:02.882952 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb38ec6a-eee4-43fe-b3e1-51aee1ca310b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"eb38ec6a-eee4-43fe-b3e1-51aee1ca310b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 02:33:02.883184 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:02.883092 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/eb38ec6a-eee4-43fe-b3e1-51aee1ca310b-web-config\") pod \"alertmanager-main-0\" (UID: \"eb38ec6a-eee4-43fe-b3e1-51aee1ca310b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 02:33:02.883184 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:02.883111 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/eb38ec6a-eee4-43fe-b3e1-51aee1ca310b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"eb38ec6a-eee4-43fe-b3e1-51aee1ca310b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 02:33:02.883184 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:02.883134 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/eb38ec6a-eee4-43fe-b3e1-51aee1ca310b-config-out\") pod \"alertmanager-main-0\" (UID: \"eb38ec6a-eee4-43fe-b3e1-51aee1ca310b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 02:33:02.883505 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:02.883485 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/eb38ec6a-eee4-43fe-b3e1-51aee1ca310b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"eb38ec6a-eee4-43fe-b3e1-51aee1ca310b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 02:33:02.883880 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:02.883852 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/eb38ec6a-eee4-43fe-b3e1-51aee1ca310b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"eb38ec6a-eee4-43fe-b3e1-51aee1ca310b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 02:33:02.884112 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:02.884079 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/eb38ec6a-eee4-43fe-b3e1-51aee1ca310b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"eb38ec6a-eee4-43fe-b3e1-51aee1ca310b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 02:33:02.884199 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:02.884181 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/eb38ec6a-eee4-43fe-b3e1-51aee1ca310b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"eb38ec6a-eee4-43fe-b3e1-51aee1ca310b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 02:33:02.884597 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:02.884580 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/eb38ec6a-eee4-43fe-b3e1-51aee1ca310b-config-volume\") pod \"alertmanager-main-0\" (UID: \"eb38ec6a-eee4-43fe-b3e1-51aee1ca310b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 02:33:02.889313 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:02.889295 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tcrb\" (UniqueName: \"kubernetes.io/projected/eb38ec6a-eee4-43fe-b3e1-51aee1ca310b-kube-api-access-4tcrb\") pod \"alertmanager-main-0\" (UID: \"eb38ec6a-eee4-43fe-b3e1-51aee1ca310b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 02:33:03.384137 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:03.384110 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/eb38ec6a-eee4-43fe-b3e1-51aee1ca310b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"eb38ec6a-eee4-43fe-b3e1-51aee1ca310b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 02:33:03.386306 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:03.386286 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/eb38ec6a-eee4-43fe-b3e1-51aee1ca310b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"eb38ec6a-eee4-43fe-b3e1-51aee1ca310b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 02:33:03.404026 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:03.403997 2579 generic.go:356] "Generic (PLEG): container finished" podID="4bf0467b-72a9-4ab5-a525-2385561f9da4" containerID="b56df7bbef14890a9b1a43cb16126682e3b75c2900e576fca195c01173b3aeff" exitCode=0 Jun 15 02:33:03.404138 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:03.404100 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l6v8k" event={"ID":"4bf0467b-72a9-4ab5-a525-2385561f9da4","Type":"ContainerDied","Data":"b56df7bbef14890a9b1a43cb16126682e3b75c2900e576fca195c01173b3aeff"} Jun 15 02:33:03.406142 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:03.406116 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-4m5dz" event={"ID":"cfbbc525-96a9-4c20-b125-742b83a9db41","Type":"ContainerStarted","Data":"34590793871a36ae1a4984c10ca9edad95d7e5a42560d894bc98da468ab07664"} Jun 15 02:33:03.406228 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:03.406172 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-4m5dz" event={"ID":"cfbbc525-96a9-4c20-b125-742b83a9db41","Type":"ContainerStarted","Data":"a93b8f80822840d73c96c8f7fa2f3f795e884cd9e7f3e60034de3d92ac5e22bc"} Jun 15 02:33:03.406228 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:03.406202 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-4m5dz" event={"ID":"cfbbc525-96a9-4c20-b125-742b83a9db41","Type":"ContainerStarted","Data":"e648b60f6c8a009e3429c5dced36ed37c21626190c9e9ae9f056a58288d98357"} Jun 15 02:33:03.657722 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:03.657686 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 15 02:33:03.931597 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:03.931569 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 15 02:33:03.932753 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:33:03.932719 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb38ec6a_eee4_43fe_b3e1_51aee1ca310b.slice/crio-a01791088265a75df2fe845c8e52d5978f7bcd6a2d954953dca3da96aaf22c77 WatchSource:0}: Error finding container a01791088265a75df2fe845c8e52d5978f7bcd6a2d954953dca3da96aaf22c77: Status 404 returned error can't find the container with id a01791088265a75df2fe845c8e52d5978f7bcd6a2d954953dca3da96aaf22c77 Jun 15 02:33:04.412986 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:04.412944 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-4m5dz" event={"ID":"cfbbc525-96a9-4c20-b125-742b83a9db41","Type":"ContainerStarted","Data":"6c03615479b097d141134c8e20f2b5693dcd7951307633adb37d3e22bee45e06"} Jun 15 02:33:04.414107 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:04.414078 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"eb38ec6a-eee4-43fe-b3e1-51aee1ca310b","Type":"ContainerStarted","Data":"a01791088265a75df2fe845c8e52d5978f7bcd6a2d954953dca3da96aaf22c77"} Jun 15 02:33:04.415923 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:04.415895 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l6v8k" event={"ID":"4bf0467b-72a9-4ab5-a525-2385561f9da4","Type":"ContainerStarted","Data":"11c765e0a74d12153a851ab5f62e0f5ffe74764599ea16b3ce01c8a7f495b9e3"} Jun 15 02:33:04.416026 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:04.415937 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l6v8k" event={"ID":"4bf0467b-72a9-4ab5-a525-2385561f9da4","Type":"ContainerStarted","Data":"6f59a7fff3cdd5a8a48a753ee1894670dcadc124fa8b452380bf770f8dcd8d10"} Jun 15 02:33:04.429911 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:04.429866 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-4m5dz" podStartSLOduration=2.419302826 podStartE2EDuration="3.429853545s" totalImagesPullingTime="1.010550719s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 02:33:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 02:33:04.428789393 +0000 UTC m=+277.296574138" watchObservedRunningTime="2026-06-15 02:33:04.429853545 +0000 UTC m=+277.297638290" Jun 15 02:33:04.449951 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:04.449912 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-l6v8k" podStartSLOduration=2.208272939 podStartE2EDuration="3.449901124s" totalImagesPullingTime="1.179734497s" totalInitContainerRuntime="61.893688ms" isStatefulPod=true podCreationTimestamp="2026-06-15 02:33:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 02:33:04.448472385 +0000 UTC m=+277.316257132" watchObservedRunningTime="2026-06-15 02:33:04.449901124 +0000 UTC m=+277.317685859" Jun 15 02:33:05.421040 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:05.421013 2579 generic.go:356] "Generic (PLEG): container finished" podID="eb38ec6a-eee4-43fe-b3e1-51aee1ca310b" containerID="abae9a1c35a0fb5b0124281dba81ceb236a7c7c5bad8141ed12585e0deea18fc" exitCode=0 Jun 15 02:33:05.421307 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:05.421096 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"eb38ec6a-eee4-43fe-b3e1-51aee1ca310b","Type":"ContainerDied","Data":"abae9a1c35a0fb5b0124281dba81ceb236a7c7c5bad8141ed12585e0deea18fc"} Jun 15 02:33:06.573727 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:06.573695 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-b897df459-svdtj"] Jun 15 02:33:06.616111 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:06.616083 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-76cfc46c9c-fcttn"] Jun 15 02:33:06.619380 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:06.619359 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76cfc46c9c-fcttn" Jun 15 02:33:06.626709 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:06.626685 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jun 15 02:33:06.631106 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:06.631083 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76cfc46c9c-fcttn"] Jun 15 02:33:06.711960 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:06.711930 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v25ft\" (UniqueName: \"kubernetes.io/projected/d748b607-99fd-4790-8b6d-2942f4d135aa-kube-api-access-v25ft\") pod \"console-76cfc46c9c-fcttn\" (UID: \"d748b607-99fd-4790-8b6d-2942f4d135aa\") " pod="openshift-console/console-76cfc46c9c-fcttn" Jun 15 02:33:06.711960 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:06.711963 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d748b607-99fd-4790-8b6d-2942f4d135aa-console-oauth-config\") pod \"console-76cfc46c9c-fcttn\" (UID: \"d748b607-99fd-4790-8b6d-2942f4d135aa\") " pod="openshift-console/console-76cfc46c9c-fcttn" Jun 15 02:33:06.712185 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:06.711994 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d748b607-99fd-4790-8b6d-2942f4d135aa-trusted-ca-bundle\") pod \"console-76cfc46c9c-fcttn\" (UID: \"d748b607-99fd-4790-8b6d-2942f4d135aa\") " pod="openshift-console/console-76cfc46c9c-fcttn" Jun 15 02:33:06.712185 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:06.712091 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d748b607-99fd-4790-8b6d-2942f4d135aa-console-config\") pod \"console-76cfc46c9c-fcttn\" (UID: \"d748b607-99fd-4790-8b6d-2942f4d135aa\") " pod="openshift-console/console-76cfc46c9c-fcttn" Jun 15 02:33:06.712185 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:06.712154 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d748b607-99fd-4790-8b6d-2942f4d135aa-oauth-serving-cert\") pod \"console-76cfc46c9c-fcttn\" (UID: \"d748b607-99fd-4790-8b6d-2942f4d135aa\") " pod="openshift-console/console-76cfc46c9c-fcttn" Jun 15 02:33:06.712331 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:06.712206 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d748b607-99fd-4790-8b6d-2942f4d135aa-console-serving-cert\") pod \"console-76cfc46c9c-fcttn\" (UID: \"d748b607-99fd-4790-8b6d-2942f4d135aa\") " pod="openshift-console/console-76cfc46c9c-fcttn" Jun 15 02:33:06.712331 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:06.712245 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d748b607-99fd-4790-8b6d-2942f4d135aa-service-ca\") pod \"console-76cfc46c9c-fcttn\" (UID: \"d748b607-99fd-4790-8b6d-2942f4d135aa\") " pod="openshift-console/console-76cfc46c9c-fcttn" Jun 15 02:33:06.813273 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:06.813243 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d748b607-99fd-4790-8b6d-2942f4d135aa-console-serving-cert\") pod \"console-76cfc46c9c-fcttn\" (UID: \"d748b607-99fd-4790-8b6d-2942f4d135aa\") " pod="openshift-console/console-76cfc46c9c-fcttn" Jun 15 02:33:06.813455 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:06.813289 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d748b607-99fd-4790-8b6d-2942f4d135aa-service-ca\") pod \"console-76cfc46c9c-fcttn\" (UID: \"d748b607-99fd-4790-8b6d-2942f4d135aa\") " pod="openshift-console/console-76cfc46c9c-fcttn" Jun 15 02:33:06.813455 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:06.813314 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v25ft\" (UniqueName: \"kubernetes.io/projected/d748b607-99fd-4790-8b6d-2942f4d135aa-kube-api-access-v25ft\") pod \"console-76cfc46c9c-fcttn\" (UID: \"d748b607-99fd-4790-8b6d-2942f4d135aa\") " pod="openshift-console/console-76cfc46c9c-fcttn" Jun 15 02:33:06.813455 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:06.813369 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d748b607-99fd-4790-8b6d-2942f4d135aa-console-oauth-config\") pod \"console-76cfc46c9c-fcttn\" (UID: \"d748b607-99fd-4790-8b6d-2942f4d135aa\") " pod="openshift-console/console-76cfc46c9c-fcttn" Jun 15 02:33:06.813455 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:06.813426 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d748b607-99fd-4790-8b6d-2942f4d135aa-trusted-ca-bundle\") pod \"console-76cfc46c9c-fcttn\" (UID: \"d748b607-99fd-4790-8b6d-2942f4d135aa\") " pod="openshift-console/console-76cfc46c9c-fcttn" Jun 15 02:33:06.813686 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:06.813497 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d748b607-99fd-4790-8b6d-2942f4d135aa-console-config\") pod \"console-76cfc46c9c-fcttn\" (UID: \"d748b607-99fd-4790-8b6d-2942f4d135aa\") " pod="openshift-console/console-76cfc46c9c-fcttn" Jun 15 02:33:06.813686 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:06.813560 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d748b607-99fd-4790-8b6d-2942f4d135aa-oauth-serving-cert\") pod \"console-76cfc46c9c-fcttn\" (UID: \"d748b607-99fd-4790-8b6d-2942f4d135aa\") " pod="openshift-console/console-76cfc46c9c-fcttn" Jun 15 02:33:06.814203 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:06.814169 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d748b607-99fd-4790-8b6d-2942f4d135aa-oauth-serving-cert\") pod \"console-76cfc46c9c-fcttn\" (UID: \"d748b607-99fd-4790-8b6d-2942f4d135aa\") " pod="openshift-console/console-76cfc46c9c-fcttn" Jun 15 02:33:06.814203 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:06.814162 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d748b607-99fd-4790-8b6d-2942f4d135aa-service-ca\") pod \"console-76cfc46c9c-fcttn\" (UID: \"d748b607-99fd-4790-8b6d-2942f4d135aa\") " pod="openshift-console/console-76cfc46c9c-fcttn" Jun 15 02:33:06.814356 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:06.814337 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d748b607-99fd-4790-8b6d-2942f4d135aa-console-config\") pod \"console-76cfc46c9c-fcttn\" (UID: \"d748b607-99fd-4790-8b6d-2942f4d135aa\") " pod="openshift-console/console-76cfc46c9c-fcttn" Jun 15 02:33:06.815024 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:06.814996 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d748b607-99fd-4790-8b6d-2942f4d135aa-trusted-ca-bundle\") pod \"console-76cfc46c9c-fcttn\" (UID: \"d748b607-99fd-4790-8b6d-2942f4d135aa\") " pod="openshift-console/console-76cfc46c9c-fcttn" Jun 15 02:33:06.816394 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:06.816358 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d748b607-99fd-4790-8b6d-2942f4d135aa-console-oauth-config\") pod \"console-76cfc46c9c-fcttn\" (UID: \"d748b607-99fd-4790-8b6d-2942f4d135aa\") " pod="openshift-console/console-76cfc46c9c-fcttn" Jun 15 02:33:06.816663 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:06.816619 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d748b607-99fd-4790-8b6d-2942f4d135aa-console-serving-cert\") pod \"console-76cfc46c9c-fcttn\" (UID: \"d748b607-99fd-4790-8b6d-2942f4d135aa\") " pod="openshift-console/console-76cfc46c9c-fcttn" Jun 15 02:33:06.821346 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:06.821328 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v25ft\" (UniqueName: \"kubernetes.io/projected/d748b607-99fd-4790-8b6d-2942f4d135aa-kube-api-access-v25ft\") pod \"console-76cfc46c9c-fcttn\" (UID: \"d748b607-99fd-4790-8b6d-2942f4d135aa\") " pod="openshift-console/console-76cfc46c9c-fcttn" Jun 15 02:33:06.931907 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:06.931835 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76cfc46c9c-fcttn" Jun 15 02:33:07.241855 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:07.241336 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76cfc46c9c-fcttn"] Jun 15 02:33:07.246400 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:33:07.246369 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd748b607_99fd_4790_8b6d_2942f4d135aa.slice/crio-47ba4bb7080a3ce808924ac473da02e98b7f6d6c0d5a30f4342fa2b3bfed68f9 WatchSource:0}: Error finding container 47ba4bb7080a3ce808924ac473da02e98b7f6d6c0d5a30f4342fa2b3bfed68f9: Status 404 returned error can't find the container with id 47ba4bb7080a3ce808924ac473da02e98b7f6d6c0d5a30f4342fa2b3bfed68f9 Jun 15 02:33:07.429833 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:07.429797 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76cfc46c9c-fcttn" event={"ID":"d748b607-99fd-4790-8b6d-2942f4d135aa","Type":"ContainerStarted","Data":"9dc5674043ba886097106fbc3eb70d48be5a933a1e3ed882af8e9bc910c9b2b8"} Jun 15 02:33:07.429946 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:07.429848 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76cfc46c9c-fcttn" event={"ID":"d748b607-99fd-4790-8b6d-2942f4d135aa","Type":"ContainerStarted","Data":"47ba4bb7080a3ce808924ac473da02e98b7f6d6c0d5a30f4342fa2b3bfed68f9"} Jun 15 02:33:07.432990 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:07.432965 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"eb38ec6a-eee4-43fe-b3e1-51aee1ca310b","Type":"ContainerStarted","Data":"cb42d3be3cc32e13e7888701153dc43fafd636fb4ebfc01afba4486331983481"} Jun 15 02:33:07.433071 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:07.433053 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"eb38ec6a-eee4-43fe-b3e1-51aee1ca310b","Type":"ContainerStarted","Data":"6022bf6dfc6104b10cb894b0c0e2e4fed0cdd1b31821c066d73a535590964a29"} Jun 15 02:33:07.433144 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:07.433098 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"eb38ec6a-eee4-43fe-b3e1-51aee1ca310b","Type":"ContainerStarted","Data":"65ab9a79cd1ffb223aab8744bb22e1452d89a8b5f490c4b585cdd1ba449c47a4"} Jun 15 02:33:07.433272 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:07.433158 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"eb38ec6a-eee4-43fe-b3e1-51aee1ca310b","Type":"ContainerStarted","Data":"27b3626a0df57fef12ec54e7741303b982390ef3b435e45d438a43e6916f7566"} Jun 15 02:33:07.447396 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:07.447322 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-76cfc46c9c-fcttn" podStartSLOduration=1.447308938 podStartE2EDuration="1.447308938s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 02:33:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 02:33:07.445786488 +0000 UTC m=+280.313571233" watchObservedRunningTime="2026-06-15 02:33:07.447308938 +0000 UTC m=+280.315093682" Jun 15 02:33:08.440496 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:08.440467 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"eb38ec6a-eee4-43fe-b3e1-51aee1ca310b","Type":"ContainerStarted","Data":"afb4f43b716fc0dbc4c7ad81fcba0820f6e79542080271e83e723b9ea676411b"} Jun 15 02:33:08.440803 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:08.440529 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"eb38ec6a-eee4-43fe-b3e1-51aee1ca310b","Type":"ContainerStarted","Data":"9468fb876cef4d69a954518749f501fd1a6e8ca6513fb1791b6daad01a91e023"} Jun 15 02:33:08.463734 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:08.463690 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.448834265 podStartE2EDuration="6.463674155s" totalImagesPullingTime="3.999747041s" totalInitContainerRuntime="15.092849ms" isStatefulPod=false podCreationTimestamp="2026-06-15 02:33:02 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 02:33:08.461692096 +0000 UTC m=+281.329476841" watchObservedRunningTime="2026-06-15 02:33:08.463674155 +0000 UTC m=+281.331458900" Jun 15 02:33:09.173996 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:09.173957 2579 patch_prober.go:28] interesting pod/image-registry-69d957bb74-n8jmq container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 15 02:33:09.174185 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:09.174011 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-69d957bb74-n8jmq" podUID="786694ac-916e-4cbd-a6fb-b2fa97ce7bb9" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 15 02:33:11.024527 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:33:11.024488 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-8wpj7" podUID="1c70b33b-a4a3-4e1e-9693-41d77a5217fd" Jun 15 02:33:11.450301 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:11.450227 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8wpj7" Jun 15 02:33:14.188786 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:14.188742 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-69d957bb74-n8jmq" podUID="786694ac-916e-4cbd-a6fb-b2fa97ce7bb9" containerName="registry" containerID="cri-o://382146b576089217cca838079767866a1d46127fb0b846d1481b14939082f1e8" gracePeriod=30 Jun 15 02:33:14.462896 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:14.462838 2579 generic.go:356] "Generic (PLEG): container finished" podID="786694ac-916e-4cbd-a6fb-b2fa97ce7bb9" containerID="382146b576089217cca838079767866a1d46127fb0b846d1481b14939082f1e8" exitCode=0 Jun 15 02:33:14.462896 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:14.462883 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69d957bb74-n8jmq" event={"ID":"786694ac-916e-4cbd-a6fb-b2fa97ce7bb9","Type":"ContainerDied","Data":"382146b576089217cca838079767866a1d46127fb0b846d1481b14939082f1e8"} Jun 15 02:33:14.477212 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:14.477189 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1c70b33b-a4a3-4e1e-9693-41d77a5217fd-metrics-tls\") pod \"dns-default-8wpj7\" (UID: \"1c70b33b-a4a3-4e1e-9693-41d77a5217fd\") " pod="openshift-dns/dns-default-8wpj7" Jun 15 02:33:14.479356 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:14.479330 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1c70b33b-a4a3-4e1e-9693-41d77a5217fd-metrics-tls\") pod \"dns-default-8wpj7\" (UID: \"1c70b33b-a4a3-4e1e-9693-41d77a5217fd\") " pod="openshift-dns/dns-default-8wpj7" Jun 15 02:33:14.753590 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:14.753564 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vrtgk\"" Jun 15 02:33:14.762305 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:14.762285 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8wpj7" Jun 15 02:33:14.966933 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:14.966904 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69d957bb74-n8jmq" Jun 15 02:33:15.082269 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:15.082211 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-registry-certificates\") pod \"786694ac-916e-4cbd-a6fb-b2fa97ce7bb9\" (UID: \"786694ac-916e-4cbd-a6fb-b2fa97ce7bb9\") " Jun 15 02:33:15.082269 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:15.082241 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-bound-sa-token\") pod \"786694ac-916e-4cbd-a6fb-b2fa97ce7bb9\" (UID: \"786694ac-916e-4cbd-a6fb-b2fa97ce7bb9\") " Jun 15 02:33:15.082432 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:15.082270 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-registry-tls\") pod \"786694ac-916e-4cbd-a6fb-b2fa97ce7bb9\" (UID: \"786694ac-916e-4cbd-a6fb-b2fa97ce7bb9\") " Jun 15 02:33:15.082432 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:15.082306 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-image-registry-private-configuration\") pod \"786694ac-916e-4cbd-a6fb-b2fa97ce7bb9\" (UID: \"786694ac-916e-4cbd-a6fb-b2fa97ce7bb9\") " Jun 15 02:33:15.082432 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:15.082349 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-installation-pull-secrets\") pod \"786694ac-916e-4cbd-a6fb-b2fa97ce7bb9\" (UID: \"786694ac-916e-4cbd-a6fb-b2fa97ce7bb9\") " Jun 15 02:33:15.082432 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:15.082416 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nrjf7\" (UniqueName: \"kubernetes.io/projected/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-kube-api-access-nrjf7\") pod \"786694ac-916e-4cbd-a6fb-b2fa97ce7bb9\" (UID: \"786694ac-916e-4cbd-a6fb-b2fa97ce7bb9\") " Jun 15 02:33:15.082675 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:15.082456 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-ca-trust-extracted\") pod \"786694ac-916e-4cbd-a6fb-b2fa97ce7bb9\" (UID: \"786694ac-916e-4cbd-a6fb-b2fa97ce7bb9\") " Jun 15 02:33:15.082675 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:15.082490 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-trusted-ca\") pod \"786694ac-916e-4cbd-a6fb-b2fa97ce7bb9\" (UID: \"786694ac-916e-4cbd-a6fb-b2fa97ce7bb9\") " Jun 15 02:33:15.083043 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:15.082811 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "786694ac-916e-4cbd-a6fb-b2fa97ce7bb9" (UID: "786694ac-916e-4cbd-a6fb-b2fa97ce7bb9"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 02:33:15.083043 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:15.082981 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "786694ac-916e-4cbd-a6fb-b2fa97ce7bb9" (UID: "786694ac-916e-4cbd-a6fb-b2fa97ce7bb9"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 02:33:15.084739 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:15.084708 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "786694ac-916e-4cbd-a6fb-b2fa97ce7bb9" (UID: "786694ac-916e-4cbd-a6fb-b2fa97ce7bb9"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 15 02:33:15.084739 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:15.084728 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "786694ac-916e-4cbd-a6fb-b2fa97ce7bb9" (UID: "786694ac-916e-4cbd-a6fb-b2fa97ce7bb9"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 15 02:33:15.084916 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:15.084896 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "786694ac-916e-4cbd-a6fb-b2fa97ce7bb9" (UID: "786694ac-916e-4cbd-a6fb-b2fa97ce7bb9"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 02:33:15.084979 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:15.084910 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "786694ac-916e-4cbd-a6fb-b2fa97ce7bb9" (UID: "786694ac-916e-4cbd-a6fb-b2fa97ce7bb9"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 02:33:15.085170 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:15.085146 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-kube-api-access-nrjf7" (OuterVolumeSpecName: "kube-api-access-nrjf7") pod "786694ac-916e-4cbd-a6fb-b2fa97ce7bb9" (UID: "786694ac-916e-4cbd-a6fb-b2fa97ce7bb9"). InnerVolumeSpecName "kube-api-access-nrjf7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 02:33:15.091347 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:15.091315 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "786694ac-916e-4cbd-a6fb-b2fa97ce7bb9" (UID: "786694ac-916e-4cbd-a6fb-b2fa97ce7bb9"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 02:33:15.124862 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:15.124798 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8wpj7"] Jun 15 02:33:15.127589 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:33:15.127563 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c70b33b_a4a3_4e1e_9693_41d77a5217fd.slice/crio-d21a4314d0b2948dceb374d7c5c46fc492f7a74965191b052a1f840939a560dd WatchSource:0}: Error finding container d21a4314d0b2948dceb374d7c5c46fc492f7a74965191b052a1f840939a560dd: Status 404 returned error can't find the container with id d21a4314d0b2948dceb374d7c5c46fc492f7a74965191b052a1f840939a560dd Jun 15 02:33:15.183897 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:15.183875 2579 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-registry-tls\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:33:15.183897 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:15.183897 2579 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-image-registry-private-configuration\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:33:15.184009 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:15.183907 2579 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-installation-pull-secrets\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:33:15.184009 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:15.183918 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nrjf7\" (UniqueName: \"kubernetes.io/projected/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-kube-api-access-nrjf7\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:33:15.184009 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:15.183932 2579 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-ca-trust-extracted\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:33:15.184009 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:15.183941 2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-trusted-ca\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:33:15.184009 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:15.183949 2579 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-registry-certificates\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:33:15.184009 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:15.183957 2579 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9-bound-sa-token\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:33:15.473081 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:15.473022 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69d957bb74-n8jmq" Jun 15 02:33:15.473450 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:15.473022 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69d957bb74-n8jmq" event={"ID":"786694ac-916e-4cbd-a6fb-b2fa97ce7bb9","Type":"ContainerDied","Data":"67bb87ff9a4697ceab17e7dafd2a36ff2159d396f201a1a4aa6e7baf8cfd6a38"} Jun 15 02:33:15.473450 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:15.473184 2579 scope.go:117] "RemoveContainer" containerID="382146b576089217cca838079767866a1d46127fb0b846d1481b14939082f1e8" Jun 15 02:33:15.474443 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:15.474243 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8wpj7" event={"ID":"1c70b33b-a4a3-4e1e-9693-41d77a5217fd","Type":"ContainerStarted","Data":"d21a4314d0b2948dceb374d7c5c46fc492f7a74965191b052a1f840939a560dd"} Jun 15 02:33:15.492710 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:15.492691 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-69d957bb74-n8jmq"] Jun 15 02:33:15.498336 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:15.498317 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-69d957bb74-n8jmq"] Jun 15 02:33:16.932566 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:16.932530 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-76cfc46c9c-fcttn" Jun 15 02:33:16.932996 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:16.932611 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-76cfc46c9c-fcttn" Jun 15 02:33:16.934160 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:16.934136 2579 patch_prober.go:28] interesting pod/console-76cfc46c9c-fcttn container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.132.0.18:8443/health\": dial tcp 10.132.0.18:8443: connect: connection refused" start-of-body= Jun 15 02:33:16.934284 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:16.934181 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-76cfc46c9c-fcttn" podUID="d748b607-99fd-4790-8b6d-2942f4d135aa" containerName="console" probeResult="failure" output="Get \"https://10.132.0.18:8443/health\": dial tcp 10.132.0.18:8443: connect: connection refused" Jun 15 02:33:17.484944 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:17.484917 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8wpj7" event={"ID":"1c70b33b-a4a3-4e1e-9693-41d77a5217fd","Type":"ContainerStarted","Data":"a313d61c3e305c3039b68d9cce32db5e554da41f21405838658def2b86600026"} Jun 15 02:33:17.485115 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:17.484963 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8wpj7" event={"ID":"1c70b33b-a4a3-4e1e-9693-41d77a5217fd","Type":"ContainerStarted","Data":"abd08346c5c0af6346166c3b5df177f186b8d6f83f3391da33da1c0f35551ae5"} Jun 15 02:33:17.485115 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:17.485061 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-8wpj7" Jun 15 02:33:17.492973 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:17.492951 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="786694ac-916e-4cbd-a6fb-b2fa97ce7bb9" path="/var/lib/kubelet/pods/786694ac-916e-4cbd-a6fb-b2fa97ce7bb9/volumes" Jun 15 02:33:17.501262 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:17.501220 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-8wpj7" podStartSLOduration=251.450488512 podStartE2EDuration="4m13.501208419s" totalImagesPullingTime="2.050719907s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 02:29:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 02:33:17.499958926 +0000 UTC m=+290.367743682" watchObservedRunningTime="2026-06-15 02:33:17.501208419 +0000 UTC m=+290.368993163" Jun 15 02:33:26.933076 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:26.933035 2579 patch_prober.go:28] interesting pod/console-76cfc46c9c-fcttn container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.132.0.18:8443/health\": dial tcp 10.132.0.18:8443: connect: connection refused" start-of-body= Jun 15 02:33:26.933453 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:26.933091 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-76cfc46c9c-fcttn" podUID="d748b607-99fd-4790-8b6d-2942f4d135aa" containerName="console" probeResult="failure" output="Get \"https://10.132.0.18:8443/health\": dial tcp 10.132.0.18:8443: connect: connection refused" Jun 15 02:33:27.443974 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:27.443952 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-w4m86_fb6051d8-9291-4309-9047-9e8d0b5d2e48/console-operator/1.log" Jun 15 02:33:27.444540 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:27.444514 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-w4m86_fb6051d8-9291-4309-9047-9e8d0b5d2e48/console-operator/1.log" Jun 15 02:33:27.455431 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:27.455413 2579 kubelet.go:1648] "Image garbage collection succeeded" Jun 15 02:33:27.491747 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:27.491616 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-8wpj7" Jun 15 02:33:31.593804 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:31.593761 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-b897df459-svdtj" podUID="0580edc3-3dea-43e9-9839-d5600831c982" containerName="console" containerID="cri-o://90d60e1540e4b636003ced235a95e6f2803158d4d62076f4ddac7a9ec371a1e4" gracePeriod=15 Jun 15 02:33:31.834912 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:31.834891 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b897df459-svdtj" Jun 15 02:33:31.906426 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:31.906361 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r2rlg\" (UniqueName: \"kubernetes.io/projected/0580edc3-3dea-43e9-9839-d5600831c982-kube-api-access-r2rlg\") pod \"0580edc3-3dea-43e9-9839-d5600831c982\" (UID: \"0580edc3-3dea-43e9-9839-d5600831c982\") " Jun 15 02:33:31.906426 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:31.906415 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0580edc3-3dea-43e9-9839-d5600831c982-oauth-serving-cert\") pod \"0580edc3-3dea-43e9-9839-d5600831c982\" (UID: \"0580edc3-3dea-43e9-9839-d5600831c982\") " Jun 15 02:33:31.906654 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:31.906438 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0580edc3-3dea-43e9-9839-d5600831c982-console-serving-cert\") pod \"0580edc3-3dea-43e9-9839-d5600831c982\" (UID: \"0580edc3-3dea-43e9-9839-d5600831c982\") " Jun 15 02:33:31.906654 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:31.906457 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0580edc3-3dea-43e9-9839-d5600831c982-console-oauth-config\") pod \"0580edc3-3dea-43e9-9839-d5600831c982\" (UID: \"0580edc3-3dea-43e9-9839-d5600831c982\") " Jun 15 02:33:31.906654 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:31.906493 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0580edc3-3dea-43e9-9839-d5600831c982-console-config\") pod \"0580edc3-3dea-43e9-9839-d5600831c982\" (UID: \"0580edc3-3dea-43e9-9839-d5600831c982\") " Jun 15 02:33:31.906654 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:31.906529 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0580edc3-3dea-43e9-9839-d5600831c982-service-ca\") pod \"0580edc3-3dea-43e9-9839-d5600831c982\" (UID: \"0580edc3-3dea-43e9-9839-d5600831c982\") " Jun 15 02:33:31.906932 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:31.906908 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0580edc3-3dea-43e9-9839-d5600831c982-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0580edc3-3dea-43e9-9839-d5600831c982" (UID: "0580edc3-3dea-43e9-9839-d5600831c982"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 02:33:31.907006 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:31.906907 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0580edc3-3dea-43e9-9839-d5600831c982-console-config" (OuterVolumeSpecName: "console-config") pod "0580edc3-3dea-43e9-9839-d5600831c982" (UID: "0580edc3-3dea-43e9-9839-d5600831c982"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 02:33:31.907006 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:31.906980 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0580edc3-3dea-43e9-9839-d5600831c982-service-ca" (OuterVolumeSpecName: "service-ca") pod "0580edc3-3dea-43e9-9839-d5600831c982" (UID: "0580edc3-3dea-43e9-9839-d5600831c982"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 02:33:31.908612 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:31.908589 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0580edc3-3dea-43e9-9839-d5600831c982-kube-api-access-r2rlg" (OuterVolumeSpecName: "kube-api-access-r2rlg") pod "0580edc3-3dea-43e9-9839-d5600831c982" (UID: "0580edc3-3dea-43e9-9839-d5600831c982"). InnerVolumeSpecName "kube-api-access-r2rlg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 02:33:31.908747 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:31.908612 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0580edc3-3dea-43e9-9839-d5600831c982-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0580edc3-3dea-43e9-9839-d5600831c982" (UID: "0580edc3-3dea-43e9-9839-d5600831c982"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 15 02:33:31.908747 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:31.908655 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0580edc3-3dea-43e9-9839-d5600831c982-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0580edc3-3dea-43e9-9839-d5600831c982" (UID: "0580edc3-3dea-43e9-9839-d5600831c982"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 15 02:33:32.007515 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:32.007493 2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0580edc3-3dea-43e9-9839-d5600831c982-console-serving-cert\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:33:32.007515 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:32.007512 2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0580edc3-3dea-43e9-9839-d5600831c982-console-oauth-config\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:33:32.007664 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:32.007523 2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0580edc3-3dea-43e9-9839-d5600831c982-console-config\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:33:32.007664 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:32.007532 2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0580edc3-3dea-43e9-9839-d5600831c982-service-ca\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:33:32.007664 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:32.007541 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r2rlg\" (UniqueName: \"kubernetes.io/projected/0580edc3-3dea-43e9-9839-d5600831c982-kube-api-access-r2rlg\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:33:32.007664 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:32.007549 2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0580edc3-3dea-43e9-9839-d5600831c982-oauth-serving-cert\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:33:32.539139 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:32.539108 2579 generic.go:356] "Generic (PLEG): container finished" podID="0580edc3-3dea-43e9-9839-d5600831c982" containerID="90d60e1540e4b636003ced235a95e6f2803158d4d62076f4ddac7a9ec371a1e4" exitCode=0 Jun 15 02:33:32.539313 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:32.539162 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b897df459-svdtj" Jun 15 02:33:32.539313 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:32.539202 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b897df459-svdtj" event={"ID":"0580edc3-3dea-43e9-9839-d5600831c982","Type":"ContainerDied","Data":"90d60e1540e4b636003ced235a95e6f2803158d4d62076f4ddac7a9ec371a1e4"} Jun 15 02:33:32.539313 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:32.539265 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b897df459-svdtj" event={"ID":"0580edc3-3dea-43e9-9839-d5600831c982","Type":"ContainerDied","Data":"e763bc498109f5fb730c2c63e5d795a200e546621faa245b06de3ec767b04d6a"} Jun 15 02:33:32.539313 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:32.539297 2579 scope.go:117] "RemoveContainer" containerID="90d60e1540e4b636003ced235a95e6f2803158d4d62076f4ddac7a9ec371a1e4" Jun 15 02:33:32.548791 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:32.548772 2579 scope.go:117] "RemoveContainer" containerID="90d60e1540e4b636003ced235a95e6f2803158d4d62076f4ddac7a9ec371a1e4" Jun 15 02:33:32.549081 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:33:32.549046 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90d60e1540e4b636003ced235a95e6f2803158d4d62076f4ddac7a9ec371a1e4\": container with ID starting with 90d60e1540e4b636003ced235a95e6f2803158d4d62076f4ddac7a9ec371a1e4 not found: ID does not exist" containerID="90d60e1540e4b636003ced235a95e6f2803158d4d62076f4ddac7a9ec371a1e4" Jun 15 02:33:32.549167 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:32.549088 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90d60e1540e4b636003ced235a95e6f2803158d4d62076f4ddac7a9ec371a1e4"} err="failed to get container status \"90d60e1540e4b636003ced235a95e6f2803158d4d62076f4ddac7a9ec371a1e4\": rpc error: code = NotFound desc = could not find container \"90d60e1540e4b636003ced235a95e6f2803158d4d62076f4ddac7a9ec371a1e4\": container with ID starting with 90d60e1540e4b636003ced235a95e6f2803158d4d62076f4ddac7a9ec371a1e4 not found: ID does not exist" Jun 15 02:33:32.560406 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:32.560382 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-b897df459-svdtj"] Jun 15 02:33:32.564747 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:32.564725 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-b897df459-svdtj"] Jun 15 02:33:33.491890 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:33.491860 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0580edc3-3dea-43e9-9839-d5600831c982" path="/var/lib/kubelet/pods/0580edc3-3dea-43e9-9839-d5600831c982/volumes" Jun 15 02:33:36.936163 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:36.936135 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-76cfc46c9c-fcttn" Jun 15 02:33:36.939828 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:36.939801 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-76cfc46c9c-fcttn" Jun 15 02:33:48.272674 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:33:48.272627 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hz7mx_c8c3e57a-ae17-4f87-ba38-acc0e14583c4/serve-healthcheck-canary/0.log" Jun 15 02:34:25.873678 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:25.873569 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-646b5685b-q2pwl"] Jun 15 02:34:25.874189 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:25.873899 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="786694ac-916e-4cbd-a6fb-b2fa97ce7bb9" containerName="registry" Jun 15 02:34:25.874189 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:25.873915 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="786694ac-916e-4cbd-a6fb-b2fa97ce7bb9" containerName="registry" Jun 15 02:34:25.874189 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:25.873934 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0580edc3-3dea-43e9-9839-d5600831c982" containerName="console" Jun 15 02:34:25.874189 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:25.873939 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="0580edc3-3dea-43e9-9839-d5600831c982" containerName="console" Jun 15 02:34:25.874189 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:25.873981 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="786694ac-916e-4cbd-a6fb-b2fa97ce7bb9" containerName="registry" Jun 15 02:34:25.874189 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:25.873994 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="0580edc3-3dea-43e9-9839-d5600831c982" containerName="console" Jun 15 02:34:25.877134 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:25.877114 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-646b5685b-q2pwl" Jun 15 02:34:25.880065 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:25.879966 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jun 15 02:34:25.880065 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:25.880017 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jun 15 02:34:25.880065 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:25.880027 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jun 15 02:34:25.880282 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:25.880017 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jun 15 02:34:25.880282 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:25.880127 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-wzlwb\"" Jun 15 02:34:25.880874 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:25.880853 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jun 15 02:34:25.884343 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:25.884317 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jun 15 02:34:25.889088 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:25.889064 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-646b5685b-q2pwl"] Jun 15 02:34:25.976023 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:25.975991 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/fcb07ac5-8ec3-45d0-abbc-76db097c3ecb-telemeter-client-tls\") pod \"telemeter-client-646b5685b-q2pwl\" (UID: \"fcb07ac5-8ec3-45d0-abbc-76db097c3ecb\") " pod="openshift-monitoring/telemeter-client-646b5685b-q2pwl" Jun 15 02:34:25.976207 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:25.976030 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fcb07ac5-8ec3-45d0-abbc-76db097c3ecb-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-646b5685b-q2pwl\" (UID: \"fcb07ac5-8ec3-45d0-abbc-76db097c3ecb\") " pod="openshift-monitoring/telemeter-client-646b5685b-q2pwl" Jun 15 02:34:25.976207 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:25.976062 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fcb07ac5-8ec3-45d0-abbc-76db097c3ecb-telemeter-trusted-ca-bundle\") pod \"telemeter-client-646b5685b-q2pwl\" (UID: \"fcb07ac5-8ec3-45d0-abbc-76db097c3ecb\") " pod="openshift-monitoring/telemeter-client-646b5685b-q2pwl" Jun 15 02:34:25.976207 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:25.976152 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/fcb07ac5-8ec3-45d0-abbc-76db097c3ecb-secret-telemeter-client\") pod \"telemeter-client-646b5685b-q2pwl\" (UID: \"fcb07ac5-8ec3-45d0-abbc-76db097c3ecb\") " pod="openshift-monitoring/telemeter-client-646b5685b-q2pwl" Jun 15 02:34:25.976207 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:25.976195 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjzrd\" (UniqueName: \"kubernetes.io/projected/fcb07ac5-8ec3-45d0-abbc-76db097c3ecb-kube-api-access-kjzrd\") pod \"telemeter-client-646b5685b-q2pwl\" (UID: \"fcb07ac5-8ec3-45d0-abbc-76db097c3ecb\") " pod="openshift-monitoring/telemeter-client-646b5685b-q2pwl" Jun 15 02:34:25.976421 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:25.976214 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fcb07ac5-8ec3-45d0-abbc-76db097c3ecb-metrics-client-ca\") pod \"telemeter-client-646b5685b-q2pwl\" (UID: \"fcb07ac5-8ec3-45d0-abbc-76db097c3ecb\") " pod="openshift-monitoring/telemeter-client-646b5685b-q2pwl" Jun 15 02:34:25.976421 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:25.976252 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/fcb07ac5-8ec3-45d0-abbc-76db097c3ecb-federate-client-tls\") pod \"telemeter-client-646b5685b-q2pwl\" (UID: \"fcb07ac5-8ec3-45d0-abbc-76db097c3ecb\") " pod="openshift-monitoring/telemeter-client-646b5685b-q2pwl" Jun 15 02:34:25.976421 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:25.976298 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fcb07ac5-8ec3-45d0-abbc-76db097c3ecb-serving-certs-ca-bundle\") pod \"telemeter-client-646b5685b-q2pwl\" (UID: \"fcb07ac5-8ec3-45d0-abbc-76db097c3ecb\") " pod="openshift-monitoring/telemeter-client-646b5685b-q2pwl" Jun 15 02:34:26.077283 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:26.077255 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/fcb07ac5-8ec3-45d0-abbc-76db097c3ecb-secret-telemeter-client\") pod \"telemeter-client-646b5685b-q2pwl\" (UID: \"fcb07ac5-8ec3-45d0-abbc-76db097c3ecb\") " pod="openshift-monitoring/telemeter-client-646b5685b-q2pwl" Jun 15 02:34:26.077408 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:26.077293 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjzrd\" (UniqueName: \"kubernetes.io/projected/fcb07ac5-8ec3-45d0-abbc-76db097c3ecb-kube-api-access-kjzrd\") pod \"telemeter-client-646b5685b-q2pwl\" (UID: \"fcb07ac5-8ec3-45d0-abbc-76db097c3ecb\") " pod="openshift-monitoring/telemeter-client-646b5685b-q2pwl" Jun 15 02:34:26.077408 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:26.077316 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fcb07ac5-8ec3-45d0-abbc-76db097c3ecb-metrics-client-ca\") pod \"telemeter-client-646b5685b-q2pwl\" (UID: \"fcb07ac5-8ec3-45d0-abbc-76db097c3ecb\") " pod="openshift-monitoring/telemeter-client-646b5685b-q2pwl" Jun 15 02:34:26.077408 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:26.077341 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/fcb07ac5-8ec3-45d0-abbc-76db097c3ecb-federate-client-tls\") pod \"telemeter-client-646b5685b-q2pwl\" (UID: \"fcb07ac5-8ec3-45d0-abbc-76db097c3ecb\") " pod="openshift-monitoring/telemeter-client-646b5685b-q2pwl" Jun 15 02:34:26.077528 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:26.077505 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fcb07ac5-8ec3-45d0-abbc-76db097c3ecb-serving-certs-ca-bundle\") pod \"telemeter-client-646b5685b-q2pwl\" (UID: \"fcb07ac5-8ec3-45d0-abbc-76db097c3ecb\") " pod="openshift-monitoring/telemeter-client-646b5685b-q2pwl" Jun 15 02:34:26.077572 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:26.077553 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/fcb07ac5-8ec3-45d0-abbc-76db097c3ecb-telemeter-client-tls\") pod \"telemeter-client-646b5685b-q2pwl\" (UID: \"fcb07ac5-8ec3-45d0-abbc-76db097c3ecb\") " pod="openshift-monitoring/telemeter-client-646b5685b-q2pwl" Jun 15 02:34:26.077676 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:26.077655 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fcb07ac5-8ec3-45d0-abbc-76db097c3ecb-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-646b5685b-q2pwl\" (UID: \"fcb07ac5-8ec3-45d0-abbc-76db097c3ecb\") " pod="openshift-monitoring/telemeter-client-646b5685b-q2pwl" Jun 15 02:34:26.077917 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:26.077890 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fcb07ac5-8ec3-45d0-abbc-76db097c3ecb-telemeter-trusted-ca-bundle\") pod \"telemeter-client-646b5685b-q2pwl\" (UID: \"fcb07ac5-8ec3-45d0-abbc-76db097c3ecb\") " pod="openshift-monitoring/telemeter-client-646b5685b-q2pwl" Jun 15 02:34:26.078056 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:26.078034 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fcb07ac5-8ec3-45d0-abbc-76db097c3ecb-metrics-client-ca\") pod \"telemeter-client-646b5685b-q2pwl\" (UID: \"fcb07ac5-8ec3-45d0-abbc-76db097c3ecb\") " pod="openshift-monitoring/telemeter-client-646b5685b-q2pwl" Jun 15 02:34:26.078266 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:26.078239 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fcb07ac5-8ec3-45d0-abbc-76db097c3ecb-serving-certs-ca-bundle\") pod \"telemeter-client-646b5685b-q2pwl\" (UID: \"fcb07ac5-8ec3-45d0-abbc-76db097c3ecb\") " pod="openshift-monitoring/telemeter-client-646b5685b-q2pwl" Jun 15 02:34:26.078723 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:26.078701 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fcb07ac5-8ec3-45d0-abbc-76db097c3ecb-telemeter-trusted-ca-bundle\") pod \"telemeter-client-646b5685b-q2pwl\" (UID: \"fcb07ac5-8ec3-45d0-abbc-76db097c3ecb\") " pod="openshift-monitoring/telemeter-client-646b5685b-q2pwl" Jun 15 02:34:26.080053 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:26.080020 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fcb07ac5-8ec3-45d0-abbc-76db097c3ecb-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-646b5685b-q2pwl\" (UID: \"fcb07ac5-8ec3-45d0-abbc-76db097c3ecb\") " pod="openshift-monitoring/telemeter-client-646b5685b-q2pwl" Jun 15 02:34:26.080132 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:26.080063 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/fcb07ac5-8ec3-45d0-abbc-76db097c3ecb-telemeter-client-tls\") pod \"telemeter-client-646b5685b-q2pwl\" (UID: \"fcb07ac5-8ec3-45d0-abbc-76db097c3ecb\") " pod="openshift-monitoring/telemeter-client-646b5685b-q2pwl" Jun 15 02:34:26.080374 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:26.080354 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/fcb07ac5-8ec3-45d0-abbc-76db097c3ecb-federate-client-tls\") pod \"telemeter-client-646b5685b-q2pwl\" (UID: \"fcb07ac5-8ec3-45d0-abbc-76db097c3ecb\") " pod="openshift-monitoring/telemeter-client-646b5685b-q2pwl" Jun 15 02:34:26.080411 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:26.080354 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/fcb07ac5-8ec3-45d0-abbc-76db097c3ecb-secret-telemeter-client\") pod \"telemeter-client-646b5685b-q2pwl\" (UID: \"fcb07ac5-8ec3-45d0-abbc-76db097c3ecb\") " pod="openshift-monitoring/telemeter-client-646b5685b-q2pwl" Jun 15 02:34:26.084785 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:26.084760 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjzrd\" (UniqueName: \"kubernetes.io/projected/fcb07ac5-8ec3-45d0-abbc-76db097c3ecb-kube-api-access-kjzrd\") pod \"telemeter-client-646b5685b-q2pwl\" (UID: \"fcb07ac5-8ec3-45d0-abbc-76db097c3ecb\") " pod="openshift-monitoring/telemeter-client-646b5685b-q2pwl" Jun 15 02:34:26.189918 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:26.189868 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-646b5685b-q2pwl" Jun 15 02:34:26.320625 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:26.320600 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-646b5685b-q2pwl"] Jun 15 02:34:26.322935 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:34:26.322905 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcb07ac5_8ec3_45d0_abbc_76db097c3ecb.slice/crio-fd68292a10ea49a3e709d576d3962d4dd6e350bb03c5926247371d12c1a8e4a6 WatchSource:0}: Error finding container fd68292a10ea49a3e709d576d3962d4dd6e350bb03c5926247371d12c1a8e4a6: Status 404 returned error can't find the container with id fd68292a10ea49a3e709d576d3962d4dd6e350bb03c5926247371d12c1a8e4a6 Jun 15 02:34:26.324836 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:26.324820 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 15 02:34:26.750159 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:26.750127 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-646b5685b-q2pwl" event={"ID":"fcb07ac5-8ec3-45d0-abbc-76db097c3ecb","Type":"ContainerStarted","Data":"fd68292a10ea49a3e709d576d3962d4dd6e350bb03c5926247371d12c1a8e4a6"} Jun 15 02:34:28.760044 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:28.760009 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-646b5685b-q2pwl" event={"ID":"fcb07ac5-8ec3-45d0-abbc-76db097c3ecb","Type":"ContainerStarted","Data":"24caaab07a167a47651f171113de9f059a3d4185ffdacd04e6458837c84700a4"} Jun 15 02:34:28.760418 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:28.760108 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-646b5685b-q2pwl" event={"ID":"fcb07ac5-8ec3-45d0-abbc-76db097c3ecb","Type":"ContainerStarted","Data":"801bb73a095351f44310974c82a12cf539b2670a08f13d353627b3cc8ef09872"} Jun 15 02:34:28.760418 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:28.760138 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-646b5685b-q2pwl" event={"ID":"fcb07ac5-8ec3-45d0-abbc-76db097c3ecb","Type":"ContainerStarted","Data":"8e9e4160f1024dff21883f705f34b16ce2d36405abac35a2df5f45a8637fd084"} Jun 15 02:34:28.781695 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:28.781625 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-646b5685b-q2pwl" podStartSLOduration=1.815304647 podStartE2EDuration="3.781611933s" totalImagesPullingTime="1.966307286s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 02:34:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 02:34:28.78092369 +0000 UTC m=+361.648708435" watchObservedRunningTime="2026-06-15 02:34:28.781611933 +0000 UTC m=+361.649396677" Jun 15 02:34:29.459541 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:29.459507 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8f8854d57-db85h"] Jun 15 02:34:29.463254 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:29.463228 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8f8854d57-db85h" Jun 15 02:34:29.473881 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:29.473831 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8f8854d57-db85h"] Jun 15 02:34:29.504875 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:29.504848 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f330b177-c419-4b10-9e07-f849ddbcb185-service-ca\") pod \"console-8f8854d57-db85h\" (UID: \"f330b177-c419-4b10-9e07-f849ddbcb185\") " pod="openshift-console/console-8f8854d57-db85h" Jun 15 02:34:29.505020 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:29.504883 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zttlp\" (UniqueName: \"kubernetes.io/projected/f330b177-c419-4b10-9e07-f849ddbcb185-kube-api-access-zttlp\") pod \"console-8f8854d57-db85h\" (UID: \"f330b177-c419-4b10-9e07-f849ddbcb185\") " pod="openshift-console/console-8f8854d57-db85h" Jun 15 02:34:29.505020 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:29.504945 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f330b177-c419-4b10-9e07-f849ddbcb185-trusted-ca-bundle\") pod \"console-8f8854d57-db85h\" (UID: \"f330b177-c419-4b10-9e07-f849ddbcb185\") " pod="openshift-console/console-8f8854d57-db85h" Jun 15 02:34:29.505020 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:29.504982 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f330b177-c419-4b10-9e07-f849ddbcb185-console-serving-cert\") pod \"console-8f8854d57-db85h\" (UID: \"f330b177-c419-4b10-9e07-f849ddbcb185\") " pod="openshift-console/console-8f8854d57-db85h" Jun 15 02:34:29.505020 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:29.505014 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f330b177-c419-4b10-9e07-f849ddbcb185-console-oauth-config\") pod \"console-8f8854d57-db85h\" (UID: \"f330b177-c419-4b10-9e07-f849ddbcb185\") " pod="openshift-console/console-8f8854d57-db85h" Jun 15 02:34:29.505159 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:29.505030 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f330b177-c419-4b10-9e07-f849ddbcb185-console-config\") pod \"console-8f8854d57-db85h\" (UID: \"f330b177-c419-4b10-9e07-f849ddbcb185\") " pod="openshift-console/console-8f8854d57-db85h" Jun 15 02:34:29.505159 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:29.505049 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f330b177-c419-4b10-9e07-f849ddbcb185-oauth-serving-cert\") pod \"console-8f8854d57-db85h\" (UID: \"f330b177-c419-4b10-9e07-f849ddbcb185\") " pod="openshift-console/console-8f8854d57-db85h" Jun 15 02:34:29.605699 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:29.605666 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f330b177-c419-4b10-9e07-f849ddbcb185-service-ca\") pod \"console-8f8854d57-db85h\" (UID: \"f330b177-c419-4b10-9e07-f849ddbcb185\") " pod="openshift-console/console-8f8854d57-db85h" Jun 15 02:34:29.605841 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:29.605703 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zttlp\" (UniqueName: \"kubernetes.io/projected/f330b177-c419-4b10-9e07-f849ddbcb185-kube-api-access-zttlp\") pod \"console-8f8854d57-db85h\" (UID: \"f330b177-c419-4b10-9e07-f849ddbcb185\") " pod="openshift-console/console-8f8854d57-db85h" Jun 15 02:34:29.605841 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:29.605764 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f330b177-c419-4b10-9e07-f849ddbcb185-trusted-ca-bundle\") pod \"console-8f8854d57-db85h\" (UID: \"f330b177-c419-4b10-9e07-f849ddbcb185\") " pod="openshift-console/console-8f8854d57-db85h" Jun 15 02:34:29.605841 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:29.605798 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f330b177-c419-4b10-9e07-f849ddbcb185-console-serving-cert\") pod \"console-8f8854d57-db85h\" (UID: \"f330b177-c419-4b10-9e07-f849ddbcb185\") " pod="openshift-console/console-8f8854d57-db85h" Jun 15 02:34:29.605995 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:29.605916 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f330b177-c419-4b10-9e07-f849ddbcb185-console-oauth-config\") pod \"console-8f8854d57-db85h\" (UID: \"f330b177-c419-4b10-9e07-f849ddbcb185\") " pod="openshift-console/console-8f8854d57-db85h" Jun 15 02:34:29.605995 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:29.605954 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f330b177-c419-4b10-9e07-f849ddbcb185-console-config\") pod \"console-8f8854d57-db85h\" (UID: \"f330b177-c419-4b10-9e07-f849ddbcb185\") " pod="openshift-console/console-8f8854d57-db85h" Jun 15 02:34:29.605995 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:29.605979 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f330b177-c419-4b10-9e07-f849ddbcb185-oauth-serving-cert\") pod \"console-8f8854d57-db85h\" (UID: \"f330b177-c419-4b10-9e07-f849ddbcb185\") " pod="openshift-console/console-8f8854d57-db85h" Jun 15 02:34:29.606677 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:29.606626 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f330b177-c419-4b10-9e07-f849ddbcb185-service-ca\") pod \"console-8f8854d57-db85h\" (UID: \"f330b177-c419-4b10-9e07-f849ddbcb185\") " pod="openshift-console/console-8f8854d57-db85h" Jun 15 02:34:29.606677 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:29.606677 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f330b177-c419-4b10-9e07-f849ddbcb185-trusted-ca-bundle\") pod \"console-8f8854d57-db85h\" (UID: \"f330b177-c419-4b10-9e07-f849ddbcb185\") " pod="openshift-console/console-8f8854d57-db85h" Jun 15 02:34:29.606880 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:29.606704 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f330b177-c419-4b10-9e07-f849ddbcb185-console-config\") pod \"console-8f8854d57-db85h\" (UID: \"f330b177-c419-4b10-9e07-f849ddbcb185\") " pod="openshift-console/console-8f8854d57-db85h" Jun 15 02:34:29.606880 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:29.606733 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f330b177-c419-4b10-9e07-f849ddbcb185-oauth-serving-cert\") pod \"console-8f8854d57-db85h\" (UID: \"f330b177-c419-4b10-9e07-f849ddbcb185\") " pod="openshift-console/console-8f8854d57-db85h" Jun 15 02:34:29.608221 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:29.608203 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f330b177-c419-4b10-9e07-f849ddbcb185-console-oauth-config\") pod \"console-8f8854d57-db85h\" (UID: \"f330b177-c419-4b10-9e07-f849ddbcb185\") " pod="openshift-console/console-8f8854d57-db85h" Jun 15 02:34:29.608322 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:29.608311 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f330b177-c419-4b10-9e07-f849ddbcb185-console-serving-cert\") pod \"console-8f8854d57-db85h\" (UID: \"f330b177-c419-4b10-9e07-f849ddbcb185\") " pod="openshift-console/console-8f8854d57-db85h" Jun 15 02:34:29.612865 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:29.612846 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zttlp\" (UniqueName: \"kubernetes.io/projected/f330b177-c419-4b10-9e07-f849ddbcb185-kube-api-access-zttlp\") pod \"console-8f8854d57-db85h\" (UID: \"f330b177-c419-4b10-9e07-f849ddbcb185\") " pod="openshift-console/console-8f8854d57-db85h" Jun 15 02:34:29.773912 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:29.773887 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8f8854d57-db85h" Jun 15 02:34:29.896998 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:29.896944 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8f8854d57-db85h"] Jun 15 02:34:29.899730 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:34:29.899698 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf330b177_c419_4b10_9e07_f849ddbcb185.slice/crio-afa5732250aaf3529b64baf0561c2e4180f6d959e8cd77bce2eafebc969eb70c WatchSource:0}: Error finding container afa5732250aaf3529b64baf0561c2e4180f6d959e8cd77bce2eafebc969eb70c: Status 404 returned error can't find the container with id afa5732250aaf3529b64baf0561c2e4180f6d959e8cd77bce2eafebc969eb70c Jun 15 02:34:30.768629 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:30.768595 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8f8854d57-db85h" event={"ID":"f330b177-c419-4b10-9e07-f849ddbcb185","Type":"ContainerStarted","Data":"b29ce039d096debc2d272278d77a97f88f612e62ede8ed84a2876e32e1ab3aad"} Jun 15 02:34:30.768825 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:30.768673 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8f8854d57-db85h" event={"ID":"f330b177-c419-4b10-9e07-f849ddbcb185","Type":"ContainerStarted","Data":"afa5732250aaf3529b64baf0561c2e4180f6d959e8cd77bce2eafebc969eb70c"} Jun 15 02:34:30.787378 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:30.787328 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8f8854d57-db85h" podStartSLOduration=1.787313084 podStartE2EDuration="1.787313084s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 02:34:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 02:34:30.785871856 +0000 UTC m=+363.653656603" watchObservedRunningTime="2026-06-15 02:34:30.787313084 +0000 UTC m=+363.655097828" Jun 15 02:34:37.625732 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:37.625688 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8f8854d57-db85h"] Jun 15 02:34:39.774262 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:39.774225 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8f8854d57-db85h" Jun 15 02:34:48.459865 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:34:48.459835 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76cfc46c9c-fcttn"] Jun 15 02:35:02.115531 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:02.115496 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvnr7"] Jun 15 02:35:02.119051 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:02.119036 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvnr7" Jun 15 02:35:02.121261 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:02.121228 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 15 02:35:02.121391 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:02.121266 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-mkns2\"" Jun 15 02:35:02.121391 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:02.121266 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 15 02:35:02.128190 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:02.128164 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvnr7"] Jun 15 02:35:02.234377 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:02.234348 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rld99\" (UniqueName: \"kubernetes.io/projected/457d5698-8dba-45a4-a50c-5ea9eed58795-kube-api-access-rld99\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvnr7\" (UID: \"457d5698-8dba-45a4-a50c-5ea9eed58795\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvnr7" Jun 15 02:35:02.234478 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:02.234380 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/457d5698-8dba-45a4-a50c-5ea9eed58795-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvnr7\" (UID: \"457d5698-8dba-45a4-a50c-5ea9eed58795\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvnr7" Jun 15 02:35:02.234478 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:02.234418 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/457d5698-8dba-45a4-a50c-5ea9eed58795-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvnr7\" (UID: \"457d5698-8dba-45a4-a50c-5ea9eed58795\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvnr7" Jun 15 02:35:02.335064 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:02.335040 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rld99\" (UniqueName: \"kubernetes.io/projected/457d5698-8dba-45a4-a50c-5ea9eed58795-kube-api-access-rld99\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvnr7\" (UID: \"457d5698-8dba-45a4-a50c-5ea9eed58795\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvnr7" Jun 15 02:35:02.335170 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:02.335069 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/457d5698-8dba-45a4-a50c-5ea9eed58795-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvnr7\" (UID: \"457d5698-8dba-45a4-a50c-5ea9eed58795\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvnr7" Jun 15 02:35:02.335170 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:02.335099 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/457d5698-8dba-45a4-a50c-5ea9eed58795-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvnr7\" (UID: \"457d5698-8dba-45a4-a50c-5ea9eed58795\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvnr7" Jun 15 02:35:02.335476 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:02.335453 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/457d5698-8dba-45a4-a50c-5ea9eed58795-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvnr7\" (UID: \"457d5698-8dba-45a4-a50c-5ea9eed58795\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvnr7" Jun 15 02:35:02.335476 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:02.335470 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/457d5698-8dba-45a4-a50c-5ea9eed58795-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvnr7\" (UID: \"457d5698-8dba-45a4-a50c-5ea9eed58795\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvnr7" Jun 15 02:35:02.342953 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:02.342932 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rld99\" (UniqueName: \"kubernetes.io/projected/457d5698-8dba-45a4-a50c-5ea9eed58795-kube-api-access-rld99\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvnr7\" (UID: \"457d5698-8dba-45a4-a50c-5ea9eed58795\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvnr7" Jun 15 02:35:02.429331 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:02.429281 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvnr7" Jun 15 02:35:02.552615 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:02.552546 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvnr7"] Jun 15 02:35:02.555044 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:35:02.555020 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod457d5698_8dba_45a4_a50c_5ea9eed58795.slice/crio-2cde6a84db06619c7486281563866c14380fd6ea29d431ed92bca5eb7f3aeb6a WatchSource:0}: Error finding container 2cde6a84db06619c7486281563866c14380fd6ea29d431ed92bca5eb7f3aeb6a: Status 404 returned error can't find the container with id 2cde6a84db06619c7486281563866c14380fd6ea29d431ed92bca5eb7f3aeb6a Jun 15 02:35:02.648743 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:02.648708 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-8f8854d57-db85h" podUID="f330b177-c419-4b10-9e07-f849ddbcb185" containerName="console" containerID="cri-o://b29ce039d096debc2d272278d77a97f88f612e62ede8ed84a2876e32e1ab3aad" gracePeriod=15 Jun 15 02:35:02.887010 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:02.886985 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8f8854d57-db85h" Jun 15 02:35:02.889395 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:02.889371 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvnr7" event={"ID":"457d5698-8dba-45a4-a50c-5ea9eed58795","Type":"ContainerStarted","Data":"2cde6a84db06619c7486281563866c14380fd6ea29d431ed92bca5eb7f3aeb6a"} Jun 15 02:35:02.890572 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:02.890549 2579 generic.go:356] "Generic (PLEG): container finished" podID="f330b177-c419-4b10-9e07-f849ddbcb185" containerID="b29ce039d096debc2d272278d77a97f88f612e62ede8ed84a2876e32e1ab3aad" exitCode=0 Jun 15 02:35:02.890699 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:02.890585 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8f8854d57-db85h" Jun 15 02:35:02.890699 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:02.890602 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8f8854d57-db85h" event={"ID":"f330b177-c419-4b10-9e07-f849ddbcb185","Type":"ContainerDied","Data":"b29ce039d096debc2d272278d77a97f88f612e62ede8ed84a2876e32e1ab3aad"} Jun 15 02:35:02.890699 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:02.890670 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8f8854d57-db85h" event={"ID":"f330b177-c419-4b10-9e07-f849ddbcb185","Type":"ContainerDied","Data":"afa5732250aaf3529b64baf0561c2e4180f6d959e8cd77bce2eafebc969eb70c"} Jun 15 02:35:02.890699 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:02.890693 2579 scope.go:117] "RemoveContainer" containerID="b29ce039d096debc2d272278d77a97f88f612e62ede8ed84a2876e32e1ab3aad" Jun 15 02:35:02.899697 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:02.899681 2579 scope.go:117] "RemoveContainer" containerID="b29ce039d096debc2d272278d77a97f88f612e62ede8ed84a2876e32e1ab3aad" Jun 15 02:35:02.900015 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:35:02.899998 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b29ce039d096debc2d272278d77a97f88f612e62ede8ed84a2876e32e1ab3aad\": container with ID starting with b29ce039d096debc2d272278d77a97f88f612e62ede8ed84a2876e32e1ab3aad not found: ID does not exist" containerID="b29ce039d096debc2d272278d77a97f88f612e62ede8ed84a2876e32e1ab3aad" Jun 15 02:35:02.900078 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:02.900026 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b29ce039d096debc2d272278d77a97f88f612e62ede8ed84a2876e32e1ab3aad"} err="failed to get container status \"b29ce039d096debc2d272278d77a97f88f612e62ede8ed84a2876e32e1ab3aad\": rpc error: code = NotFound desc = could not find container \"b29ce039d096debc2d272278d77a97f88f612e62ede8ed84a2876e32e1ab3aad\": container with ID starting with b29ce039d096debc2d272278d77a97f88f612e62ede8ed84a2876e32e1ab3aad not found: ID does not exist" Jun 15 02:35:03.041567 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:03.041507 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f330b177-c419-4b10-9e07-f849ddbcb185-console-oauth-config\") pod \"f330b177-c419-4b10-9e07-f849ddbcb185\" (UID: \"f330b177-c419-4b10-9e07-f849ddbcb185\") " Jun 15 02:35:03.041567 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:03.041548 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f330b177-c419-4b10-9e07-f849ddbcb185-service-ca\") pod \"f330b177-c419-4b10-9e07-f849ddbcb185\" (UID: \"f330b177-c419-4b10-9e07-f849ddbcb185\") " Jun 15 02:35:03.041567 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:03.041566 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f330b177-c419-4b10-9e07-f849ddbcb185-console-config\") pod \"f330b177-c419-4b10-9e07-f849ddbcb185\" (UID: \"f330b177-c419-4b10-9e07-f849ddbcb185\") " Jun 15 02:35:03.041824 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:03.041608 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zttlp\" (UniqueName: \"kubernetes.io/projected/f330b177-c419-4b10-9e07-f849ddbcb185-kube-api-access-zttlp\") pod \"f330b177-c419-4b10-9e07-f849ddbcb185\" (UID: \"f330b177-c419-4b10-9e07-f849ddbcb185\") " Jun 15 02:35:03.041824 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:03.041680 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f330b177-c419-4b10-9e07-f849ddbcb185-console-serving-cert\") pod \"f330b177-c419-4b10-9e07-f849ddbcb185\" (UID: \"f330b177-c419-4b10-9e07-f849ddbcb185\") " Jun 15 02:35:03.041824 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:03.041717 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f330b177-c419-4b10-9e07-f849ddbcb185-trusted-ca-bundle\") pod \"f330b177-c419-4b10-9e07-f849ddbcb185\" (UID: \"f330b177-c419-4b10-9e07-f849ddbcb185\") " Jun 15 02:35:03.041824 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:03.041801 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f330b177-c419-4b10-9e07-f849ddbcb185-oauth-serving-cert\") pod \"f330b177-c419-4b10-9e07-f849ddbcb185\" (UID: \"f330b177-c419-4b10-9e07-f849ddbcb185\") " Jun 15 02:35:03.042023 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:03.041955 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f330b177-c419-4b10-9e07-f849ddbcb185-service-ca" (OuterVolumeSpecName: "service-ca") pod "f330b177-c419-4b10-9e07-f849ddbcb185" (UID: "f330b177-c419-4b10-9e07-f849ddbcb185"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 02:35:03.042106 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:03.042074 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f330b177-c419-4b10-9e07-f849ddbcb185-console-config" (OuterVolumeSpecName: "console-config") pod "f330b177-c419-4b10-9e07-f849ddbcb185" (UID: "f330b177-c419-4b10-9e07-f849ddbcb185"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 02:35:03.042205 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:03.042118 2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f330b177-c419-4b10-9e07-f849ddbcb185-service-ca\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:35:03.042270 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:03.042230 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f330b177-c419-4b10-9e07-f849ddbcb185-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "f330b177-c419-4b10-9e07-f849ddbcb185" (UID: "f330b177-c419-4b10-9e07-f849ddbcb185"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 02:35:03.042270 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:03.042240 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f330b177-c419-4b10-9e07-f849ddbcb185-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f330b177-c419-4b10-9e07-f849ddbcb185" (UID: "f330b177-c419-4b10-9e07-f849ddbcb185"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 02:35:03.043580 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:03.043563 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f330b177-c419-4b10-9e07-f849ddbcb185-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f330b177-c419-4b10-9e07-f849ddbcb185" (UID: "f330b177-c419-4b10-9e07-f849ddbcb185"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 15 02:35:03.044097 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:03.044080 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f330b177-c419-4b10-9e07-f849ddbcb185-kube-api-access-zttlp" (OuterVolumeSpecName: "kube-api-access-zttlp") pod "f330b177-c419-4b10-9e07-f849ddbcb185" (UID: "f330b177-c419-4b10-9e07-f849ddbcb185"). InnerVolumeSpecName "kube-api-access-zttlp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 02:35:03.044145 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:03.044086 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f330b177-c419-4b10-9e07-f849ddbcb185-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f330b177-c419-4b10-9e07-f849ddbcb185" (UID: "f330b177-c419-4b10-9e07-f849ddbcb185"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 15 02:35:03.142852 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:03.142831 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zttlp\" (UniqueName: \"kubernetes.io/projected/f330b177-c419-4b10-9e07-f849ddbcb185-kube-api-access-zttlp\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:35:03.142852 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:03.142849 2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f330b177-c419-4b10-9e07-f849ddbcb185-console-serving-cert\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:35:03.143166 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:03.142861 2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f330b177-c419-4b10-9e07-f849ddbcb185-trusted-ca-bundle\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:35:03.143166 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:03.142870 2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f330b177-c419-4b10-9e07-f849ddbcb185-oauth-serving-cert\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:35:03.143166 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:03.142878 2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f330b177-c419-4b10-9e07-f849ddbcb185-console-oauth-config\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:35:03.143166 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:03.142886 2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f330b177-c419-4b10-9e07-f849ddbcb185-console-config\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:35:03.209978 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:03.209939 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8f8854d57-db85h"] Jun 15 02:35:03.213705 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:03.213683 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8f8854d57-db85h"] Jun 15 02:35:03.493387 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:03.493359 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f330b177-c419-4b10-9e07-f849ddbcb185" path="/var/lib/kubelet/pods/f330b177-c419-4b10-9e07-f849ddbcb185/volumes" Jun 15 02:35:10.925087 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:10.925049 2579 generic.go:356] "Generic (PLEG): container finished" podID="457d5698-8dba-45a4-a50c-5ea9eed58795" containerID="f60b387c9fd508d7cc44ed618443ff726e5630dfbb462f8e420cad9dd626e2d8" exitCode=0 Jun 15 02:35:10.925500 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:10.925129 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvnr7" event={"ID":"457d5698-8dba-45a4-a50c-5ea9eed58795","Type":"ContainerDied","Data":"f60b387c9fd508d7cc44ed618443ff726e5630dfbb462f8e420cad9dd626e2d8"} Jun 15 02:35:13.479506 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:13.479457 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-76cfc46c9c-fcttn" podUID="d748b607-99fd-4790-8b6d-2942f4d135aa" containerName="console" containerID="cri-o://9dc5674043ba886097106fbc3eb70d48be5a933a1e3ed882af8e9bc910c9b2b8" gracePeriod=15 Jun 15 02:35:13.718387 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:13.718366 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76cfc46c9c-fcttn" Jun 15 02:35:13.828552 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:13.828495 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d748b607-99fd-4790-8b6d-2942f4d135aa-console-oauth-config\") pod \"d748b607-99fd-4790-8b6d-2942f4d135aa\" (UID: \"d748b607-99fd-4790-8b6d-2942f4d135aa\") " Jun 15 02:35:13.828552 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:13.828531 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d748b607-99fd-4790-8b6d-2942f4d135aa-oauth-serving-cert\") pod \"d748b607-99fd-4790-8b6d-2942f4d135aa\" (UID: \"d748b607-99fd-4790-8b6d-2942f4d135aa\") " Jun 15 02:35:13.828737 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:13.828580 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d748b607-99fd-4790-8b6d-2942f4d135aa-service-ca\") pod \"d748b607-99fd-4790-8b6d-2942f4d135aa\" (UID: \"d748b607-99fd-4790-8b6d-2942f4d135aa\") " Jun 15 02:35:13.828737 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:13.828612 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d748b607-99fd-4790-8b6d-2942f4d135aa-trusted-ca-bundle\") pod \"d748b607-99fd-4790-8b6d-2942f4d135aa\" (UID: \"d748b607-99fd-4790-8b6d-2942f4d135aa\") " Jun 15 02:35:13.828737 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:13.828690 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d748b607-99fd-4790-8b6d-2942f4d135aa-console-serving-cert\") pod \"d748b607-99fd-4790-8b6d-2942f4d135aa\" (UID: \"d748b607-99fd-4790-8b6d-2942f4d135aa\") " Jun 15 02:35:13.828906 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:13.828740 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v25ft\" (UniqueName: \"kubernetes.io/projected/d748b607-99fd-4790-8b6d-2942f4d135aa-kube-api-access-v25ft\") pod \"d748b607-99fd-4790-8b6d-2942f4d135aa\" (UID: \"d748b607-99fd-4790-8b6d-2942f4d135aa\") " Jun 15 02:35:13.828906 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:13.828763 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d748b607-99fd-4790-8b6d-2942f4d135aa-console-config\") pod \"d748b607-99fd-4790-8b6d-2942f4d135aa\" (UID: \"d748b607-99fd-4790-8b6d-2942f4d135aa\") " Jun 15 02:35:13.829058 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:13.829029 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d748b607-99fd-4790-8b6d-2942f4d135aa-service-ca" (OuterVolumeSpecName: "service-ca") pod "d748b607-99fd-4790-8b6d-2942f4d135aa" (UID: "d748b607-99fd-4790-8b6d-2942f4d135aa"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 02:35:13.829107 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:13.829038 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d748b607-99fd-4790-8b6d-2942f4d135aa-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d748b607-99fd-4790-8b6d-2942f4d135aa" (UID: "d748b607-99fd-4790-8b6d-2942f4d135aa"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 02:35:13.829155 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:13.829134 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d748b607-99fd-4790-8b6d-2942f4d135aa-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d748b607-99fd-4790-8b6d-2942f4d135aa" (UID: "d748b607-99fd-4790-8b6d-2942f4d135aa"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 02:35:13.829311 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:13.829282 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d748b607-99fd-4790-8b6d-2942f4d135aa-console-config" (OuterVolumeSpecName: "console-config") pod "d748b607-99fd-4790-8b6d-2942f4d135aa" (UID: "d748b607-99fd-4790-8b6d-2942f4d135aa"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 02:35:13.830590 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:13.830566 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d748b607-99fd-4790-8b6d-2942f4d135aa-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d748b607-99fd-4790-8b6d-2942f4d135aa" (UID: "d748b607-99fd-4790-8b6d-2942f4d135aa"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 15 02:35:13.830951 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:13.830924 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d748b607-99fd-4790-8b6d-2942f4d135aa-kube-api-access-v25ft" (OuterVolumeSpecName: "kube-api-access-v25ft") pod "d748b607-99fd-4790-8b6d-2942f4d135aa" (UID: "d748b607-99fd-4790-8b6d-2942f4d135aa"). InnerVolumeSpecName "kube-api-access-v25ft". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 02:35:13.831010 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:13.830950 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d748b607-99fd-4790-8b6d-2942f4d135aa-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d748b607-99fd-4790-8b6d-2942f4d135aa" (UID: "d748b607-99fd-4790-8b6d-2942f4d135aa"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 15 02:35:13.929660 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:13.929616 2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d748b607-99fd-4790-8b6d-2942f4d135aa-service-ca\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:35:13.929660 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:13.929659 2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d748b607-99fd-4790-8b6d-2942f4d135aa-trusted-ca-bundle\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:35:13.929775 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:13.929669 2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d748b607-99fd-4790-8b6d-2942f4d135aa-console-serving-cert\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:35:13.929775 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:13.929679 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v25ft\" (UniqueName: \"kubernetes.io/projected/d748b607-99fd-4790-8b6d-2942f4d135aa-kube-api-access-v25ft\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:35:13.929775 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:13.929689 2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d748b607-99fd-4790-8b6d-2942f4d135aa-console-config\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:35:13.929775 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:13.929697 2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d748b607-99fd-4790-8b6d-2942f4d135aa-console-oauth-config\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:35:13.929775 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:13.929705 2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d748b607-99fd-4790-8b6d-2942f4d135aa-oauth-serving-cert\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:35:13.937993 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:13.937965 2579 generic.go:356] "Generic (PLEG): container finished" podID="d748b607-99fd-4790-8b6d-2942f4d135aa" containerID="9dc5674043ba886097106fbc3eb70d48be5a933a1e3ed882af8e9bc910c9b2b8" exitCode=0 Jun 15 02:35:13.938072 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:13.938004 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76cfc46c9c-fcttn" Jun 15 02:35:13.938072 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:13.938042 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76cfc46c9c-fcttn" event={"ID":"d748b607-99fd-4790-8b6d-2942f4d135aa","Type":"ContainerDied","Data":"9dc5674043ba886097106fbc3eb70d48be5a933a1e3ed882af8e9bc910c9b2b8"} Jun 15 02:35:13.938159 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:13.938092 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76cfc46c9c-fcttn" event={"ID":"d748b607-99fd-4790-8b6d-2942f4d135aa","Type":"ContainerDied","Data":"47ba4bb7080a3ce808924ac473da02e98b7f6d6c0d5a30f4342fa2b3bfed68f9"} Jun 15 02:35:13.938159 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:13.938113 2579 scope.go:117] "RemoveContainer" containerID="9dc5674043ba886097106fbc3eb70d48be5a933a1e3ed882af8e9bc910c9b2b8" Jun 15 02:35:13.939995 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:13.939975 2579 generic.go:356] "Generic (PLEG): container finished" podID="457d5698-8dba-45a4-a50c-5ea9eed58795" containerID="98b735c581562315cfad1321b31ae74d15294cd191b2d18ebac263964aacc65e" exitCode=0 Jun 15 02:35:13.940084 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:13.940038 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvnr7" event={"ID":"457d5698-8dba-45a4-a50c-5ea9eed58795","Type":"ContainerDied","Data":"98b735c581562315cfad1321b31ae74d15294cd191b2d18ebac263964aacc65e"} Jun 15 02:35:13.946393 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:13.946376 2579 scope.go:117] "RemoveContainer" containerID="9dc5674043ba886097106fbc3eb70d48be5a933a1e3ed882af8e9bc910c9b2b8" Jun 15 02:35:13.946810 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:35:13.946781 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9dc5674043ba886097106fbc3eb70d48be5a933a1e3ed882af8e9bc910c9b2b8\": container with ID starting with 9dc5674043ba886097106fbc3eb70d48be5a933a1e3ed882af8e9bc910c9b2b8 not found: ID does not exist" containerID="9dc5674043ba886097106fbc3eb70d48be5a933a1e3ed882af8e9bc910c9b2b8" Jun 15 02:35:13.946899 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:13.946819 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9dc5674043ba886097106fbc3eb70d48be5a933a1e3ed882af8e9bc910c9b2b8"} err="failed to get container status \"9dc5674043ba886097106fbc3eb70d48be5a933a1e3ed882af8e9bc910c9b2b8\": rpc error: code = NotFound desc = could not find container \"9dc5674043ba886097106fbc3eb70d48be5a933a1e3ed882af8e9bc910c9b2b8\": container with ID starting with 9dc5674043ba886097106fbc3eb70d48be5a933a1e3ed882af8e9bc910c9b2b8 not found: ID does not exist" Jun 15 02:35:13.972022 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:13.971999 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76cfc46c9c-fcttn"] Jun 15 02:35:13.975901 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:13.975882 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-76cfc46c9c-fcttn"] Jun 15 02:35:15.492292 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:15.492255 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d748b607-99fd-4790-8b6d-2942f4d135aa" path="/var/lib/kubelet/pods/d748b607-99fd-4790-8b6d-2942f4d135aa/volumes" Jun 15 02:35:22.980875 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:22.980841 2579 generic.go:356] "Generic (PLEG): container finished" podID="457d5698-8dba-45a4-a50c-5ea9eed58795" containerID="1c586e42f69be75ffc3b3efd357fa31d21749bd35d4522d56d3aee8d299e897d" exitCode=0 Jun 15 02:35:22.981230 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:22.980918 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvnr7" event={"ID":"457d5698-8dba-45a4-a50c-5ea9eed58795","Type":"ContainerDied","Data":"1c586e42f69be75ffc3b3efd357fa31d21749bd35d4522d56d3aee8d299e897d"} Jun 15 02:35:24.111431 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:24.111405 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvnr7" Jun 15 02:35:24.208343 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:24.208311 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/457d5698-8dba-45a4-a50c-5ea9eed58795-util\") pod \"457d5698-8dba-45a4-a50c-5ea9eed58795\" (UID: \"457d5698-8dba-45a4-a50c-5ea9eed58795\") " Jun 15 02:35:24.208471 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:24.208372 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/457d5698-8dba-45a4-a50c-5ea9eed58795-bundle\") pod \"457d5698-8dba-45a4-a50c-5ea9eed58795\" (UID: \"457d5698-8dba-45a4-a50c-5ea9eed58795\") " Jun 15 02:35:24.208471 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:24.208427 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rld99\" (UniqueName: \"kubernetes.io/projected/457d5698-8dba-45a4-a50c-5ea9eed58795-kube-api-access-rld99\") pod \"457d5698-8dba-45a4-a50c-5ea9eed58795\" (UID: \"457d5698-8dba-45a4-a50c-5ea9eed58795\") " Jun 15 02:35:24.208960 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:24.208939 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/457d5698-8dba-45a4-a50c-5ea9eed58795-bundle" (OuterVolumeSpecName: "bundle") pod "457d5698-8dba-45a4-a50c-5ea9eed58795" (UID: "457d5698-8dba-45a4-a50c-5ea9eed58795"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 02:35:24.210557 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:24.210538 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/457d5698-8dba-45a4-a50c-5ea9eed58795-kube-api-access-rld99" (OuterVolumeSpecName: "kube-api-access-rld99") pod "457d5698-8dba-45a4-a50c-5ea9eed58795" (UID: "457d5698-8dba-45a4-a50c-5ea9eed58795"). InnerVolumeSpecName "kube-api-access-rld99". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 02:35:24.213014 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:24.212990 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/457d5698-8dba-45a4-a50c-5ea9eed58795-util" (OuterVolumeSpecName: "util") pod "457d5698-8dba-45a4-a50c-5ea9eed58795" (UID: "457d5698-8dba-45a4-a50c-5ea9eed58795"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 02:35:24.309068 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:24.309014 2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/457d5698-8dba-45a4-a50c-5ea9eed58795-util\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:35:24.309068 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:24.309037 2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/457d5698-8dba-45a4-a50c-5ea9eed58795-bundle\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:35:24.309068 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:24.309045 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rld99\" (UniqueName: \"kubernetes.io/projected/457d5698-8dba-45a4-a50c-5ea9eed58795-kube-api-access-rld99\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:35:24.990591 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:24.990567 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvnr7" Jun 15 02:35:24.990591 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:24.990578 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dtvnr7" event={"ID":"457d5698-8dba-45a4-a50c-5ea9eed58795","Type":"ContainerDied","Data":"2cde6a84db06619c7486281563866c14380fd6ea29d431ed92bca5eb7f3aeb6a"} Jun 15 02:35:24.990841 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:24.990622 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2cde6a84db06619c7486281563866c14380fd6ea29d431ed92bca5eb7f3aeb6a" Jun 15 02:35:30.078762 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:30.078728 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-469rc"] Jun 15 02:35:30.079135 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:30.078994 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="457d5698-8dba-45a4-a50c-5ea9eed58795" containerName="extract" Jun 15 02:35:30.079135 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:30.079005 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="457d5698-8dba-45a4-a50c-5ea9eed58795" containerName="extract" Jun 15 02:35:30.079135 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:30.079017 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d748b607-99fd-4790-8b6d-2942f4d135aa" containerName="console" Jun 15 02:35:30.079135 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:30.079022 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d748b607-99fd-4790-8b6d-2942f4d135aa" containerName="console" Jun 15 02:35:30.079135 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:30.079028 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="457d5698-8dba-45a4-a50c-5ea9eed58795" containerName="util" Jun 15 02:35:30.079135 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:30.079034 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="457d5698-8dba-45a4-a50c-5ea9eed58795" containerName="util" Jun 15 02:35:30.079135 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:30.079040 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="457d5698-8dba-45a4-a50c-5ea9eed58795" containerName="pull" Jun 15 02:35:30.079135 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:30.079045 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="457d5698-8dba-45a4-a50c-5ea9eed58795" containerName="pull" Jun 15 02:35:30.079135 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:30.079056 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f330b177-c419-4b10-9e07-f849ddbcb185" containerName="console" Jun 15 02:35:30.079135 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:30.079061 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f330b177-c419-4b10-9e07-f849ddbcb185" containerName="console" Jun 15 02:35:30.079135 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:30.079103 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f330b177-c419-4b10-9e07-f849ddbcb185" containerName="console" Jun 15 02:35:30.079135 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:30.079112 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="457d5698-8dba-45a4-a50c-5ea9eed58795" containerName="extract" Jun 15 02:35:30.079135 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:30.079119 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d748b607-99fd-4790-8b6d-2942f4d135aa" containerName="console" Jun 15 02:35:30.084703 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:30.084685 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-469rc" Jun 15 02:35:30.088928 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:30.088909 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-blpz6\"" Jun 15 02:35:30.089557 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:30.089541 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jun 15 02:35:30.089616 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:30.089592 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jun 15 02:35:30.096722 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:30.096703 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-469rc"] Jun 15 02:35:30.250117 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:30.250093 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/596cf16c-cb29-4933-b874-db516e452cd9-tmp\") pod \"cert-manager-operator-controller-manager-57757f7cfc-469rc\" (UID: \"596cf16c-cb29-4933-b874-db516e452cd9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-469rc" Jun 15 02:35:30.250219 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:30.250123 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7jvp\" (UniqueName: \"kubernetes.io/projected/596cf16c-cb29-4933-b874-db516e452cd9-kube-api-access-z7jvp\") pod \"cert-manager-operator-controller-manager-57757f7cfc-469rc\" (UID: \"596cf16c-cb29-4933-b874-db516e452cd9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-469rc" Jun 15 02:35:30.351547 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:30.351459 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/596cf16c-cb29-4933-b874-db516e452cd9-tmp\") pod \"cert-manager-operator-controller-manager-57757f7cfc-469rc\" (UID: \"596cf16c-cb29-4933-b874-db516e452cd9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-469rc" Jun 15 02:35:30.351547 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:30.351501 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7jvp\" (UniqueName: \"kubernetes.io/projected/596cf16c-cb29-4933-b874-db516e452cd9-kube-api-access-z7jvp\") pod \"cert-manager-operator-controller-manager-57757f7cfc-469rc\" (UID: \"596cf16c-cb29-4933-b874-db516e452cd9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-469rc" Jun 15 02:35:30.351869 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:30.351849 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/596cf16c-cb29-4933-b874-db516e452cd9-tmp\") pod \"cert-manager-operator-controller-manager-57757f7cfc-469rc\" (UID: \"596cf16c-cb29-4933-b874-db516e452cd9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-469rc" Jun 15 02:35:30.359134 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:30.359105 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7jvp\" (UniqueName: \"kubernetes.io/projected/596cf16c-cb29-4933-b874-db516e452cd9-kube-api-access-z7jvp\") pod \"cert-manager-operator-controller-manager-57757f7cfc-469rc\" (UID: \"596cf16c-cb29-4933-b874-db516e452cd9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-469rc" Jun 15 02:35:30.393251 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:30.393226 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-469rc" Jun 15 02:35:30.522053 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:30.522019 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-469rc"] Jun 15 02:35:30.525462 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:35:30.525438 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod596cf16c_cb29_4933_b874_db516e452cd9.slice/crio-d50465fbf290721c1c34c7eaa138a3cb866d3bf66f77674bc7df8ff0ad611009 WatchSource:0}: Error finding container d50465fbf290721c1c34c7eaa138a3cb866d3bf66f77674bc7df8ff0ad611009: Status 404 returned error can't find the container with id d50465fbf290721c1c34c7eaa138a3cb866d3bf66f77674bc7df8ff0ad611009 Jun 15 02:35:31.014975 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:31.014935 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-469rc" event={"ID":"596cf16c-cb29-4933-b874-db516e452cd9","Type":"ContainerStarted","Data":"d50465fbf290721c1c34c7eaa138a3cb866d3bf66f77674bc7df8ff0ad611009"} Jun 15 02:35:33.025417 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:33.025381 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-469rc" event={"ID":"596cf16c-cb29-4933-b874-db516e452cd9","Type":"ContainerStarted","Data":"7b33147003b4b5c4301c3862288990f74b94c23962490981700bcd664e8a60ba"} Jun 15 02:35:33.044278 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:33.044217 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-469rc" podStartSLOduration=1.472154889 podStartE2EDuration="3.044203105s" totalImagesPullingTime="1.572048216s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 02:35:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 02:35:33.042095829 +0000 UTC m=+425.909880573" watchObservedRunningTime="2026-06-15 02:35:33.044203105 +0000 UTC m=+425.911987850" Jun 15 02:35:34.127487 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:34.127449 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqj8rz"] Jun 15 02:35:34.130980 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:34.130958 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqj8rz" Jun 15 02:35:34.133054 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:34.133037 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 15 02:35:34.133143 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:34.133100 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-mkns2\"" Jun 15 02:35:34.133143 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:34.133107 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 15 02:35:34.138231 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:34.138207 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqj8rz"] Jun 15 02:35:34.180431 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:34.180408 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b83c1ca9-56d2-4f27-b07c-05f9668aef95-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqj8rz\" (UID: \"b83c1ca9-56d2-4f27-b07c-05f9668aef95\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqj8rz" Jun 15 02:35:34.180532 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:34.180467 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b83c1ca9-56d2-4f27-b07c-05f9668aef95-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqj8rz\" (UID: \"b83c1ca9-56d2-4f27-b07c-05f9668aef95\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqj8rz" Jun 15 02:35:34.180603 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:34.180541 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8zkh\" (UniqueName: \"kubernetes.io/projected/b83c1ca9-56d2-4f27-b07c-05f9668aef95-kube-api-access-b8zkh\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqj8rz\" (UID: \"b83c1ca9-56d2-4f27-b07c-05f9668aef95\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqj8rz" Jun 15 02:35:34.280786 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:34.280762 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b83c1ca9-56d2-4f27-b07c-05f9668aef95-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqj8rz\" (UID: \"b83c1ca9-56d2-4f27-b07c-05f9668aef95\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqj8rz" Jun 15 02:35:34.280882 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:34.280799 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8zkh\" (UniqueName: \"kubernetes.io/projected/b83c1ca9-56d2-4f27-b07c-05f9668aef95-kube-api-access-b8zkh\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqj8rz\" (UID: \"b83c1ca9-56d2-4f27-b07c-05f9668aef95\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqj8rz" Jun 15 02:35:34.280882 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:34.280821 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b83c1ca9-56d2-4f27-b07c-05f9668aef95-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqj8rz\" (UID: \"b83c1ca9-56d2-4f27-b07c-05f9668aef95\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqj8rz" Jun 15 02:35:34.281137 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:34.281118 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b83c1ca9-56d2-4f27-b07c-05f9668aef95-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqj8rz\" (UID: \"b83c1ca9-56d2-4f27-b07c-05f9668aef95\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqj8rz" Jun 15 02:35:34.281199 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:34.281149 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b83c1ca9-56d2-4f27-b07c-05f9668aef95-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqj8rz\" (UID: \"b83c1ca9-56d2-4f27-b07c-05f9668aef95\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqj8rz" Jun 15 02:35:34.291136 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:34.291108 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8zkh\" (UniqueName: \"kubernetes.io/projected/b83c1ca9-56d2-4f27-b07c-05f9668aef95-kube-api-access-b8zkh\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqj8rz\" (UID: \"b83c1ca9-56d2-4f27-b07c-05f9668aef95\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqj8rz" Jun 15 02:35:34.440979 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:34.440916 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqj8rz" Jun 15 02:35:34.778933 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:34.777575 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqj8rz"] Jun 15 02:35:34.781683 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:35:34.781624 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb83c1ca9_56d2_4f27_b07c_05f9668aef95.slice/crio-85b664c515c26be17319a36e4bf5deb441022dc403f91160ad857a2d0698b81c WatchSource:0}: Error finding container 85b664c515c26be17319a36e4bf5deb441022dc403f91160ad857a2d0698b81c: Status 404 returned error can't find the container with id 85b664c515c26be17319a36e4bf5deb441022dc403f91160ad857a2d0698b81c Jun 15 02:35:35.034596 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:35.034518 2579 generic.go:356] "Generic (PLEG): container finished" podID="b83c1ca9-56d2-4f27-b07c-05f9668aef95" containerID="b052fc505f8fe802cd14ca73cb500fa769b785ba68fe611247be6b01dbedb58e" exitCode=0 Jun 15 02:35:35.034766 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:35.034605 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqj8rz" event={"ID":"b83c1ca9-56d2-4f27-b07c-05f9668aef95","Type":"ContainerDied","Data":"b052fc505f8fe802cd14ca73cb500fa769b785ba68fe611247be6b01dbedb58e"} Jun 15 02:35:35.034766 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:35.034672 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqj8rz" event={"ID":"b83c1ca9-56d2-4f27-b07c-05f9668aef95","Type":"ContainerStarted","Data":"85b664c515c26be17319a36e4bf5deb441022dc403f91160ad857a2d0698b81c"} Jun 15 02:35:35.124982 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:35.124951 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-7dc76cbb94-rn7sp"] Jun 15 02:35:35.128108 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:35.128091 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-7dc76cbb94-rn7sp" Jun 15 02:35:35.129952 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:35.129932 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jun 15 02:35:35.130008 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:35.129969 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jun 15 02:35:35.130054 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:35.130017 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-kbqxj\"" Jun 15 02:35:35.135048 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:35.135025 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-7dc76cbb94-rn7sp"] Jun 15 02:35:35.186652 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:35.186606 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5sbb4\" (UniqueName: \"kubernetes.io/projected/18654557-1d3c-413a-bb4b-d0bc9e612202-kube-api-access-5sbb4\") pod \"cert-manager-webhook-7dc76cbb94-rn7sp\" (UID: \"18654557-1d3c-413a-bb4b-d0bc9e612202\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-rn7sp" Jun 15 02:35:35.186762 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:35.186688 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/18654557-1d3c-413a-bb4b-d0bc9e612202-bound-sa-token\") pod \"cert-manager-webhook-7dc76cbb94-rn7sp\" (UID: \"18654557-1d3c-413a-bb4b-d0bc9e612202\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-rn7sp" Jun 15 02:35:35.287225 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:35.287163 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/18654557-1d3c-413a-bb4b-d0bc9e612202-bound-sa-token\") pod \"cert-manager-webhook-7dc76cbb94-rn7sp\" (UID: \"18654557-1d3c-413a-bb4b-d0bc9e612202\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-rn7sp" Jun 15 02:35:35.287326 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:35.287228 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5sbb4\" (UniqueName: \"kubernetes.io/projected/18654557-1d3c-413a-bb4b-d0bc9e612202-kube-api-access-5sbb4\") pod \"cert-manager-webhook-7dc76cbb94-rn7sp\" (UID: \"18654557-1d3c-413a-bb4b-d0bc9e612202\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-rn7sp" Jun 15 02:35:35.297663 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:35.297624 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/18654557-1d3c-413a-bb4b-d0bc9e612202-bound-sa-token\") pod \"cert-manager-webhook-7dc76cbb94-rn7sp\" (UID: \"18654557-1d3c-413a-bb4b-d0bc9e612202\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-rn7sp" Jun 15 02:35:35.297745 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:35.297702 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5sbb4\" (UniqueName: \"kubernetes.io/projected/18654557-1d3c-413a-bb4b-d0bc9e612202-kube-api-access-5sbb4\") pod \"cert-manager-webhook-7dc76cbb94-rn7sp\" (UID: \"18654557-1d3c-413a-bb4b-d0bc9e612202\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-rn7sp" Jun 15 02:35:35.437686 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:35.437661 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-7dc76cbb94-rn7sp" Jun 15 02:35:35.559106 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:35.559078 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-7dc76cbb94-rn7sp"] Jun 15 02:35:35.561461 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:35:35.561438 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod18654557_1d3c_413a_bb4b_d0bc9e612202.slice/crio-4baadcabf1ebc6d100831bcc6d6a9c103ee9d2d59a6bc1208832b600bd17e9f6 WatchSource:0}: Error finding container 4baadcabf1ebc6d100831bcc6d6a9c103ee9d2d59a6bc1208832b600bd17e9f6: Status 404 returned error can't find the container with id 4baadcabf1ebc6d100831bcc6d6a9c103ee9d2d59a6bc1208832b600bd17e9f6 Jun 15 02:35:36.040041 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:36.040001 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-7dc76cbb94-rn7sp" event={"ID":"18654557-1d3c-413a-bb4b-d0bc9e612202","Type":"ContainerStarted","Data":"4baadcabf1ebc6d100831bcc6d6a9c103ee9d2d59a6bc1208832b600bd17e9f6"} Jun 15 02:35:37.048602 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:37.048569 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqj8rz" event={"ID":"b83c1ca9-56d2-4f27-b07c-05f9668aef95","Type":"ContainerStarted","Data":"938efb7ba76c103453131aede9d23ef77d4fa0a3b67c32e21bc57bbb11dcdbac"} Jun 15 02:35:38.056653 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:38.056603 2579 generic.go:356] "Generic (PLEG): container finished" podID="b83c1ca9-56d2-4f27-b07c-05f9668aef95" containerID="938efb7ba76c103453131aede9d23ef77d4fa0a3b67c32e21bc57bbb11dcdbac" exitCode=0 Jun 15 02:35:38.057085 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:38.056678 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqj8rz" event={"ID":"b83c1ca9-56d2-4f27-b07c-05f9668aef95","Type":"ContainerDied","Data":"938efb7ba76c103453131aede9d23ef77d4fa0a3b67c32e21bc57bbb11dcdbac"} Jun 15 02:35:38.077375 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:38.077349 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-5d99449ffd-9kbhp"] Jun 15 02:35:38.081824 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:38.081804 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5d99449ffd-9kbhp" Jun 15 02:35:38.083870 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:38.083852 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-4p5dt\"" Jun 15 02:35:38.087399 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:38.087376 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5d99449ffd-9kbhp"] Jun 15 02:35:38.105230 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:38.105206 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gflv\" (UniqueName: \"kubernetes.io/projected/b317df48-08bd-4473-898e-e1a965de9965-kube-api-access-2gflv\") pod \"cert-manager-cainjector-5d99449ffd-9kbhp\" (UID: \"b317df48-08bd-4473-898e-e1a965de9965\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-9kbhp" Jun 15 02:35:38.105351 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:38.105253 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b317df48-08bd-4473-898e-e1a965de9965-bound-sa-token\") pod \"cert-manager-cainjector-5d99449ffd-9kbhp\" (UID: \"b317df48-08bd-4473-898e-e1a965de9965\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-9kbhp" Jun 15 02:35:38.206006 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:38.205979 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gflv\" (UniqueName: \"kubernetes.io/projected/b317df48-08bd-4473-898e-e1a965de9965-kube-api-access-2gflv\") pod \"cert-manager-cainjector-5d99449ffd-9kbhp\" (UID: \"b317df48-08bd-4473-898e-e1a965de9965\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-9kbhp" Jun 15 02:35:38.206125 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:38.206025 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b317df48-08bd-4473-898e-e1a965de9965-bound-sa-token\") pod \"cert-manager-cainjector-5d99449ffd-9kbhp\" (UID: \"b317df48-08bd-4473-898e-e1a965de9965\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-9kbhp" Jun 15 02:35:38.213284 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:38.213260 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b317df48-08bd-4473-898e-e1a965de9965-bound-sa-token\") pod \"cert-manager-cainjector-5d99449ffd-9kbhp\" (UID: \"b317df48-08bd-4473-898e-e1a965de9965\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-9kbhp" Jun 15 02:35:38.213431 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:38.213412 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gflv\" (UniqueName: \"kubernetes.io/projected/b317df48-08bd-4473-898e-e1a965de9965-kube-api-access-2gflv\") pod \"cert-manager-cainjector-5d99449ffd-9kbhp\" (UID: \"b317df48-08bd-4473-898e-e1a965de9965\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-9kbhp" Jun 15 02:35:38.417871 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:38.417795 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5d99449ffd-9kbhp" Jun 15 02:35:39.062811 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:39.062778 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqj8rz" event={"ID":"b83c1ca9-56d2-4f27-b07c-05f9668aef95","Type":"ContainerStarted","Data":"fe04bc85aa89adb6959d544ef0cbd9cc99b4f3c1cae99958873937d4b122e2d0"} Jun 15 02:35:39.090476 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:39.090416 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqj8rz" podStartSLOduration=3.155719865 podStartE2EDuration="5.090397497s" totalImagesPullingTime="1.934677632s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 02:35:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 02:35:39.089216099 +0000 UTC m=+431.957000845" watchObservedRunningTime="2026-06-15 02:35:39.090397497 +0000 UTC m=+431.958182241" Jun 15 02:35:39.196175 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:39.196149 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5d99449ffd-9kbhp"] Jun 15 02:35:39.198376 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:35:39.198349 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb317df48_08bd_4473_898e_e1a965de9965.slice/crio-ddfa6d377e487aca1109b45340438c117a4912f0b6fd6cdc719e6d3e1f8d9d1c WatchSource:0}: Error finding container ddfa6d377e487aca1109b45340438c117a4912f0b6fd6cdc719e6d3e1f8d9d1c: Status 404 returned error can't find the container with id ddfa6d377e487aca1109b45340438c117a4912f0b6fd6cdc719e6d3e1f8d9d1c Jun 15 02:35:40.068680 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:40.068645 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-7dc76cbb94-rn7sp" event={"ID":"18654557-1d3c-413a-bb4b-d0bc9e612202","Type":"ContainerStarted","Data":"cf7e809a158b5d78f9cc329663a47e3fa56700e950b4e58ce384662402c49e5a"} Jun 15 02:35:40.069088 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:40.068726 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-7dc76cbb94-rn7sp" Jun 15 02:35:40.070368 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:40.070335 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5d99449ffd-9kbhp" event={"ID":"b317df48-08bd-4473-898e-e1a965de9965","Type":"ContainerStarted","Data":"e8fe3d039c5be68e63a2873187cdc3388d91eb31102542df999a153228416563"} Jun 15 02:35:40.070465 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:40.070380 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5d99449ffd-9kbhp" event={"ID":"b317df48-08bd-4473-898e-e1a965de9965","Type":"ContainerStarted","Data":"ddfa6d377e487aca1109b45340438c117a4912f0b6fd6cdc719e6d3e1f8d9d1c"} Jun 15 02:35:40.072720 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:40.072697 2579 generic.go:356] "Generic (PLEG): container finished" podID="b83c1ca9-56d2-4f27-b07c-05f9668aef95" containerID="fe04bc85aa89adb6959d544ef0cbd9cc99b4f3c1cae99958873937d4b122e2d0" exitCode=0 Jun 15 02:35:40.072809 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:40.072733 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqj8rz" event={"ID":"b83c1ca9-56d2-4f27-b07c-05f9668aef95","Type":"ContainerDied","Data":"fe04bc85aa89adb6959d544ef0cbd9cc99b4f3c1cae99958873937d4b122e2d0"} Jun 15 02:35:40.085114 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:40.085069 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-7dc76cbb94-rn7sp" podStartSLOduration=1.51536205 podStartE2EDuration="5.085055041s" totalImagesPullingTime="3.569692991s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 02:35:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 02:35:40.08368774 +0000 UTC m=+432.951472485" watchObservedRunningTime="2026-06-15 02:35:40.085055041 +0000 UTC m=+432.952839785" Jun 15 02:35:40.102104 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:40.102065 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-5d99449ffd-9kbhp" podStartSLOduration=2.102053336 podStartE2EDuration="2.102053336s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 02:35:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 02:35:40.100869128 +0000 UTC m=+432.968653873" watchObservedRunningTime="2026-06-15 02:35:40.102053336 +0000 UTC m=+432.969838081" Jun 15 02:35:41.196773 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:41.196751 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqj8rz" Jun 15 02:35:41.226950 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:41.226923 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b83c1ca9-56d2-4f27-b07c-05f9668aef95-util\") pod \"b83c1ca9-56d2-4f27-b07c-05f9668aef95\" (UID: \"b83c1ca9-56d2-4f27-b07c-05f9668aef95\") " Jun 15 02:35:41.227071 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:41.226972 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b8zkh\" (UniqueName: \"kubernetes.io/projected/b83c1ca9-56d2-4f27-b07c-05f9668aef95-kube-api-access-b8zkh\") pod \"b83c1ca9-56d2-4f27-b07c-05f9668aef95\" (UID: \"b83c1ca9-56d2-4f27-b07c-05f9668aef95\") " Jun 15 02:35:41.227071 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:41.227040 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b83c1ca9-56d2-4f27-b07c-05f9668aef95-bundle\") pod \"b83c1ca9-56d2-4f27-b07c-05f9668aef95\" (UID: \"b83c1ca9-56d2-4f27-b07c-05f9668aef95\") " Jun 15 02:35:41.227454 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:41.227432 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b83c1ca9-56d2-4f27-b07c-05f9668aef95-bundle" (OuterVolumeSpecName: "bundle") pod "b83c1ca9-56d2-4f27-b07c-05f9668aef95" (UID: "b83c1ca9-56d2-4f27-b07c-05f9668aef95"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 02:35:41.229076 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:41.229053 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b83c1ca9-56d2-4f27-b07c-05f9668aef95-kube-api-access-b8zkh" (OuterVolumeSpecName: "kube-api-access-b8zkh") pod "b83c1ca9-56d2-4f27-b07c-05f9668aef95" (UID: "b83c1ca9-56d2-4f27-b07c-05f9668aef95"). InnerVolumeSpecName "kube-api-access-b8zkh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 02:35:41.231089 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:41.231065 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b83c1ca9-56d2-4f27-b07c-05f9668aef95-util" (OuterVolumeSpecName: "util") pod "b83c1ca9-56d2-4f27-b07c-05f9668aef95" (UID: "b83c1ca9-56d2-4f27-b07c-05f9668aef95"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 02:35:41.327967 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:41.327918 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b8zkh\" (UniqueName: \"kubernetes.io/projected/b83c1ca9-56d2-4f27-b07c-05f9668aef95-kube-api-access-b8zkh\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:35:41.327967 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:41.327939 2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b83c1ca9-56d2-4f27-b07c-05f9668aef95-bundle\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:35:41.327967 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:41.327948 2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b83c1ca9-56d2-4f27-b07c-05f9668aef95-util\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:35:42.082329 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:42.082306 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqj8rz" Jun 15 02:35:42.082477 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:42.082308 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqj8rz" event={"ID":"b83c1ca9-56d2-4f27-b07c-05f9668aef95","Type":"ContainerDied","Data":"85b664c515c26be17319a36e4bf5deb441022dc403f91160ad857a2d0698b81c"} Jun 15 02:35:42.082477 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:42.082418 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="85b664c515c26be17319a36e4bf5deb441022dc403f91160ad857a2d0698b81c" Jun 15 02:35:46.078843 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:46.078810 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-7dc76cbb94-rn7sp" Jun 15 02:35:59.559052 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:59.559023 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5qnwh"] Jun 15 02:35:59.559417 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:59.559289 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b83c1ca9-56d2-4f27-b07c-05f9668aef95" containerName="util" Jun 15 02:35:59.559417 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:59.559299 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b83c1ca9-56d2-4f27-b07c-05f9668aef95" containerName="util" Jun 15 02:35:59.559417 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:59.559309 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b83c1ca9-56d2-4f27-b07c-05f9668aef95" containerName="pull" Jun 15 02:35:59.559417 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:59.559315 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b83c1ca9-56d2-4f27-b07c-05f9668aef95" containerName="pull" Jun 15 02:35:59.559417 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:59.559328 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b83c1ca9-56d2-4f27-b07c-05f9668aef95" containerName="extract" Jun 15 02:35:59.559417 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:59.559333 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b83c1ca9-56d2-4f27-b07c-05f9668aef95" containerName="extract" Jun 15 02:35:59.559417 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:59.559387 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b83c1ca9-56d2-4f27-b07c-05f9668aef95" containerName="extract" Jun 15 02:35:59.563096 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:59.563080 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5qnwh" Jun 15 02:35:59.565158 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:59.565131 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 15 02:35:59.565888 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:59.565869 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-mkns2\"" Jun 15 02:35:59.566001 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:59.565872 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 15 02:35:59.570489 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:59.570469 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5qnwh"] Jun 15 02:35:59.656534 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:59.656513 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3e8dceaa-dc24-4af0-be12-daef4c8f8015-util\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5qnwh\" (UID: \"3e8dceaa-dc24-4af0-be12-daef4c8f8015\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5qnwh" Jun 15 02:35:59.656661 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:59.656557 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3e8dceaa-dc24-4af0-be12-daef4c8f8015-bundle\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5qnwh\" (UID: \"3e8dceaa-dc24-4af0-be12-daef4c8f8015\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5qnwh" Jun 15 02:35:59.656661 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:59.656598 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lt6z\" (UniqueName: \"kubernetes.io/projected/3e8dceaa-dc24-4af0-be12-daef4c8f8015-kube-api-access-6lt6z\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5qnwh\" (UID: \"3e8dceaa-dc24-4af0-be12-daef4c8f8015\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5qnwh" Jun 15 02:35:59.757357 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:59.757335 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3e8dceaa-dc24-4af0-be12-daef4c8f8015-util\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5qnwh\" (UID: \"3e8dceaa-dc24-4af0-be12-daef4c8f8015\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5qnwh" Jun 15 02:35:59.757465 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:59.757379 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3e8dceaa-dc24-4af0-be12-daef4c8f8015-bundle\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5qnwh\" (UID: \"3e8dceaa-dc24-4af0-be12-daef4c8f8015\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5qnwh" Jun 15 02:35:59.757465 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:59.757417 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lt6z\" (UniqueName: \"kubernetes.io/projected/3e8dceaa-dc24-4af0-be12-daef4c8f8015-kube-api-access-6lt6z\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5qnwh\" (UID: \"3e8dceaa-dc24-4af0-be12-daef4c8f8015\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5qnwh" Jun 15 02:35:59.757761 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:59.757743 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3e8dceaa-dc24-4af0-be12-daef4c8f8015-util\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5qnwh\" (UID: \"3e8dceaa-dc24-4af0-be12-daef4c8f8015\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5qnwh" Jun 15 02:35:59.757831 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:59.757812 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3e8dceaa-dc24-4af0-be12-daef4c8f8015-bundle\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5qnwh\" (UID: \"3e8dceaa-dc24-4af0-be12-daef4c8f8015\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5qnwh" Jun 15 02:35:59.764980 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:59.764955 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lt6z\" (UniqueName: \"kubernetes.io/projected/3e8dceaa-dc24-4af0-be12-daef4c8f8015-kube-api-access-6lt6z\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5qnwh\" (UID: \"3e8dceaa-dc24-4af0-be12-daef4c8f8015\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5qnwh" Jun 15 02:35:59.872553 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:59.872497 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5qnwh" Jun 15 02:35:59.997690 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:35:59.997658 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5qnwh"] Jun 15 02:36:00.000265 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:36:00.000237 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e8dceaa_dc24_4af0_be12_daef4c8f8015.slice/crio-5157a00a26d1ca4461a6e53ac54a851c2597994fa2e177d512faad1d1bd8c34d WatchSource:0}: Error finding container 5157a00a26d1ca4461a6e53ac54a851c2597994fa2e177d512faad1d1bd8c34d: Status 404 returned error can't find the container with id 5157a00a26d1ca4461a6e53ac54a851c2597994fa2e177d512faad1d1bd8c34d Jun 15 02:36:00.170510 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:00.170442 2579 generic.go:356] "Generic (PLEG): container finished" podID="3e8dceaa-dc24-4af0-be12-daef4c8f8015" containerID="673fdd1657595b14a3c7fd4914c557235f1a124819278691a5da1e28d7f46cd0" exitCode=0 Jun 15 02:36:00.170510 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:00.170493 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5qnwh" event={"ID":"3e8dceaa-dc24-4af0-be12-daef4c8f8015","Type":"ContainerDied","Data":"673fdd1657595b14a3c7fd4914c557235f1a124819278691a5da1e28d7f46cd0"} Jun 15 02:36:00.170668 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:00.170542 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5qnwh" event={"ID":"3e8dceaa-dc24-4af0-be12-daef4c8f8015","Type":"ContainerStarted","Data":"5157a00a26d1ca4461a6e53ac54a851c2597994fa2e177d512faad1d1bd8c34d"} Jun 15 02:36:01.176159 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:01.176089 2579 generic.go:356] "Generic (PLEG): container finished" podID="3e8dceaa-dc24-4af0-be12-daef4c8f8015" containerID="a7305c09ef369a2847ce09afc6a09418a691b43989161b5f3bc216bd88189249" exitCode=0 Jun 15 02:36:01.176492 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:01.176159 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5qnwh" event={"ID":"3e8dceaa-dc24-4af0-be12-daef4c8f8015","Type":"ContainerDied","Data":"a7305c09ef369a2847ce09afc6a09418a691b43989161b5f3bc216bd88189249"} Jun 15 02:36:02.182846 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:02.182813 2579 generic.go:356] "Generic (PLEG): container finished" podID="3e8dceaa-dc24-4af0-be12-daef4c8f8015" containerID="c327b4ec4d24eb4c3adbf978aa5ebc915d869c23962c470636ebbcb7c7e02449" exitCode=0 Jun 15 02:36:02.183234 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:02.182868 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5qnwh" event={"ID":"3e8dceaa-dc24-4af0-be12-daef4c8f8015","Type":"ContainerDied","Data":"c327b4ec4d24eb4c3adbf978aa5ebc915d869c23962c470636ebbcb7c7e02449"} Jun 15 02:36:03.311077 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:03.311053 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5qnwh" Jun 15 02:36:03.381340 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:03.381314 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3e8dceaa-dc24-4af0-be12-daef4c8f8015-bundle\") pod \"3e8dceaa-dc24-4af0-be12-daef4c8f8015\" (UID: \"3e8dceaa-dc24-4af0-be12-daef4c8f8015\") " Jun 15 02:36:03.381472 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:03.381352 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6lt6z\" (UniqueName: \"kubernetes.io/projected/3e8dceaa-dc24-4af0-be12-daef4c8f8015-kube-api-access-6lt6z\") pod \"3e8dceaa-dc24-4af0-be12-daef4c8f8015\" (UID: \"3e8dceaa-dc24-4af0-be12-daef4c8f8015\") " Jun 15 02:36:03.381472 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:03.381379 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3e8dceaa-dc24-4af0-be12-daef4c8f8015-util\") pod \"3e8dceaa-dc24-4af0-be12-daef4c8f8015\" (UID: \"3e8dceaa-dc24-4af0-be12-daef4c8f8015\") " Jun 15 02:36:03.382070 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:03.382047 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e8dceaa-dc24-4af0-be12-daef4c8f8015-bundle" (OuterVolumeSpecName: "bundle") pod "3e8dceaa-dc24-4af0-be12-daef4c8f8015" (UID: "3e8dceaa-dc24-4af0-be12-daef4c8f8015"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 02:36:03.383391 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:03.383361 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e8dceaa-dc24-4af0-be12-daef4c8f8015-kube-api-access-6lt6z" (OuterVolumeSpecName: "kube-api-access-6lt6z") pod "3e8dceaa-dc24-4af0-be12-daef4c8f8015" (UID: "3e8dceaa-dc24-4af0-be12-daef4c8f8015"). InnerVolumeSpecName "kube-api-access-6lt6z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 02:36:03.386789 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:03.386764 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e8dceaa-dc24-4af0-be12-daef4c8f8015-util" (OuterVolumeSpecName: "util") pod "3e8dceaa-dc24-4af0-be12-daef4c8f8015" (UID: "3e8dceaa-dc24-4af0-be12-daef4c8f8015"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 02:36:03.482555 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:03.482533 2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3e8dceaa-dc24-4af0-be12-daef4c8f8015-util\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:36:03.482555 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:03.482553 2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3e8dceaa-dc24-4af0-be12-daef4c8f8015-bundle\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:36:03.482698 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:03.482562 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6lt6z\" (UniqueName: \"kubernetes.io/projected/3e8dceaa-dc24-4af0-be12-daef4c8f8015-kube-api-access-6lt6z\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:36:04.194215 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:04.194180 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5qnwh" event={"ID":"3e8dceaa-dc24-4af0-be12-daef4c8f8015","Type":"ContainerDied","Data":"5157a00a26d1ca4461a6e53ac54a851c2597994fa2e177d512faad1d1bd8c34d"} Jun 15 02:36:04.194215 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:04.194209 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5qnwh" Jun 15 02:36:04.194419 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:04.194233 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5157a00a26d1ca4461a6e53ac54a851c2597994fa2e177d512faad1d1bd8c34d" Jun 15 02:36:10.804755 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:10.804725 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7d59cb4d96-gj9zl"] Jun 15 02:36:10.805122 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:10.805011 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e8dceaa-dc24-4af0-be12-daef4c8f8015" containerName="pull" Jun 15 02:36:10.805122 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:10.805026 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e8dceaa-dc24-4af0-be12-daef4c8f8015" containerName="pull" Jun 15 02:36:10.805122 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:10.805036 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e8dceaa-dc24-4af0-be12-daef4c8f8015" containerName="util" Jun 15 02:36:10.805122 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:10.805043 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e8dceaa-dc24-4af0-be12-daef4c8f8015" containerName="util" Jun 15 02:36:10.805122 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:10.805059 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e8dceaa-dc24-4af0-be12-daef4c8f8015" containerName="extract" Jun 15 02:36:10.805122 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:10.805065 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e8dceaa-dc24-4af0-be12-daef4c8f8015" containerName="extract" Jun 15 02:36:10.805122 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:10.805120 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e8dceaa-dc24-4af0-be12-daef4c8f8015" containerName="extract" Jun 15 02:36:10.808200 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:10.808181 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7d59cb4d96-gj9zl" Jun 15 02:36:10.811210 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:10.811189 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-md8st\"" Jun 15 02:36:10.811333 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:10.811212 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jun 15 02:36:10.811333 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:10.811267 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jun 15 02:36:10.811333 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:10.811188 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jun 15 02:36:10.811333 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:10.811197 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jun 15 02:36:10.811333 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:10.811196 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jun 15 02:36:10.822884 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:10.822865 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7d59cb4d96-gj9zl"] Jun 15 02:36:10.933675 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:10.933648 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86jlv\" (UniqueName: \"kubernetes.io/projected/bea9d430-60c3-4aca-a77b-8caf8160f58a-kube-api-access-86jlv\") pod \"lws-controller-manager-7d59cb4d96-gj9zl\" (UID: \"bea9d430-60c3-4aca-a77b-8caf8160f58a\") " pod="openshift-lws-operator/lws-controller-manager-7d59cb4d96-gj9zl" Jun 15 02:36:10.933806 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:10.933685 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bea9d430-60c3-4aca-a77b-8caf8160f58a-cert\") pod \"lws-controller-manager-7d59cb4d96-gj9zl\" (UID: \"bea9d430-60c3-4aca-a77b-8caf8160f58a\") " pod="openshift-lws-operator/lws-controller-manager-7d59cb4d96-gj9zl" Jun 15 02:36:10.933806 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:10.933720 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/bea9d430-60c3-4aca-a77b-8caf8160f58a-manager-config\") pod \"lws-controller-manager-7d59cb4d96-gj9zl\" (UID: \"bea9d430-60c3-4aca-a77b-8caf8160f58a\") " pod="openshift-lws-operator/lws-controller-manager-7d59cb4d96-gj9zl" Jun 15 02:36:10.933913 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:10.933825 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/bea9d430-60c3-4aca-a77b-8caf8160f58a-metrics-cert\") pod \"lws-controller-manager-7d59cb4d96-gj9zl\" (UID: \"bea9d430-60c3-4aca-a77b-8caf8160f58a\") " pod="openshift-lws-operator/lws-controller-manager-7d59cb4d96-gj9zl" Jun 15 02:36:11.034921 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:11.034892 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/bea9d430-60c3-4aca-a77b-8caf8160f58a-manager-config\") pod \"lws-controller-manager-7d59cb4d96-gj9zl\" (UID: \"bea9d430-60c3-4aca-a77b-8caf8160f58a\") " pod="openshift-lws-operator/lws-controller-manager-7d59cb4d96-gj9zl" Jun 15 02:36:11.035033 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:11.034954 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/bea9d430-60c3-4aca-a77b-8caf8160f58a-metrics-cert\") pod \"lws-controller-manager-7d59cb4d96-gj9zl\" (UID: \"bea9d430-60c3-4aca-a77b-8caf8160f58a\") " pod="openshift-lws-operator/lws-controller-manager-7d59cb4d96-gj9zl" Jun 15 02:36:11.035033 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:11.035003 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86jlv\" (UniqueName: \"kubernetes.io/projected/bea9d430-60c3-4aca-a77b-8caf8160f58a-kube-api-access-86jlv\") pod \"lws-controller-manager-7d59cb4d96-gj9zl\" (UID: \"bea9d430-60c3-4aca-a77b-8caf8160f58a\") " pod="openshift-lws-operator/lws-controller-manager-7d59cb4d96-gj9zl" Jun 15 02:36:11.035138 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:11.035031 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bea9d430-60c3-4aca-a77b-8caf8160f58a-cert\") pod \"lws-controller-manager-7d59cb4d96-gj9zl\" (UID: \"bea9d430-60c3-4aca-a77b-8caf8160f58a\") " pod="openshift-lws-operator/lws-controller-manager-7d59cb4d96-gj9zl" Jun 15 02:36:11.035506 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:11.035486 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/bea9d430-60c3-4aca-a77b-8caf8160f58a-manager-config\") pod \"lws-controller-manager-7d59cb4d96-gj9zl\" (UID: \"bea9d430-60c3-4aca-a77b-8caf8160f58a\") " pod="openshift-lws-operator/lws-controller-manager-7d59cb4d96-gj9zl" Jun 15 02:36:11.037371 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:11.037350 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bea9d430-60c3-4aca-a77b-8caf8160f58a-cert\") pod \"lws-controller-manager-7d59cb4d96-gj9zl\" (UID: \"bea9d430-60c3-4aca-a77b-8caf8160f58a\") " pod="openshift-lws-operator/lws-controller-manager-7d59cb4d96-gj9zl" Jun 15 02:36:11.037468 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:11.037450 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/bea9d430-60c3-4aca-a77b-8caf8160f58a-metrics-cert\") pod \"lws-controller-manager-7d59cb4d96-gj9zl\" (UID: \"bea9d430-60c3-4aca-a77b-8caf8160f58a\") " pod="openshift-lws-operator/lws-controller-manager-7d59cb4d96-gj9zl" Jun 15 02:36:11.043696 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:11.043676 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-86jlv\" (UniqueName: \"kubernetes.io/projected/bea9d430-60c3-4aca-a77b-8caf8160f58a-kube-api-access-86jlv\") pod \"lws-controller-manager-7d59cb4d96-gj9zl\" (UID: \"bea9d430-60c3-4aca-a77b-8caf8160f58a\") " pod="openshift-lws-operator/lws-controller-manager-7d59cb4d96-gj9zl" Jun 15 02:36:11.117215 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:11.117159 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7d59cb4d96-gj9zl" Jun 15 02:36:11.244815 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:11.244785 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7d59cb4d96-gj9zl"] Jun 15 02:36:11.248315 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:36:11.248287 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbea9d430_60c3_4aca_a77b_8caf8160f58a.slice/crio-afb4f423bc33a5cb7ef1cad9cd521dfd5bf7eebd487e52bdc0070a1451490db1 WatchSource:0}: Error finding container afb4f423bc33a5cb7ef1cad9cd521dfd5bf7eebd487e52bdc0070a1451490db1: Status 404 returned error can't find the container with id afb4f423bc33a5cb7ef1cad9cd521dfd5bf7eebd487e52bdc0070a1451490db1 Jun 15 02:36:11.728773 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:11.728738 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-785bc74785-wd7m6"] Jun 15 02:36:11.733239 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:11.733218 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-785bc74785-wd7m6" Jun 15 02:36:11.735494 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:11.735431 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-f8fh8\"" Jun 15 02:36:11.735609 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:11.735514 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 15 02:36:11.735609 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:11.735554 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 15 02:36:11.735815 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:11.735800 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 15 02:36:11.736197 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:11.736178 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 15 02:36:11.750175 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:11.750157 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-785bc74785-wd7m6"] Jun 15 02:36:11.845038 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:11.845016 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7e1b26b3-7e13-45d8-94d5-9cc30ef55807-webhook-cert\") pod \"opendatahub-operator-controller-manager-785bc74785-wd7m6\" (UID: \"7e1b26b3-7e13-45d8-94d5-9cc30ef55807\") " pod="opendatahub/opendatahub-operator-controller-manager-785bc74785-wd7m6" Jun 15 02:36:11.845383 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:11.845054 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7e1b26b3-7e13-45d8-94d5-9cc30ef55807-manifests\") pod \"opendatahub-operator-controller-manager-785bc74785-wd7m6\" (UID: \"7e1b26b3-7e13-45d8-94d5-9cc30ef55807\") " pod="opendatahub/opendatahub-operator-controller-manager-785bc74785-wd7m6" Jun 15 02:36:11.845383 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:11.845101 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkjrc\" (UniqueName: \"kubernetes.io/projected/7e1b26b3-7e13-45d8-94d5-9cc30ef55807-kube-api-access-mkjrc\") pod \"opendatahub-operator-controller-manager-785bc74785-wd7m6\" (UID: \"7e1b26b3-7e13-45d8-94d5-9cc30ef55807\") " pod="opendatahub/opendatahub-operator-controller-manager-785bc74785-wd7m6" Jun 15 02:36:11.845383 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:11.845167 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7e1b26b3-7e13-45d8-94d5-9cc30ef55807-tmp\") pod \"opendatahub-operator-controller-manager-785bc74785-wd7m6\" (UID: \"7e1b26b3-7e13-45d8-94d5-9cc30ef55807\") " pod="opendatahub/opendatahub-operator-controller-manager-785bc74785-wd7m6" Jun 15 02:36:11.845383 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:11.845196 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7e1b26b3-7e13-45d8-94d5-9cc30ef55807-apiservice-cert\") pod \"opendatahub-operator-controller-manager-785bc74785-wd7m6\" (UID: \"7e1b26b3-7e13-45d8-94d5-9cc30ef55807\") " pod="opendatahub/opendatahub-operator-controller-manager-785bc74785-wd7m6" Jun 15 02:36:11.945507 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:11.945482 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7e1b26b3-7e13-45d8-94d5-9cc30ef55807-tmp\") pod \"opendatahub-operator-controller-manager-785bc74785-wd7m6\" (UID: \"7e1b26b3-7e13-45d8-94d5-9cc30ef55807\") " pod="opendatahub/opendatahub-operator-controller-manager-785bc74785-wd7m6" Jun 15 02:36:11.945604 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:11.945517 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7e1b26b3-7e13-45d8-94d5-9cc30ef55807-apiservice-cert\") pod \"opendatahub-operator-controller-manager-785bc74785-wd7m6\" (UID: \"7e1b26b3-7e13-45d8-94d5-9cc30ef55807\") " pod="opendatahub/opendatahub-operator-controller-manager-785bc74785-wd7m6" Jun 15 02:36:11.945604 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:11.945553 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7e1b26b3-7e13-45d8-94d5-9cc30ef55807-webhook-cert\") pod \"opendatahub-operator-controller-manager-785bc74785-wd7m6\" (UID: \"7e1b26b3-7e13-45d8-94d5-9cc30ef55807\") " pod="opendatahub/opendatahub-operator-controller-manager-785bc74785-wd7m6" Jun 15 02:36:11.945604 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:11.945579 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7e1b26b3-7e13-45d8-94d5-9cc30ef55807-manifests\") pod \"opendatahub-operator-controller-manager-785bc74785-wd7m6\" (UID: \"7e1b26b3-7e13-45d8-94d5-9cc30ef55807\") " pod="opendatahub/opendatahub-operator-controller-manager-785bc74785-wd7m6" Jun 15 02:36:11.945731 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:11.945622 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkjrc\" (UniqueName: \"kubernetes.io/projected/7e1b26b3-7e13-45d8-94d5-9cc30ef55807-kube-api-access-mkjrc\") pod \"opendatahub-operator-controller-manager-785bc74785-wd7m6\" (UID: \"7e1b26b3-7e13-45d8-94d5-9cc30ef55807\") " pod="opendatahub/opendatahub-operator-controller-manager-785bc74785-wd7m6" Jun 15 02:36:11.946012 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:11.945993 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7e1b26b3-7e13-45d8-94d5-9cc30ef55807-manifests\") pod \"opendatahub-operator-controller-manager-785bc74785-wd7m6\" (UID: \"7e1b26b3-7e13-45d8-94d5-9cc30ef55807\") " pod="opendatahub/opendatahub-operator-controller-manager-785bc74785-wd7m6" Jun 15 02:36:11.947943 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:11.947924 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7e1b26b3-7e13-45d8-94d5-9cc30ef55807-tmp\") pod \"opendatahub-operator-controller-manager-785bc74785-wd7m6\" (UID: \"7e1b26b3-7e13-45d8-94d5-9cc30ef55807\") " pod="opendatahub/opendatahub-operator-controller-manager-785bc74785-wd7m6" Jun 15 02:36:11.948026 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:11.947949 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7e1b26b3-7e13-45d8-94d5-9cc30ef55807-apiservice-cert\") pod \"opendatahub-operator-controller-manager-785bc74785-wd7m6\" (UID: \"7e1b26b3-7e13-45d8-94d5-9cc30ef55807\") " pod="opendatahub/opendatahub-operator-controller-manager-785bc74785-wd7m6" Jun 15 02:36:11.948063 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:11.948035 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7e1b26b3-7e13-45d8-94d5-9cc30ef55807-webhook-cert\") pod \"opendatahub-operator-controller-manager-785bc74785-wd7m6\" (UID: \"7e1b26b3-7e13-45d8-94d5-9cc30ef55807\") " pod="opendatahub/opendatahub-operator-controller-manager-785bc74785-wd7m6" Jun 15 02:36:11.954485 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:11.954463 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkjrc\" (UniqueName: \"kubernetes.io/projected/7e1b26b3-7e13-45d8-94d5-9cc30ef55807-kube-api-access-mkjrc\") pod \"opendatahub-operator-controller-manager-785bc74785-wd7m6\" (UID: \"7e1b26b3-7e13-45d8-94d5-9cc30ef55807\") " pod="opendatahub/opendatahub-operator-controller-manager-785bc74785-wd7m6" Jun 15 02:36:12.044101 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:12.044051 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-785bc74785-wd7m6" Jun 15 02:36:12.175494 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:12.175474 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-785bc74785-wd7m6"] Jun 15 02:36:12.178154 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:36:12.178125 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e1b26b3_7e13_45d8_94d5_9cc30ef55807.slice/crio-aa524493954c7f39e98ba89c1789ac4592a751e28d2c26f6fe8e16ec24cf92ee WatchSource:0}: Error finding container aa524493954c7f39e98ba89c1789ac4592a751e28d2c26f6fe8e16ec24cf92ee: Status 404 returned error can't find the container with id aa524493954c7f39e98ba89c1789ac4592a751e28d2c26f6fe8e16ec24cf92ee Jun 15 02:36:12.232887 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:12.232849 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7d59cb4d96-gj9zl" event={"ID":"bea9d430-60c3-4aca-a77b-8caf8160f58a","Type":"ContainerStarted","Data":"afb4f423bc33a5cb7ef1cad9cd521dfd5bf7eebd487e52bdc0070a1451490db1"} Jun 15 02:36:12.234346 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:12.234310 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-785bc74785-wd7m6" event={"ID":"7e1b26b3-7e13-45d8-94d5-9cc30ef55807","Type":"ContainerStarted","Data":"aa524493954c7f39e98ba89c1789ac4592a751e28d2c26f6fe8e16ec24cf92ee"} Jun 15 02:36:14.247656 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:14.247599 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7d59cb4d96-gj9zl" event={"ID":"bea9d430-60c3-4aca-a77b-8caf8160f58a","Type":"ContainerStarted","Data":"ba5b5df91a60926c37f8b56d12863ad681acfdb16616c1ec8a90a6cec743feae"} Jun 15 02:36:14.248088 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:14.247682 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7d59cb4d96-gj9zl" Jun 15 02:36:14.265206 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:14.265160 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7d59cb4d96-gj9zl" podStartSLOduration=2.180233285 podStartE2EDuration="4.265142886s" totalImagesPullingTime="2.084909601s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 02:36:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 02:36:14.26427528 +0000 UTC m=+467.132060027" watchObservedRunningTime="2026-06-15 02:36:14.265142886 +0000 UTC m=+467.132927632" Jun 15 02:36:15.253710 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:15.253674 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-785bc74785-wd7m6" event={"ID":"7e1b26b3-7e13-45d8-94d5-9cc30ef55807","Type":"ContainerStarted","Data":"c32154448a4928cb088d6097992561bf66295dbf04026c2728e138995bb749f4"} Jun 15 02:36:16.260591 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:16.260558 2579 generic.go:356] "Generic (PLEG): container finished" podID="7e1b26b3-7e13-45d8-94d5-9cc30ef55807" containerID="c32154448a4928cb088d6097992561bf66295dbf04026c2728e138995bb749f4" exitCode=0 Jun 15 02:36:16.260908 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:16.260656 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-785bc74785-wd7m6" event={"ID":"7e1b26b3-7e13-45d8-94d5-9cc30ef55807","Type":"ContainerDied","Data":"c32154448a4928cb088d6097992561bf66295dbf04026c2728e138995bb749f4"} Jun 15 02:36:17.267643 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:17.267607 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-785bc74785-wd7m6" event={"ID":"7e1b26b3-7e13-45d8-94d5-9cc30ef55807","Type":"ContainerStarted","Data":"0b52c737237a405ee75094204891e37ef1b7c1ab7e9e8cbfa12be0abde42b929"} Jun 15 02:36:17.268122 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:17.267758 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-785bc74785-wd7m6" Jun 15 02:36:17.288009 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:17.287962 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-785bc74785-wd7m6" podStartSLOduration=2.199500665 podStartE2EDuration="6.287945498s" totalImagesPullingTime="3.146467198s" totalInitContainerRuntime="941.977635ms" isStatefulPod=false podCreationTimestamp="2026-06-15 02:36:11 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 02:36:17.286329338 +0000 UTC m=+470.154114084" watchObservedRunningTime="2026-06-15 02:36:17.287945498 +0000 UTC m=+470.155730242" Jun 15 02:36:25.256575 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:25.256544 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7d59cb4d96-gj9zl" Jun 15 02:36:28.274248 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:28.274217 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-785bc74785-wd7m6" Jun 15 02:36:33.664968 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:33.664934 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-59c549f7b4-gpxhv"] Jun 15 02:36:33.678756 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:33.678730 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-59c549f7b4-gpxhv" Jun 15 02:36:33.679435 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:33.679396 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-59c549f7b4-gpxhv"] Jun 15 02:36:33.680906 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:33.680868 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jun 15 02:36:33.681058 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:33.681039 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-5fqzr\"" Jun 15 02:36:33.681129 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:33.681116 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 15 02:36:33.681817 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:33.681737 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jun 15 02:36:33.681817 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:33.681775 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 15 02:36:33.809855 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:33.809826 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bd72927c-a9ed-46a5-879a-b86ddfce6a3e-tmp\") pod \"kube-auth-proxy-59c549f7b4-gpxhv\" (UID: \"bd72927c-a9ed-46a5-879a-b86ddfce6a3e\") " pod="openshift-ingress/kube-auth-proxy-59c549f7b4-gpxhv" Jun 15 02:36:33.809855 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:33.809859 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd72927c-a9ed-46a5-879a-b86ddfce6a3e-tls-certs\") pod \"kube-auth-proxy-59c549f7b4-gpxhv\" (UID: \"bd72927c-a9ed-46a5-879a-b86ddfce6a3e\") " pod="openshift-ingress/kube-auth-proxy-59c549f7b4-gpxhv" Jun 15 02:36:33.810074 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:33.809914 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsg2s\" (UniqueName: \"kubernetes.io/projected/bd72927c-a9ed-46a5-879a-b86ddfce6a3e-kube-api-access-bsg2s\") pod \"kube-auth-proxy-59c549f7b4-gpxhv\" (UID: \"bd72927c-a9ed-46a5-879a-b86ddfce6a3e\") " pod="openshift-ingress/kube-auth-proxy-59c549f7b4-gpxhv" Jun 15 02:36:33.910402 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:33.910374 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bd72927c-a9ed-46a5-879a-b86ddfce6a3e-tmp\") pod \"kube-auth-proxy-59c549f7b4-gpxhv\" (UID: \"bd72927c-a9ed-46a5-879a-b86ddfce6a3e\") " pod="openshift-ingress/kube-auth-proxy-59c549f7b4-gpxhv" Jun 15 02:36:33.910402 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:33.910408 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd72927c-a9ed-46a5-879a-b86ddfce6a3e-tls-certs\") pod \"kube-auth-proxy-59c549f7b4-gpxhv\" (UID: \"bd72927c-a9ed-46a5-879a-b86ddfce6a3e\") " pod="openshift-ingress/kube-auth-proxy-59c549f7b4-gpxhv" Jun 15 02:36:33.910604 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:33.910431 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsg2s\" (UniqueName: \"kubernetes.io/projected/bd72927c-a9ed-46a5-879a-b86ddfce6a3e-kube-api-access-bsg2s\") pod \"kube-auth-proxy-59c549f7b4-gpxhv\" (UID: \"bd72927c-a9ed-46a5-879a-b86ddfce6a3e\") " pod="openshift-ingress/kube-auth-proxy-59c549f7b4-gpxhv" Jun 15 02:36:33.910604 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:36:33.910540 2579 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jun 15 02:36:33.910604 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:36:33.910602 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd72927c-a9ed-46a5-879a-b86ddfce6a3e-tls-certs podName:bd72927c-a9ed-46a5-879a-b86ddfce6a3e nodeName:}" failed. No retries permitted until 2026-06-15 02:36:34.410583532 +0000 UTC m=+487.278368255 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/bd72927c-a9ed-46a5-879a-b86ddfce6a3e-tls-certs") pod "kube-auth-proxy-59c549f7b4-gpxhv" (UID: "bd72927c-a9ed-46a5-879a-b86ddfce6a3e") : secret "kube-auth-proxy-tls" not found Jun 15 02:36:33.912522 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:33.912495 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bd72927c-a9ed-46a5-879a-b86ddfce6a3e-tmp\") pod \"kube-auth-proxy-59c549f7b4-gpxhv\" (UID: \"bd72927c-a9ed-46a5-879a-b86ddfce6a3e\") " pod="openshift-ingress/kube-auth-proxy-59c549f7b4-gpxhv" Jun 15 02:36:33.920061 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:33.920009 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsg2s\" (UniqueName: \"kubernetes.io/projected/bd72927c-a9ed-46a5-879a-b86ddfce6a3e-kube-api-access-bsg2s\") pod \"kube-auth-proxy-59c549f7b4-gpxhv\" (UID: \"bd72927c-a9ed-46a5-879a-b86ddfce6a3e\") " pod="openshift-ingress/kube-auth-proxy-59c549f7b4-gpxhv" Jun 15 02:36:34.415291 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:34.415260 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd72927c-a9ed-46a5-879a-b86ddfce6a3e-tls-certs\") pod \"kube-auth-proxy-59c549f7b4-gpxhv\" (UID: \"bd72927c-a9ed-46a5-879a-b86ddfce6a3e\") " pod="openshift-ingress/kube-auth-proxy-59c549f7b4-gpxhv" Jun 15 02:36:34.417672 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:34.417645 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd72927c-a9ed-46a5-879a-b86ddfce6a3e-tls-certs\") pod \"kube-auth-proxy-59c549f7b4-gpxhv\" (UID: \"bd72927c-a9ed-46a5-879a-b86ddfce6a3e\") " pod="openshift-ingress/kube-auth-proxy-59c549f7b4-gpxhv" Jun 15 02:36:34.590394 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:34.590358 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-59c549f7b4-gpxhv" Jun 15 02:36:34.742255 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:34.742224 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-59c549f7b4-gpxhv"] Jun 15 02:36:34.744450 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:36:34.744421 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd72927c_a9ed_46a5_879a_b86ddfce6a3e.slice/crio-1c82cb07f381a1b4f8752addb8458abc7c43ce054605c05be38f320544a557b5 WatchSource:0}: Error finding container 1c82cb07f381a1b4f8752addb8458abc7c43ce054605c05be38f320544a557b5: Status 404 returned error can't find the container with id 1c82cb07f381a1b4f8752addb8458abc7c43ce054605c05be38f320544a557b5 Jun 15 02:36:35.276492 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:35.276441 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06qckg4"] Jun 15 02:36:35.280544 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:35.280510 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06qckg4" Jun 15 02:36:35.282888 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:35.282861 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 15 02:36:35.283045 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:35.283026 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 15 02:36:35.283704 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:35.283620 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-mkns2\"" Jun 15 02:36:35.287279 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:35.287227 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06qckg4"] Jun 15 02:36:35.358127 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:35.358087 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-59c549f7b4-gpxhv" event={"ID":"bd72927c-a9ed-46a5-879a-b86ddfce6a3e","Type":"ContainerStarted","Data":"1c82cb07f381a1b4f8752addb8458abc7c43ce054605c05be38f320544a557b5"} Jun 15 02:36:35.423350 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:35.423313 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzvz2\" (UniqueName: \"kubernetes.io/projected/c19332c0-aaa6-4f86-9fe3-1fbd8b4b08cb-kube-api-access-bzvz2\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06qckg4\" (UID: \"c19332c0-aaa6-4f86-9fe3-1fbd8b4b08cb\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06qckg4" Jun 15 02:36:35.423536 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:35.423372 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c19332c0-aaa6-4f86-9fe3-1fbd8b4b08cb-bundle\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06qckg4\" (UID: \"c19332c0-aaa6-4f86-9fe3-1fbd8b4b08cb\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06qckg4" Jun 15 02:36:35.423536 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:35.423489 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c19332c0-aaa6-4f86-9fe3-1fbd8b4b08cb-util\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06qckg4\" (UID: \"c19332c0-aaa6-4f86-9fe3-1fbd8b4b08cb\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06qckg4" Jun 15 02:36:35.524066 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:35.524023 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c19332c0-aaa6-4f86-9fe3-1fbd8b4b08cb-util\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06qckg4\" (UID: \"c19332c0-aaa6-4f86-9fe3-1fbd8b4b08cb\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06qckg4" Jun 15 02:36:35.524066 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:35.524064 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzvz2\" (UniqueName: \"kubernetes.io/projected/c19332c0-aaa6-4f86-9fe3-1fbd8b4b08cb-kube-api-access-bzvz2\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06qckg4\" (UID: \"c19332c0-aaa6-4f86-9fe3-1fbd8b4b08cb\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06qckg4" Jun 15 02:36:35.524309 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:35.524099 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c19332c0-aaa6-4f86-9fe3-1fbd8b4b08cb-bundle\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06qckg4\" (UID: \"c19332c0-aaa6-4f86-9fe3-1fbd8b4b08cb\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06qckg4" Jun 15 02:36:35.524524 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:35.524478 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c19332c0-aaa6-4f86-9fe3-1fbd8b4b08cb-bundle\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06qckg4\" (UID: \"c19332c0-aaa6-4f86-9fe3-1fbd8b4b08cb\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06qckg4" Jun 15 02:36:35.524717 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:35.524683 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c19332c0-aaa6-4f86-9fe3-1fbd8b4b08cb-util\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06qckg4\" (UID: \"c19332c0-aaa6-4f86-9fe3-1fbd8b4b08cb\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06qckg4" Jun 15 02:36:35.536597 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:35.536528 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzvz2\" (UniqueName: \"kubernetes.io/projected/c19332c0-aaa6-4f86-9fe3-1fbd8b4b08cb-kube-api-access-bzvz2\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06qckg4\" (UID: \"c19332c0-aaa6-4f86-9fe3-1fbd8b4b08cb\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06qckg4" Jun 15 02:36:35.593576 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:35.593537 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06qckg4" Jun 15 02:36:35.753252 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:35.753174 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06qckg4"] Jun 15 02:36:35.758287 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:36:35.758256 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc19332c0_aaa6_4f86_9fe3_1fbd8b4b08cb.slice/crio-6ee35014274c6fa81546bb8f055fea1df8ca7ddc185dec0e815e624756d5add1 WatchSource:0}: Error finding container 6ee35014274c6fa81546bb8f055fea1df8ca7ddc185dec0e815e624756d5add1: Status 404 returned error can't find the container with id 6ee35014274c6fa81546bb8f055fea1df8ca7ddc185dec0e815e624756d5add1 Jun 15 02:36:36.367395 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:36.367225 2579 generic.go:356] "Generic (PLEG): container finished" podID="c19332c0-aaa6-4f86-9fe3-1fbd8b4b08cb" containerID="daff250f15a8936eb5934555538ff358e62a0d68b88f2d324d145c029d2a773b" exitCode=0 Jun 15 02:36:36.367395 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:36.367322 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06qckg4" event={"ID":"c19332c0-aaa6-4f86-9fe3-1fbd8b4b08cb","Type":"ContainerDied","Data":"daff250f15a8936eb5934555538ff358e62a0d68b88f2d324d145c029d2a773b"} Jun 15 02:36:36.367395 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:36.367364 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06qckg4" event={"ID":"c19332c0-aaa6-4f86-9fe3-1fbd8b4b08cb","Type":"ContainerStarted","Data":"6ee35014274c6fa81546bb8f055fea1df8ca7ddc185dec0e815e624756d5add1"} Jun 15 02:36:38.380685 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:38.380627 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-59c549f7b4-gpxhv" event={"ID":"bd72927c-a9ed-46a5-879a-b86ddfce6a3e","Type":"ContainerStarted","Data":"3d6ebc761fdd9d7e01ff98f645413dd1abf3df0de3020223d70ac63f02d0be74"} Jun 15 02:36:38.383049 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:38.383008 2579 generic.go:356] "Generic (PLEG): container finished" podID="c19332c0-aaa6-4f86-9fe3-1fbd8b4b08cb" containerID="24811ccd39f1fde377e95b1af54e57fcbfa6d543b1781b592dc3b4267f51a778" exitCode=0 Jun 15 02:36:38.383184 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:38.383081 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06qckg4" event={"ID":"c19332c0-aaa6-4f86-9fe3-1fbd8b4b08cb","Type":"ContainerDied","Data":"24811ccd39f1fde377e95b1af54e57fcbfa6d543b1781b592dc3b4267f51a778"} Jun 15 02:36:38.396017 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:38.395968 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-59c549f7b4-gpxhv" podStartSLOduration=2.29720345 podStartE2EDuration="5.395950772s" totalImagesPullingTime="3.098747322s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 02:36:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 02:36:38.39509651 +0000 UTC m=+491.262881265" watchObservedRunningTime="2026-06-15 02:36:38.395950772 +0000 UTC m=+491.263735519" Jun 15 02:36:39.389889 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:39.389853 2579 generic.go:356] "Generic (PLEG): container finished" podID="c19332c0-aaa6-4f86-9fe3-1fbd8b4b08cb" containerID="e11eb65bb2b6d5b7e89627f9ee39c58ddca09b9db54863022eb244358a436e5b" exitCode=0 Jun 15 02:36:39.390292 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:39.389899 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06qckg4" event={"ID":"c19332c0-aaa6-4f86-9fe3-1fbd8b4b08cb","Type":"ContainerDied","Data":"e11eb65bb2b6d5b7e89627f9ee39c58ddca09b9db54863022eb244358a436e5b"} Jun 15 02:36:40.523072 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:40.523045 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06qckg4" Jun 15 02:36:40.672179 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:40.672104 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bzvz2\" (UniqueName: \"kubernetes.io/projected/c19332c0-aaa6-4f86-9fe3-1fbd8b4b08cb-kube-api-access-bzvz2\") pod \"c19332c0-aaa6-4f86-9fe3-1fbd8b4b08cb\" (UID: \"c19332c0-aaa6-4f86-9fe3-1fbd8b4b08cb\") " Jun 15 02:36:40.672179 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:40.672142 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c19332c0-aaa6-4f86-9fe3-1fbd8b4b08cb-util\") pod \"c19332c0-aaa6-4f86-9fe3-1fbd8b4b08cb\" (UID: \"c19332c0-aaa6-4f86-9fe3-1fbd8b4b08cb\") " Jun 15 02:36:40.672179 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:40.672162 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c19332c0-aaa6-4f86-9fe3-1fbd8b4b08cb-bundle\") pod \"c19332c0-aaa6-4f86-9fe3-1fbd8b4b08cb\" (UID: \"c19332c0-aaa6-4f86-9fe3-1fbd8b4b08cb\") " Jun 15 02:36:40.672950 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:40.672925 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c19332c0-aaa6-4f86-9fe3-1fbd8b4b08cb-bundle" (OuterVolumeSpecName: "bundle") pod "c19332c0-aaa6-4f86-9fe3-1fbd8b4b08cb" (UID: "c19332c0-aaa6-4f86-9fe3-1fbd8b4b08cb"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 02:36:40.674164 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:40.674142 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c19332c0-aaa6-4f86-9fe3-1fbd8b4b08cb-kube-api-access-bzvz2" (OuterVolumeSpecName: "kube-api-access-bzvz2") pod "c19332c0-aaa6-4f86-9fe3-1fbd8b4b08cb" (UID: "c19332c0-aaa6-4f86-9fe3-1fbd8b4b08cb"). InnerVolumeSpecName "kube-api-access-bzvz2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 02:36:40.677288 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:40.677266 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c19332c0-aaa6-4f86-9fe3-1fbd8b4b08cb-util" (OuterVolumeSpecName: "util") pod "c19332c0-aaa6-4f86-9fe3-1fbd8b4b08cb" (UID: "c19332c0-aaa6-4f86-9fe3-1fbd8b4b08cb"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 02:36:40.773390 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:40.773361 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bzvz2\" (UniqueName: \"kubernetes.io/projected/c19332c0-aaa6-4f86-9fe3-1fbd8b4b08cb-kube-api-access-bzvz2\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:36:40.773390 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:40.773386 2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c19332c0-aaa6-4f86-9fe3-1fbd8b4b08cb-util\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:36:40.773523 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:40.773398 2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c19332c0-aaa6-4f86-9fe3-1fbd8b4b08cb-bundle\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:36:41.402142 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:41.402109 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06qckg4" event={"ID":"c19332c0-aaa6-4f86-9fe3-1fbd8b4b08cb","Type":"ContainerDied","Data":"6ee35014274c6fa81546bb8f055fea1df8ca7ddc185dec0e815e624756d5add1"} Jun 15 02:36:41.402302 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:41.402154 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6ee35014274c6fa81546bb8f055fea1df8ca7ddc185dec0e815e624756d5add1" Jun 15 02:36:41.402302 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:41.402155 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06qckg4" Jun 15 02:36:49.995846 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:49.995811 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aflsk5"] Jun 15 02:36:49.996478 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:49.996107 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c19332c0-aaa6-4f86-9fe3-1fbd8b4b08cb" containerName="util" Jun 15 02:36:49.996478 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:49.996117 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c19332c0-aaa6-4f86-9fe3-1fbd8b4b08cb" containerName="util" Jun 15 02:36:49.996478 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:49.996128 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c19332c0-aaa6-4f86-9fe3-1fbd8b4b08cb" containerName="pull" Jun 15 02:36:49.996478 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:49.996133 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c19332c0-aaa6-4f86-9fe3-1fbd8b4b08cb" containerName="pull" Jun 15 02:36:49.996478 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:49.996154 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c19332c0-aaa6-4f86-9fe3-1fbd8b4b08cb" containerName="extract" Jun 15 02:36:49.996478 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:49.996160 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c19332c0-aaa6-4f86-9fe3-1fbd8b4b08cb" containerName="extract" Jun 15 02:36:49.996478 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:49.996204 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c19332c0-aaa6-4f86-9fe3-1fbd8b4b08cb" containerName="extract" Jun 15 02:36:49.999436 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:49.999417 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aflsk5" Jun 15 02:36:50.001934 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:50.001913 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 15 02:36:50.002171 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:50.002154 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-mkns2\"" Jun 15 02:36:50.002765 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:50.002748 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 15 02:36:50.009213 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:50.009192 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aflsk5"] Jun 15 02:36:50.042494 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:50.042475 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ba995dbd-ea11-4df4-ba0c-6ab4287bbd61-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aflsk5\" (UID: \"ba995dbd-ea11-4df4-ba0c-6ab4287bbd61\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aflsk5" Jun 15 02:36:50.042606 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:50.042513 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ba995dbd-ea11-4df4-ba0c-6ab4287bbd61-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aflsk5\" (UID: \"ba995dbd-ea11-4df4-ba0c-6ab4287bbd61\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aflsk5" Jun 15 02:36:50.042606 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:50.042593 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9t8d\" (UniqueName: \"kubernetes.io/projected/ba995dbd-ea11-4df4-ba0c-6ab4287bbd61-kube-api-access-b9t8d\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aflsk5\" (UID: \"ba995dbd-ea11-4df4-ba0c-6ab4287bbd61\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aflsk5" Jun 15 02:36:50.143365 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:50.143340 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9t8d\" (UniqueName: \"kubernetes.io/projected/ba995dbd-ea11-4df4-ba0c-6ab4287bbd61-kube-api-access-b9t8d\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aflsk5\" (UID: \"ba995dbd-ea11-4df4-ba0c-6ab4287bbd61\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aflsk5" Jun 15 02:36:50.143469 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:50.143385 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ba995dbd-ea11-4df4-ba0c-6ab4287bbd61-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aflsk5\" (UID: \"ba995dbd-ea11-4df4-ba0c-6ab4287bbd61\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aflsk5" Jun 15 02:36:50.143525 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:50.143506 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ba995dbd-ea11-4df4-ba0c-6ab4287bbd61-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aflsk5\" (UID: \"ba995dbd-ea11-4df4-ba0c-6ab4287bbd61\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aflsk5" Jun 15 02:36:50.143790 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:50.143771 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ba995dbd-ea11-4df4-ba0c-6ab4287bbd61-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aflsk5\" (UID: \"ba995dbd-ea11-4df4-ba0c-6ab4287bbd61\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aflsk5" Jun 15 02:36:50.143837 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:50.143804 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ba995dbd-ea11-4df4-ba0c-6ab4287bbd61-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aflsk5\" (UID: \"ba995dbd-ea11-4df4-ba0c-6ab4287bbd61\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aflsk5" Jun 15 02:36:50.153757 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:50.153737 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9t8d\" (UniqueName: \"kubernetes.io/projected/ba995dbd-ea11-4df4-ba0c-6ab4287bbd61-kube-api-access-b9t8d\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aflsk5\" (UID: \"ba995dbd-ea11-4df4-ba0c-6ab4287bbd61\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aflsk5" Jun 15 02:36:50.309252 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:50.309193 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aflsk5" Jun 15 02:36:50.440201 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:50.440180 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aflsk5"] Jun 15 02:36:50.442574 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:36:50.442551 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba995dbd_ea11_4df4_ba0c_6ab4287bbd61.slice/crio-b81d569721f1ee704f3f3a18136b90859899214420c66d2b69b1af9c856f7416 WatchSource:0}: Error finding container b81d569721f1ee704f3f3a18136b90859899214420c66d2b69b1af9c856f7416: Status 404 returned error can't find the container with id b81d569721f1ee704f3f3a18136b90859899214420c66d2b69b1af9c856f7416 Jun 15 02:36:51.449246 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:51.449169 2579 generic.go:356] "Generic (PLEG): container finished" podID="ba995dbd-ea11-4df4-ba0c-6ab4287bbd61" containerID="01e98adeb27f522354795a936c54a0bf7fbae8cae47d50e49190e006696c8dc9" exitCode=0 Jun 15 02:36:51.449620 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:51.449263 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aflsk5" event={"ID":"ba995dbd-ea11-4df4-ba0c-6ab4287bbd61","Type":"ContainerDied","Data":"01e98adeb27f522354795a936c54a0bf7fbae8cae47d50e49190e006696c8dc9"} Jun 15 02:36:51.449620 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:51.449326 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aflsk5" event={"ID":"ba995dbd-ea11-4df4-ba0c-6ab4287bbd61","Type":"ContainerStarted","Data":"b81d569721f1ee704f3f3a18136b90859899214420c66d2b69b1af9c856f7416"} Jun 15 02:36:52.455392 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:52.455359 2579 generic.go:356] "Generic (PLEG): container finished" podID="ba995dbd-ea11-4df4-ba0c-6ab4287bbd61" containerID="1e51560240ab0653872434b66deb7f8af03b70528b60ea092e19dc90aefbf72e" exitCode=0 Jun 15 02:36:52.455815 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:52.455421 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aflsk5" event={"ID":"ba995dbd-ea11-4df4-ba0c-6ab4287bbd61","Type":"ContainerDied","Data":"1e51560240ab0653872434b66deb7f8af03b70528b60ea092e19dc90aefbf72e"} Jun 15 02:36:53.461837 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:53.461807 2579 generic.go:356] "Generic (PLEG): container finished" podID="ba995dbd-ea11-4df4-ba0c-6ab4287bbd61" containerID="312cdff440ddee0e7b8ef0c3c44b082e6c4e916011f9cbba9222d802babfad3a" exitCode=0 Jun 15 02:36:53.462198 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:53.461861 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aflsk5" event={"ID":"ba995dbd-ea11-4df4-ba0c-6ab4287bbd61","Type":"ContainerDied","Data":"312cdff440ddee0e7b8ef0c3c44b082e6c4e916011f9cbba9222d802babfad3a"} Jun 15 02:36:54.597885 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:54.597863 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aflsk5" Jun 15 02:36:54.677452 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:54.677424 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ba995dbd-ea11-4df4-ba0c-6ab4287bbd61-bundle\") pod \"ba995dbd-ea11-4df4-ba0c-6ab4287bbd61\" (UID: \"ba995dbd-ea11-4df4-ba0c-6ab4287bbd61\") " Jun 15 02:36:54.677574 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:54.677467 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b9t8d\" (UniqueName: \"kubernetes.io/projected/ba995dbd-ea11-4df4-ba0c-6ab4287bbd61-kube-api-access-b9t8d\") pod \"ba995dbd-ea11-4df4-ba0c-6ab4287bbd61\" (UID: \"ba995dbd-ea11-4df4-ba0c-6ab4287bbd61\") " Jun 15 02:36:54.677574 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:54.677493 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ba995dbd-ea11-4df4-ba0c-6ab4287bbd61-util\") pod \"ba995dbd-ea11-4df4-ba0c-6ab4287bbd61\" (UID: \"ba995dbd-ea11-4df4-ba0c-6ab4287bbd61\") " Jun 15 02:36:54.678348 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:54.678317 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba995dbd-ea11-4df4-ba0c-6ab4287bbd61-bundle" (OuterVolumeSpecName: "bundle") pod "ba995dbd-ea11-4df4-ba0c-6ab4287bbd61" (UID: "ba995dbd-ea11-4df4-ba0c-6ab4287bbd61"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 02:36:54.679668 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:54.679615 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba995dbd-ea11-4df4-ba0c-6ab4287bbd61-kube-api-access-b9t8d" (OuterVolumeSpecName: "kube-api-access-b9t8d") pod "ba995dbd-ea11-4df4-ba0c-6ab4287bbd61" (UID: "ba995dbd-ea11-4df4-ba0c-6ab4287bbd61"). InnerVolumeSpecName "kube-api-access-b9t8d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 02:36:54.682832 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:54.682797 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba995dbd-ea11-4df4-ba0c-6ab4287bbd61-util" (OuterVolumeSpecName: "util") pod "ba995dbd-ea11-4df4-ba0c-6ab4287bbd61" (UID: "ba995dbd-ea11-4df4-ba0c-6ab4287bbd61"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 02:36:54.778591 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:54.778565 2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ba995dbd-ea11-4df4-ba0c-6ab4287bbd61-bundle\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:36:54.778591 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:54.778588 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b9t8d\" (UniqueName: \"kubernetes.io/projected/ba995dbd-ea11-4df4-ba0c-6ab4287bbd61-kube-api-access-b9t8d\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:36:54.778795 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:54.778598 2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ba995dbd-ea11-4df4-ba0c-6ab4287bbd61-util\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:36:55.473792 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:55.473766 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aflsk5" Jun 15 02:36:55.473953 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:55.473758 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762aflsk5" event={"ID":"ba995dbd-ea11-4df4-ba0c-6ab4287bbd61","Type":"ContainerDied","Data":"b81d569721f1ee704f3f3a18136b90859899214420c66d2b69b1af9c856f7416"} Jun 15 02:36:55.473953 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:36:55.473890 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b81d569721f1ee704f3f3a18136b90859899214420c66d2b69b1af9c856f7416" Jun 15 02:37:47.754925 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:47.754896 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gq984"] Jun 15 02:37:47.755335 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:47.755197 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba995dbd-ea11-4df4-ba0c-6ab4287bbd61" containerName="util" Jun 15 02:37:47.755335 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:47.755209 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba995dbd-ea11-4df4-ba0c-6ab4287bbd61" containerName="util" Jun 15 02:37:47.755335 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:47.755226 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba995dbd-ea11-4df4-ba0c-6ab4287bbd61" containerName="pull" Jun 15 02:37:47.755335 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:47.755231 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba995dbd-ea11-4df4-ba0c-6ab4287bbd61" containerName="pull" Jun 15 02:37:47.755335 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:47.755237 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba995dbd-ea11-4df4-ba0c-6ab4287bbd61" containerName="extract" Jun 15 02:37:47.755335 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:47.755242 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba995dbd-ea11-4df4-ba0c-6ab4287bbd61" containerName="extract" Jun 15 02:37:47.755335 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:47.755299 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba995dbd-ea11-4df4-ba0c-6ab4287bbd61" containerName="extract" Jun 15 02:37:47.758283 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:47.758268 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gq984" Jun 15 02:37:47.760354 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:47.760329 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 15 02:37:47.760491 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:47.760468 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 15 02:37:47.760568 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:47.760522 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-jxn47\"" Jun 15 02:37:47.765835 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:47.765803 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gq984"] Jun 15 02:37:47.865722 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:47.865699 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5f0a46ca-b767-486a-b97e-f3b0f34098c4-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gq984\" (UID: \"5f0a46ca-b767-486a-b97e-f3b0f34098c4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gq984" Jun 15 02:37:47.865845 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:47.865728 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qph74\" (UniqueName: \"kubernetes.io/projected/5f0a46ca-b767-486a-b97e-f3b0f34098c4-kube-api-access-qph74\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gq984\" (UID: \"5f0a46ca-b767-486a-b97e-f3b0f34098c4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gq984" Jun 15 02:37:47.865845 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:47.865755 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5f0a46ca-b767-486a-b97e-f3b0f34098c4-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gq984\" (UID: \"5f0a46ca-b767-486a-b97e-f3b0f34098c4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gq984" Jun 15 02:37:47.966892 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:47.966869 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5f0a46ca-b767-486a-b97e-f3b0f34098c4-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gq984\" (UID: \"5f0a46ca-b767-486a-b97e-f3b0f34098c4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gq984" Jun 15 02:37:47.966987 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:47.966897 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qph74\" (UniqueName: \"kubernetes.io/projected/5f0a46ca-b767-486a-b97e-f3b0f34098c4-kube-api-access-qph74\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gq984\" (UID: \"5f0a46ca-b767-486a-b97e-f3b0f34098c4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gq984" Jun 15 02:37:47.966987 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:47.966933 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5f0a46ca-b767-486a-b97e-f3b0f34098c4-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gq984\" (UID: \"5f0a46ca-b767-486a-b97e-f3b0f34098c4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gq984" Jun 15 02:37:47.967278 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:47.967259 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5f0a46ca-b767-486a-b97e-f3b0f34098c4-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gq984\" (UID: \"5f0a46ca-b767-486a-b97e-f3b0f34098c4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gq984" Jun 15 02:37:47.967333 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:47.967273 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5f0a46ca-b767-486a-b97e-f3b0f34098c4-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gq984\" (UID: \"5f0a46ca-b767-486a-b97e-f3b0f34098c4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gq984" Jun 15 02:37:47.974648 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:47.974599 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qph74\" (UniqueName: \"kubernetes.io/projected/5f0a46ca-b767-486a-b97e-f3b0f34098c4-kube-api-access-qph74\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gq984\" (UID: \"5f0a46ca-b767-486a-b97e-f3b0f34098c4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gq984" Jun 15 02:37:48.068140 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:48.068090 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gq984" Jun 15 02:37:48.201973 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:48.201947 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gq984"] Jun 15 02:37:48.204142 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:37:48.204114 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f0a46ca_b767_486a_b97e_f3b0f34098c4.slice/crio-00ee895ac6861a1342b342da94a1569ed54e26b36bacb3fc05d2254c3a48dfc9 WatchSource:0}: Error finding container 00ee895ac6861a1342b342da94a1569ed54e26b36bacb3fc05d2254c3a48dfc9: Status 404 returned error can't find the container with id 00ee895ac6861a1342b342da94a1569ed54e26b36bacb3fc05d2254c3a48dfc9 Jun 15 02:37:48.355781 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:48.355727 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t6zwb"] Jun 15 02:37:48.359033 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:48.359014 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t6zwb" Jun 15 02:37:48.366054 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:48.366035 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t6zwb"] Jun 15 02:37:48.472121 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:48.472094 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/48c71614-5ca4-40b8-8ac2-fed4e8989831-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t6zwb\" (UID: \"48c71614-5ca4-40b8-8ac2-fed4e8989831\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t6zwb" Jun 15 02:37:48.472218 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:48.472129 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/48c71614-5ca4-40b8-8ac2-fed4e8989831-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t6zwb\" (UID: \"48c71614-5ca4-40b8-8ac2-fed4e8989831\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t6zwb" Jun 15 02:37:48.472263 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:48.472228 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57cbh\" (UniqueName: \"kubernetes.io/projected/48c71614-5ca4-40b8-8ac2-fed4e8989831-kube-api-access-57cbh\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t6zwb\" (UID: \"48c71614-5ca4-40b8-8ac2-fed4e8989831\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t6zwb" Jun 15 02:37:48.573164 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:48.573129 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/48c71614-5ca4-40b8-8ac2-fed4e8989831-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t6zwb\" (UID: \"48c71614-5ca4-40b8-8ac2-fed4e8989831\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t6zwb" Jun 15 02:37:48.573265 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:48.573216 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57cbh\" (UniqueName: \"kubernetes.io/projected/48c71614-5ca4-40b8-8ac2-fed4e8989831-kube-api-access-57cbh\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t6zwb\" (UID: \"48c71614-5ca4-40b8-8ac2-fed4e8989831\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t6zwb" Jun 15 02:37:48.573265 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:48.573240 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/48c71614-5ca4-40b8-8ac2-fed4e8989831-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t6zwb\" (UID: \"48c71614-5ca4-40b8-8ac2-fed4e8989831\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t6zwb" Jun 15 02:37:48.573473 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:48.573457 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/48c71614-5ca4-40b8-8ac2-fed4e8989831-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t6zwb\" (UID: \"48c71614-5ca4-40b8-8ac2-fed4e8989831\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t6zwb" Jun 15 02:37:48.573511 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:48.573491 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/48c71614-5ca4-40b8-8ac2-fed4e8989831-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t6zwb\" (UID: \"48c71614-5ca4-40b8-8ac2-fed4e8989831\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t6zwb" Jun 15 02:37:48.581249 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:48.581224 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-57cbh\" (UniqueName: \"kubernetes.io/projected/48c71614-5ca4-40b8-8ac2-fed4e8989831-kube-api-access-57cbh\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t6zwb\" (UID: \"48c71614-5ca4-40b8-8ac2-fed4e8989831\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t6zwb" Jun 15 02:37:48.684295 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:48.684245 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t6zwb" Jun 15 02:37:48.723500 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:48.723190 2579 generic.go:356] "Generic (PLEG): container finished" podID="5f0a46ca-b767-486a-b97e-f3b0f34098c4" containerID="cba2c79ecf4615b2362fe0596da883856ad864b8832cae7e31caf36e24fc61db" exitCode=0 Jun 15 02:37:48.723500 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:48.723292 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gq984" event={"ID":"5f0a46ca-b767-486a-b97e-f3b0f34098c4","Type":"ContainerDied","Data":"cba2c79ecf4615b2362fe0596da883856ad864b8832cae7e31caf36e24fc61db"} Jun 15 02:37:48.723500 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:48.723366 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gq984" event={"ID":"5f0a46ca-b767-486a-b97e-f3b0f34098c4","Type":"ContainerStarted","Data":"00ee895ac6861a1342b342da94a1569ed54e26b36bacb3fc05d2254c3a48dfc9"} Jun 15 02:37:48.810280 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:48.810256 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t6zwb"] Jun 15 02:37:48.812356 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:37:48.812327 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48c71614_5ca4_40b8_8ac2_fed4e8989831.slice/crio-603fa3376934cdf9de7849211a346eb5797f1deae3bff688fa50f5faa83a3c59 WatchSource:0}: Error finding container 603fa3376934cdf9de7849211a346eb5797f1deae3bff688fa50f5faa83a3c59: Status 404 returned error can't find the container with id 603fa3376934cdf9de7849211a346eb5797f1deae3bff688fa50f5faa83a3c59 Jun 15 02:37:48.955381 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:48.955320 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjmwq"] Jun 15 02:37:48.958785 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:48.958769 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjmwq" Jun 15 02:37:48.966521 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:48.966498 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjmwq"] Jun 15 02:37:49.077199 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:49.077178 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gg54r\" (UniqueName: \"kubernetes.io/projected/7e76d04b-a546-422a-8a7b-98936020ed85-kube-api-access-gg54r\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjmwq\" (UID: \"7e76d04b-a546-422a-8a7b-98936020ed85\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjmwq" Jun 15 02:37:49.077308 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:49.077217 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7e76d04b-a546-422a-8a7b-98936020ed85-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjmwq\" (UID: \"7e76d04b-a546-422a-8a7b-98936020ed85\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjmwq" Jun 15 02:37:49.077308 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:49.077306 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7e76d04b-a546-422a-8a7b-98936020ed85-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjmwq\" (UID: \"7e76d04b-a546-422a-8a7b-98936020ed85\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjmwq" Jun 15 02:37:49.177785 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:49.177763 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gg54r\" (UniqueName: \"kubernetes.io/projected/7e76d04b-a546-422a-8a7b-98936020ed85-kube-api-access-gg54r\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjmwq\" (UID: \"7e76d04b-a546-422a-8a7b-98936020ed85\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjmwq" Jun 15 02:37:49.177878 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:49.177805 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7e76d04b-a546-422a-8a7b-98936020ed85-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjmwq\" (UID: \"7e76d04b-a546-422a-8a7b-98936020ed85\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjmwq" Jun 15 02:37:49.177878 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:49.177851 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7e76d04b-a546-422a-8a7b-98936020ed85-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjmwq\" (UID: \"7e76d04b-a546-422a-8a7b-98936020ed85\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjmwq" Jun 15 02:37:49.178144 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:49.178130 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7e76d04b-a546-422a-8a7b-98936020ed85-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjmwq\" (UID: \"7e76d04b-a546-422a-8a7b-98936020ed85\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjmwq" Jun 15 02:37:49.178183 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:49.178164 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7e76d04b-a546-422a-8a7b-98936020ed85-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjmwq\" (UID: \"7e76d04b-a546-422a-8a7b-98936020ed85\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjmwq" Jun 15 02:37:49.185949 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:49.185929 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gg54r\" (UniqueName: \"kubernetes.io/projected/7e76d04b-a546-422a-8a7b-98936020ed85-kube-api-access-gg54r\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjmwq\" (UID: \"7e76d04b-a546-422a-8a7b-98936020ed85\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjmwq" Jun 15 02:37:49.268373 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:49.268351 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjmwq" Jun 15 02:37:49.354535 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:49.354502 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mbjjv"] Jun 15 02:37:49.359115 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:49.359091 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mbjjv" Jun 15 02:37:49.365160 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:49.365108 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mbjjv"] Jun 15 02:37:49.411359 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:49.411331 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjmwq"] Jun 15 02:37:49.425702 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:37:49.425671 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e76d04b_a546_422a_8a7b_98936020ed85.slice/crio-9bceff6a43b7e1269a43fe75828b9dcd2373938926fe3e8e9caeff678b1d90be WatchSource:0}: Error finding container 9bceff6a43b7e1269a43fe75828b9dcd2373938926fe3e8e9caeff678b1d90be: Status 404 returned error can't find the container with id 9bceff6a43b7e1269a43fe75828b9dcd2373938926fe3e8e9caeff678b1d90be Jun 15 02:37:49.480476 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:49.480456 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfbf6\" (UniqueName: \"kubernetes.io/projected/9fa08f07-32c5-4a19-a836-0f0bc4a33280-kube-api-access-hfbf6\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mbjjv\" (UID: \"9fa08f07-32c5-4a19-a836-0f0bc4a33280\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mbjjv" Jun 15 02:37:49.480565 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:49.480501 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9fa08f07-32c5-4a19-a836-0f0bc4a33280-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mbjjv\" (UID: \"9fa08f07-32c5-4a19-a836-0f0bc4a33280\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mbjjv" Jun 15 02:37:49.480565 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:49.480528 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9fa08f07-32c5-4a19-a836-0f0bc4a33280-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mbjjv\" (UID: \"9fa08f07-32c5-4a19-a836-0f0bc4a33280\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mbjjv" Jun 15 02:37:49.581546 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:49.581526 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9fa08f07-32c5-4a19-a836-0f0bc4a33280-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mbjjv\" (UID: \"9fa08f07-32c5-4a19-a836-0f0bc4a33280\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mbjjv" Jun 15 02:37:49.581688 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:49.581665 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfbf6\" (UniqueName: \"kubernetes.io/projected/9fa08f07-32c5-4a19-a836-0f0bc4a33280-kube-api-access-hfbf6\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mbjjv\" (UID: \"9fa08f07-32c5-4a19-a836-0f0bc4a33280\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mbjjv" Jun 15 02:37:49.581745 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:49.581726 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9fa08f07-32c5-4a19-a836-0f0bc4a33280-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mbjjv\" (UID: \"9fa08f07-32c5-4a19-a836-0f0bc4a33280\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mbjjv" Jun 15 02:37:49.581968 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:49.581942 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9fa08f07-32c5-4a19-a836-0f0bc4a33280-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mbjjv\" (UID: \"9fa08f07-32c5-4a19-a836-0f0bc4a33280\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mbjjv" Jun 15 02:37:49.582085 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:49.582069 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9fa08f07-32c5-4a19-a836-0f0bc4a33280-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mbjjv\" (UID: \"9fa08f07-32c5-4a19-a836-0f0bc4a33280\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mbjjv" Jun 15 02:37:49.589112 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:49.589093 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfbf6\" (UniqueName: \"kubernetes.io/projected/9fa08f07-32c5-4a19-a836-0f0bc4a33280-kube-api-access-hfbf6\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mbjjv\" (UID: \"9fa08f07-32c5-4a19-a836-0f0bc4a33280\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mbjjv" Jun 15 02:37:49.672489 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:49.672468 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mbjjv" Jun 15 02:37:49.731975 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:49.731944 2579 generic.go:356] "Generic (PLEG): container finished" podID="5f0a46ca-b767-486a-b97e-f3b0f34098c4" containerID="22fa1326c73bfd516bab2ff29ff608123f047f99f6de13a3b4b3b8858d09ce18" exitCode=0 Jun 15 02:37:49.732086 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:49.732022 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gq984" event={"ID":"5f0a46ca-b767-486a-b97e-f3b0f34098c4","Type":"ContainerDied","Data":"22fa1326c73bfd516bab2ff29ff608123f047f99f6de13a3b4b3b8858d09ce18"} Jun 15 02:37:49.733914 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:49.733890 2579 generic.go:356] "Generic (PLEG): container finished" podID="7e76d04b-a546-422a-8a7b-98936020ed85" containerID="45e12a71092c88ea383c1479c7fefb58f5fabe620ef5f6c5667d48d44291a1a4" exitCode=0 Jun 15 02:37:49.734015 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:49.733973 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjmwq" event={"ID":"7e76d04b-a546-422a-8a7b-98936020ed85","Type":"ContainerDied","Data":"45e12a71092c88ea383c1479c7fefb58f5fabe620ef5f6c5667d48d44291a1a4"} Jun 15 02:37:49.734081 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:49.734027 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjmwq" event={"ID":"7e76d04b-a546-422a-8a7b-98936020ed85","Type":"ContainerStarted","Data":"9bceff6a43b7e1269a43fe75828b9dcd2373938926fe3e8e9caeff678b1d90be"} Jun 15 02:37:49.735660 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:49.735615 2579 generic.go:356] "Generic (PLEG): container finished" podID="48c71614-5ca4-40b8-8ac2-fed4e8989831" containerID="673b5fe8d7238382c7147f1e34e92a8f0e4546058099ff45f0b9b22dde446b01" exitCode=0 Jun 15 02:37:49.735760 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:49.735716 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t6zwb" event={"ID":"48c71614-5ca4-40b8-8ac2-fed4e8989831","Type":"ContainerDied","Data":"673b5fe8d7238382c7147f1e34e92a8f0e4546058099ff45f0b9b22dde446b01"} Jun 15 02:37:49.735760 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:49.735748 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t6zwb" event={"ID":"48c71614-5ca4-40b8-8ac2-fed4e8989831","Type":"ContainerStarted","Data":"603fa3376934cdf9de7849211a346eb5797f1deae3bff688fa50f5faa83a3c59"} Jun 15 02:37:49.806362 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:49.806336 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mbjjv"] Jun 15 02:37:49.807640 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:37:49.807602 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fa08f07_32c5_4a19_a836_0f0bc4a33280.slice/crio-dafb5f3184f997d0de52c217f08adfbf1cc973b8bd7e75d5cae99e3e8e8b9f7e WatchSource:0}: Error finding container dafb5f3184f997d0de52c217f08adfbf1cc973b8bd7e75d5cae99e3e8e8b9f7e: Status 404 returned error can't find the container with id dafb5f3184f997d0de52c217f08adfbf1cc973b8bd7e75d5cae99e3e8e8b9f7e Jun 15 02:37:50.742451 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:50.742374 2579 generic.go:356] "Generic (PLEG): container finished" podID="5f0a46ca-b767-486a-b97e-f3b0f34098c4" containerID="275cfbba8235d0fc53aeccccc21fd900c06ca0d4980f203932622b1c6a2e51ae" exitCode=0 Jun 15 02:37:50.742862 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:50.742461 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gq984" event={"ID":"5f0a46ca-b767-486a-b97e-f3b0f34098c4","Type":"ContainerDied","Data":"275cfbba8235d0fc53aeccccc21fd900c06ca0d4980f203932622b1c6a2e51ae"} Jun 15 02:37:50.743849 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:50.743824 2579 generic.go:356] "Generic (PLEG): container finished" podID="9fa08f07-32c5-4a19-a836-0f0bc4a33280" containerID="3bb9f887aa3ac00f0ed50b1b0510e45e33aa44703d9e01fb38a2a64840ab3e0e" exitCode=0 Jun 15 02:37:50.743943 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:50.743904 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mbjjv" event={"ID":"9fa08f07-32c5-4a19-a836-0f0bc4a33280","Type":"ContainerDied","Data":"3bb9f887aa3ac00f0ed50b1b0510e45e33aa44703d9e01fb38a2a64840ab3e0e"} Jun 15 02:37:50.744005 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:50.743947 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mbjjv" event={"ID":"9fa08f07-32c5-4a19-a836-0f0bc4a33280","Type":"ContainerStarted","Data":"dafb5f3184f997d0de52c217f08adfbf1cc973b8bd7e75d5cae99e3e8e8b9f7e"} Jun 15 02:37:50.745708 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:50.745677 2579 generic.go:356] "Generic (PLEG): container finished" podID="7e76d04b-a546-422a-8a7b-98936020ed85" containerID="4a17c625bb01f25a79a5959a61b013f25c821a0db6111607f75c1a15c4997d80" exitCode=0 Jun 15 02:37:50.745803 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:50.745732 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjmwq" event={"ID":"7e76d04b-a546-422a-8a7b-98936020ed85","Type":"ContainerDied","Data":"4a17c625bb01f25a79a5959a61b013f25c821a0db6111607f75c1a15c4997d80"} Jun 15 02:37:50.747696 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:50.747672 2579 generic.go:356] "Generic (PLEG): container finished" podID="48c71614-5ca4-40b8-8ac2-fed4e8989831" containerID="d2d8b192991b43325ec4e417054b63f6ef2b9371d1b463644acbaebced4a420e" exitCode=0 Jun 15 02:37:50.747784 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:50.747736 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t6zwb" event={"ID":"48c71614-5ca4-40b8-8ac2-fed4e8989831","Type":"ContainerDied","Data":"d2d8b192991b43325ec4e417054b63f6ef2b9371d1b463644acbaebced4a420e"} Jun 15 02:37:51.754629 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:51.754600 2579 generic.go:356] "Generic (PLEG): container finished" podID="48c71614-5ca4-40b8-8ac2-fed4e8989831" containerID="878b3f2457a9aaaabece43b60cefeb7a6eff13bb3744a43c4246c0e68b611ec0" exitCode=0 Jun 15 02:37:51.754988 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:51.754681 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t6zwb" event={"ID":"48c71614-5ca4-40b8-8ac2-fed4e8989831","Type":"ContainerDied","Data":"878b3f2457a9aaaabece43b60cefeb7a6eff13bb3744a43c4246c0e68b611ec0"} Jun 15 02:37:51.756399 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:51.756381 2579 generic.go:356] "Generic (PLEG): container finished" podID="9fa08f07-32c5-4a19-a836-0f0bc4a33280" containerID="c47fe9ba80f7da1d8f8d9c76c3c775af0c82bad35bedc21ad7e4c95a5054e5c9" exitCode=0 Jun 15 02:37:51.756510 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:51.756461 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mbjjv" event={"ID":"9fa08f07-32c5-4a19-a836-0f0bc4a33280","Type":"ContainerDied","Data":"c47fe9ba80f7da1d8f8d9c76c3c775af0c82bad35bedc21ad7e4c95a5054e5c9"} Jun 15 02:37:51.758588 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:51.758547 2579 generic.go:356] "Generic (PLEG): container finished" podID="7e76d04b-a546-422a-8a7b-98936020ed85" containerID="cbebd31a218252dbb9644a25a636091bc2d75876da5e7bf78ff9c92d0243ffe4" exitCode=0 Jun 15 02:37:51.758694 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:51.758627 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjmwq" event={"ID":"7e76d04b-a546-422a-8a7b-98936020ed85","Type":"ContainerDied","Data":"cbebd31a218252dbb9644a25a636091bc2d75876da5e7bf78ff9c92d0243ffe4"} Jun 15 02:37:51.888972 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:51.888949 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gq984" Jun 15 02:37:52.002110 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:52.002087 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qph74\" (UniqueName: \"kubernetes.io/projected/5f0a46ca-b767-486a-b97e-f3b0f34098c4-kube-api-access-qph74\") pod \"5f0a46ca-b767-486a-b97e-f3b0f34098c4\" (UID: \"5f0a46ca-b767-486a-b97e-f3b0f34098c4\") " Jun 15 02:37:52.002211 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:52.002127 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5f0a46ca-b767-486a-b97e-f3b0f34098c4-bundle\") pod \"5f0a46ca-b767-486a-b97e-f3b0f34098c4\" (UID: \"5f0a46ca-b767-486a-b97e-f3b0f34098c4\") " Jun 15 02:37:52.002211 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:52.002186 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5f0a46ca-b767-486a-b97e-f3b0f34098c4-util\") pod \"5f0a46ca-b767-486a-b97e-f3b0f34098c4\" (UID: \"5f0a46ca-b767-486a-b97e-f3b0f34098c4\") " Jun 15 02:37:52.002592 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:52.002565 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f0a46ca-b767-486a-b97e-f3b0f34098c4-bundle" (OuterVolumeSpecName: "bundle") pod "5f0a46ca-b767-486a-b97e-f3b0f34098c4" (UID: "5f0a46ca-b767-486a-b97e-f3b0f34098c4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 02:37:52.004145 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:52.004116 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f0a46ca-b767-486a-b97e-f3b0f34098c4-kube-api-access-qph74" (OuterVolumeSpecName: "kube-api-access-qph74") pod "5f0a46ca-b767-486a-b97e-f3b0f34098c4" (UID: "5f0a46ca-b767-486a-b97e-f3b0f34098c4"). InnerVolumeSpecName "kube-api-access-qph74". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 02:37:52.009608 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:52.009562 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f0a46ca-b767-486a-b97e-f3b0f34098c4-util" (OuterVolumeSpecName: "util") pod "5f0a46ca-b767-486a-b97e-f3b0f34098c4" (UID: "5f0a46ca-b767-486a-b97e-f3b0f34098c4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 02:37:52.102680 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:52.102658 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qph74\" (UniqueName: \"kubernetes.io/projected/5f0a46ca-b767-486a-b97e-f3b0f34098c4-kube-api-access-qph74\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:37:52.102680 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:52.102679 2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5f0a46ca-b767-486a-b97e-f3b0f34098c4-bundle\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:37:52.102795 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:52.102688 2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5f0a46ca-b767-486a-b97e-f3b0f34098c4-util\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:37:52.765471 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:52.765433 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gq984" event={"ID":"5f0a46ca-b767-486a-b97e-f3b0f34098c4","Type":"ContainerDied","Data":"00ee895ac6861a1342b342da94a1569ed54e26b36bacb3fc05d2254c3a48dfc9"} Jun 15 02:37:52.765935 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:52.765481 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="00ee895ac6861a1342b342da94a1569ed54e26b36bacb3fc05d2254c3a48dfc9" Jun 15 02:37:52.765935 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:52.765437 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gq984" Jun 15 02:37:52.767447 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:52.767417 2579 generic.go:356] "Generic (PLEG): container finished" podID="9fa08f07-32c5-4a19-a836-0f0bc4a33280" containerID="89c593db6265778a825a4fb23fb89de73fb2553bf817adccae8b9f907173efde" exitCode=0 Jun 15 02:37:52.767572 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:52.767496 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mbjjv" event={"ID":"9fa08f07-32c5-4a19-a836-0f0bc4a33280","Type":"ContainerDied","Data":"89c593db6265778a825a4fb23fb89de73fb2553bf817adccae8b9f907173efde"} Jun 15 02:37:52.923689 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:52.923666 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t6zwb" Jun 15 02:37:52.926740 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:52.926718 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjmwq" Jun 15 02:37:53.008282 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:53.008260 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/48c71614-5ca4-40b8-8ac2-fed4e8989831-bundle\") pod \"48c71614-5ca4-40b8-8ac2-fed4e8989831\" (UID: \"48c71614-5ca4-40b8-8ac2-fed4e8989831\") " Jun 15 02:37:53.008399 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:53.008290 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7e76d04b-a546-422a-8a7b-98936020ed85-util\") pod \"7e76d04b-a546-422a-8a7b-98936020ed85\" (UID: \"7e76d04b-a546-422a-8a7b-98936020ed85\") " Jun 15 02:37:53.008399 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:53.008319 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/48c71614-5ca4-40b8-8ac2-fed4e8989831-util\") pod \"48c71614-5ca4-40b8-8ac2-fed4e8989831\" (UID: \"48c71614-5ca4-40b8-8ac2-fed4e8989831\") " Jun 15 02:37:53.008399 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:53.008341 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gg54r\" (UniqueName: \"kubernetes.io/projected/7e76d04b-a546-422a-8a7b-98936020ed85-kube-api-access-gg54r\") pod \"7e76d04b-a546-422a-8a7b-98936020ed85\" (UID: \"7e76d04b-a546-422a-8a7b-98936020ed85\") " Jun 15 02:37:53.008399 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:53.008392 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-57cbh\" (UniqueName: \"kubernetes.io/projected/48c71614-5ca4-40b8-8ac2-fed4e8989831-kube-api-access-57cbh\") pod \"48c71614-5ca4-40b8-8ac2-fed4e8989831\" (UID: \"48c71614-5ca4-40b8-8ac2-fed4e8989831\") " Jun 15 02:37:53.008553 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:53.008470 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7e76d04b-a546-422a-8a7b-98936020ed85-bundle\") pod \"7e76d04b-a546-422a-8a7b-98936020ed85\" (UID: \"7e76d04b-a546-422a-8a7b-98936020ed85\") " Jun 15 02:37:53.009451 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:53.008862 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48c71614-5ca4-40b8-8ac2-fed4e8989831-bundle" (OuterVolumeSpecName: "bundle") pod "48c71614-5ca4-40b8-8ac2-fed4e8989831" (UID: "48c71614-5ca4-40b8-8ac2-fed4e8989831"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 02:37:53.009451 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:53.009414 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e76d04b-a546-422a-8a7b-98936020ed85-bundle" (OuterVolumeSpecName: "bundle") pod "7e76d04b-a546-422a-8a7b-98936020ed85" (UID: "7e76d04b-a546-422a-8a7b-98936020ed85"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 02:37:53.010702 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:53.010608 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e76d04b-a546-422a-8a7b-98936020ed85-kube-api-access-gg54r" (OuterVolumeSpecName: "kube-api-access-gg54r") pod "7e76d04b-a546-422a-8a7b-98936020ed85" (UID: "7e76d04b-a546-422a-8a7b-98936020ed85"). InnerVolumeSpecName "kube-api-access-gg54r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 02:37:53.010891 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:53.010866 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48c71614-5ca4-40b8-8ac2-fed4e8989831-kube-api-access-57cbh" (OuterVolumeSpecName: "kube-api-access-57cbh") pod "48c71614-5ca4-40b8-8ac2-fed4e8989831" (UID: "48c71614-5ca4-40b8-8ac2-fed4e8989831"). InnerVolumeSpecName "kube-api-access-57cbh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 02:37:53.014684 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:53.014653 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e76d04b-a546-422a-8a7b-98936020ed85-util" (OuterVolumeSpecName: "util") pod "7e76d04b-a546-422a-8a7b-98936020ed85" (UID: "7e76d04b-a546-422a-8a7b-98936020ed85"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 02:37:53.016609 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:53.016562 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48c71614-5ca4-40b8-8ac2-fed4e8989831-util" (OuterVolumeSpecName: "util") pod "48c71614-5ca4-40b8-8ac2-fed4e8989831" (UID: "48c71614-5ca4-40b8-8ac2-fed4e8989831"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 02:37:53.109300 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:53.109279 2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7e76d04b-a546-422a-8a7b-98936020ed85-bundle\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:37:53.109300 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:53.109300 2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/48c71614-5ca4-40b8-8ac2-fed4e8989831-bundle\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:37:53.109409 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:53.109308 2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7e76d04b-a546-422a-8a7b-98936020ed85-util\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:37:53.109409 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:53.109316 2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/48c71614-5ca4-40b8-8ac2-fed4e8989831-util\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:37:53.109409 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:53.109325 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gg54r\" (UniqueName: \"kubernetes.io/projected/7e76d04b-a546-422a-8a7b-98936020ed85-kube-api-access-gg54r\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:37:53.109409 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:53.109334 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-57cbh\" (UniqueName: \"kubernetes.io/projected/48c71614-5ca4-40b8-8ac2-fed4e8989831-kube-api-access-57cbh\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:37:53.774439 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:53.774375 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjmwq" Jun 15 02:37:53.774862 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:53.774370 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjmwq" event={"ID":"7e76d04b-a546-422a-8a7b-98936020ed85","Type":"ContainerDied","Data":"9bceff6a43b7e1269a43fe75828b9dcd2373938926fe3e8e9caeff678b1d90be"} Jun 15 02:37:53.774862 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:53.774498 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9bceff6a43b7e1269a43fe75828b9dcd2373938926fe3e8e9caeff678b1d90be" Jun 15 02:37:53.776543 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:53.776514 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t6zwb" Jun 15 02:37:53.776668 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:53.776547 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t6zwb" event={"ID":"48c71614-5ca4-40b8-8ac2-fed4e8989831","Type":"ContainerDied","Data":"603fa3376934cdf9de7849211a346eb5797f1deae3bff688fa50f5faa83a3c59"} Jun 15 02:37:53.776668 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:53.776587 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="603fa3376934cdf9de7849211a346eb5797f1deae3bff688fa50f5faa83a3c59" Jun 15 02:37:53.904748 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:53.904723 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mbjjv" Jun 15 02:37:54.015532 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:54.015503 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9fa08f07-32c5-4a19-a836-0f0bc4a33280-bundle\") pod \"9fa08f07-32c5-4a19-a836-0f0bc4a33280\" (UID: \"9fa08f07-32c5-4a19-a836-0f0bc4a33280\") " Jun 15 02:37:54.015695 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:54.015602 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9fa08f07-32c5-4a19-a836-0f0bc4a33280-util\") pod \"9fa08f07-32c5-4a19-a836-0f0bc4a33280\" (UID: \"9fa08f07-32c5-4a19-a836-0f0bc4a33280\") " Jun 15 02:37:54.015695 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:54.015672 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hfbf6\" (UniqueName: \"kubernetes.io/projected/9fa08f07-32c5-4a19-a836-0f0bc4a33280-kube-api-access-hfbf6\") pod \"9fa08f07-32c5-4a19-a836-0f0bc4a33280\" (UID: \"9fa08f07-32c5-4a19-a836-0f0bc4a33280\") " Jun 15 02:37:54.016081 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:54.016057 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fa08f07-32c5-4a19-a836-0f0bc4a33280-bundle" (OuterVolumeSpecName: "bundle") pod "9fa08f07-32c5-4a19-a836-0f0bc4a33280" (UID: "9fa08f07-32c5-4a19-a836-0f0bc4a33280"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 02:37:54.017670 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:54.017625 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fa08f07-32c5-4a19-a836-0f0bc4a33280-kube-api-access-hfbf6" (OuterVolumeSpecName: "kube-api-access-hfbf6") pod "9fa08f07-32c5-4a19-a836-0f0bc4a33280" (UID: "9fa08f07-32c5-4a19-a836-0f0bc4a33280"). InnerVolumeSpecName "kube-api-access-hfbf6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 02:37:54.021215 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:54.021186 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fa08f07-32c5-4a19-a836-0f0bc4a33280-util" (OuterVolumeSpecName: "util") pod "9fa08f07-32c5-4a19-a836-0f0bc4a33280" (UID: "9fa08f07-32c5-4a19-a836-0f0bc4a33280"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 02:37:54.116537 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:54.116466 2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9fa08f07-32c5-4a19-a836-0f0bc4a33280-util\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:37:54.116537 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:54.116498 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hfbf6\" (UniqueName: \"kubernetes.io/projected/9fa08f07-32c5-4a19-a836-0f0bc4a33280-kube-api-access-hfbf6\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:37:54.116537 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:54.116511 2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9fa08f07-32c5-4a19-a836-0f0bc4a33280-bundle\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:37:54.784308 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:54.784276 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mbjjv" event={"ID":"9fa08f07-32c5-4a19-a836-0f0bc4a33280","Type":"ContainerDied","Data":"dafb5f3184f997d0de52c217f08adfbf1cc973b8bd7e75d5cae99e3e8e8b9f7e"} Jun 15 02:37:54.784658 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:54.784316 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mbjjv" Jun 15 02:37:54.784658 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:37:54.784320 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dafb5f3184f997d0de52c217f08adfbf1cc973b8bd7e75d5cae99e3e8e8b9f7e" Jun 15 02:38:17.553346 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:17.553316 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-m4kcm"] Jun 15 02:38:17.553846 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:17.553610 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48c71614-5ca4-40b8-8ac2-fed4e8989831" containerName="util" Jun 15 02:38:17.553846 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:17.553622 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="48c71614-5ca4-40b8-8ac2-fed4e8989831" containerName="util" Jun 15 02:38:17.553846 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:17.553645 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f0a46ca-b767-486a-b97e-f3b0f34098c4" containerName="extract" Jun 15 02:38:17.553846 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:17.553651 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f0a46ca-b767-486a-b97e-f3b0f34098c4" containerName="extract" Jun 15 02:38:17.553846 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:17.553660 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48c71614-5ca4-40b8-8ac2-fed4e8989831" containerName="pull" Jun 15 02:38:17.553846 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:17.553665 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="48c71614-5ca4-40b8-8ac2-fed4e8989831" containerName="pull" Jun 15 02:38:17.553846 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:17.553675 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e76d04b-a546-422a-8a7b-98936020ed85" containerName="pull" Jun 15 02:38:17.553846 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:17.553679 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e76d04b-a546-422a-8a7b-98936020ed85" containerName="pull" Jun 15 02:38:17.553846 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:17.553688 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48c71614-5ca4-40b8-8ac2-fed4e8989831" containerName="extract" Jun 15 02:38:17.553846 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:17.553693 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="48c71614-5ca4-40b8-8ac2-fed4e8989831" containerName="extract" Jun 15 02:38:17.553846 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:17.553701 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e76d04b-a546-422a-8a7b-98936020ed85" containerName="util" Jun 15 02:38:17.553846 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:17.553706 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e76d04b-a546-422a-8a7b-98936020ed85" containerName="util" Jun 15 02:38:17.553846 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:17.553712 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fa08f07-32c5-4a19-a836-0f0bc4a33280" containerName="pull" Jun 15 02:38:17.553846 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:17.553717 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fa08f07-32c5-4a19-a836-0f0bc4a33280" containerName="pull" Jun 15 02:38:17.553846 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:17.553723 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fa08f07-32c5-4a19-a836-0f0bc4a33280" containerName="extract" Jun 15 02:38:17.553846 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:17.553728 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fa08f07-32c5-4a19-a836-0f0bc4a33280" containerName="extract" Jun 15 02:38:17.553846 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:17.553734 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e76d04b-a546-422a-8a7b-98936020ed85" containerName="extract" Jun 15 02:38:17.553846 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:17.553739 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e76d04b-a546-422a-8a7b-98936020ed85" containerName="extract" Jun 15 02:38:17.553846 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:17.553745 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fa08f07-32c5-4a19-a836-0f0bc4a33280" containerName="util" Jun 15 02:38:17.553846 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:17.553750 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fa08f07-32c5-4a19-a836-0f0bc4a33280" containerName="util" Jun 15 02:38:17.553846 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:17.553759 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f0a46ca-b767-486a-b97e-f3b0f34098c4" containerName="pull" Jun 15 02:38:17.553846 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:17.553764 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f0a46ca-b767-486a-b97e-f3b0f34098c4" containerName="pull" Jun 15 02:38:17.553846 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:17.553771 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f0a46ca-b767-486a-b97e-f3b0f34098c4" containerName="util" Jun 15 02:38:17.553846 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:17.553776 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f0a46ca-b767-486a-b97e-f3b0f34098c4" containerName="util" Jun 15 02:38:17.553846 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:17.553834 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="48c71614-5ca4-40b8-8ac2-fed4e8989831" containerName="extract" Jun 15 02:38:17.553846 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:17.553842 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f0a46ca-b767-486a-b97e-f3b0f34098c4" containerName="extract" Jun 15 02:38:17.553846 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:17.553850 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fa08f07-32c5-4a19-a836-0f0bc4a33280" containerName="extract" Jun 15 02:38:17.553846 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:17.553857 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e76d04b-a546-422a-8a7b-98936020ed85" containerName="extract" Jun 15 02:38:17.556741 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:17.556721 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-m4kcm" Jun 15 02:38:17.558728 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:17.558707 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jun 15 02:38:17.558870 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:17.558848 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 15 02:38:17.559471 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:17.559449 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-jxn47\"" Jun 15 02:38:17.559471 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:17.559462 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 15 02:38:17.559605 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:17.559455 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jun 15 02:38:17.563279 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:17.563259 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-m4kcm"] Jun 15 02:38:17.686913 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:17.686883 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c09566f4-206a-4394-934d-6acb05947718-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-m4kcm\" (UID: \"c09566f4-206a-4394-934d-6acb05947718\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-m4kcm" Jun 15 02:38:17.687065 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:17.686919 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/c09566f4-206a-4394-934d-6acb05947718-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-m4kcm\" (UID: \"c09566f4-206a-4394-934d-6acb05947718\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-m4kcm" Jun 15 02:38:17.687065 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:17.686987 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgsm8\" (UniqueName: \"kubernetes.io/projected/c09566f4-206a-4394-934d-6acb05947718-kube-api-access-cgsm8\") pod \"kuadrant-console-plugin-66b5774985-m4kcm\" (UID: \"c09566f4-206a-4394-934d-6acb05947718\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-m4kcm" Jun 15 02:38:17.787773 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:17.787739 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c09566f4-206a-4394-934d-6acb05947718-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-m4kcm\" (UID: \"c09566f4-206a-4394-934d-6acb05947718\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-m4kcm" Jun 15 02:38:17.787919 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:17.787780 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/c09566f4-206a-4394-934d-6acb05947718-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-m4kcm\" (UID: \"c09566f4-206a-4394-934d-6acb05947718\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-m4kcm" Jun 15 02:38:17.787919 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:17.787823 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgsm8\" (UniqueName: \"kubernetes.io/projected/c09566f4-206a-4394-934d-6acb05947718-kube-api-access-cgsm8\") pod \"kuadrant-console-plugin-66b5774985-m4kcm\" (UID: \"c09566f4-206a-4394-934d-6acb05947718\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-m4kcm" Jun 15 02:38:17.788347 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:17.788325 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c09566f4-206a-4394-934d-6acb05947718-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-m4kcm\" (UID: \"c09566f4-206a-4394-934d-6acb05947718\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-m4kcm" Jun 15 02:38:17.790186 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:17.790162 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/c09566f4-206a-4394-934d-6acb05947718-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-m4kcm\" (UID: \"c09566f4-206a-4394-934d-6acb05947718\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-m4kcm" Jun 15 02:38:17.796389 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:17.796361 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgsm8\" (UniqueName: \"kubernetes.io/projected/c09566f4-206a-4394-934d-6acb05947718-kube-api-access-cgsm8\") pod \"kuadrant-console-plugin-66b5774985-m4kcm\" (UID: \"c09566f4-206a-4394-934d-6acb05947718\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-m4kcm" Jun 15 02:38:17.867164 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:17.867112 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-m4kcm" Jun 15 02:38:17.991072 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:17.991049 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-m4kcm"] Jun 15 02:38:17.993111 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:38:17.993079 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc09566f4_206a_4394_934d_6acb05947718.slice/crio-4345aac14b5a15cfefd98943c1a5659c456a52e42d94a0d67e8252d2ed9e7874 WatchSource:0}: Error finding container 4345aac14b5a15cfefd98943c1a5659c456a52e42d94a0d67e8252d2ed9e7874: Status 404 returned error can't find the container with id 4345aac14b5a15cfefd98943c1a5659c456a52e42d94a0d67e8252d2ed9e7874 Jun 15 02:38:18.904172 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:18.904140 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-m4kcm" event={"ID":"c09566f4-206a-4394-934d-6acb05947718","Type":"ContainerStarted","Data":"4345aac14b5a15cfefd98943c1a5659c456a52e42d94a0d67e8252d2ed9e7874"} Jun 15 02:38:27.898983 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:27.898894 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-n8l6n"] Jun 15 02:38:27.903887 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:27.903864 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-n8l6n" Jun 15 02:38:27.909260 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:27.909231 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-qtrb9\"" Jun 15 02:38:27.917323 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:27.917300 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-n8l6n"] Jun 15 02:38:27.977490 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:27.977453 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d95vz\" (UniqueName: \"kubernetes.io/projected/0c6298d1-bf7e-41b7-b7ff-cc2ad4ea2f92-kube-api-access-d95vz\") pod \"kuadrant-operator-controller-manager-55467d9f7-n8l6n\" (UID: \"0c6298d1-bf7e-41b7-b7ff-cc2ad4ea2f92\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-n8l6n" Jun 15 02:38:27.977685 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:27.977519 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0c6298d1-bf7e-41b7-b7ff-cc2ad4ea2f92-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-n8l6n\" (UID: \"0c6298d1-bf7e-41b7-b7ff-cc2ad4ea2f92\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-n8l6n" Jun 15 02:38:28.002280 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:28.002253 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-n8l6n"] Jun 15 02:38:28.002587 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:38:28.002543 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-d95vz], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-n8l6n" podUID="0c6298d1-bf7e-41b7-b7ff-cc2ad4ea2f92" Jun 15 02:38:28.078679 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:28.078625 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d95vz\" (UniqueName: \"kubernetes.io/projected/0c6298d1-bf7e-41b7-b7ff-cc2ad4ea2f92-kube-api-access-d95vz\") pod \"kuadrant-operator-controller-manager-55467d9f7-n8l6n\" (UID: \"0c6298d1-bf7e-41b7-b7ff-cc2ad4ea2f92\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-n8l6n" Jun 15 02:38:28.078841 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:28.078731 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0c6298d1-bf7e-41b7-b7ff-cc2ad4ea2f92-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-n8l6n\" (UID: \"0c6298d1-bf7e-41b7-b7ff-cc2ad4ea2f92\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-n8l6n" Jun 15 02:38:28.079154 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:28.079133 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0c6298d1-bf7e-41b7-b7ff-cc2ad4ea2f92-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-n8l6n\" (UID: \"0c6298d1-bf7e-41b7-b7ff-cc2ad4ea2f92\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-n8l6n" Jun 15 02:38:28.089627 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:28.089593 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-n8l6n"] Jun 15 02:38:28.093159 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:28.093135 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d95vz\" (UniqueName: \"kubernetes.io/projected/0c6298d1-bf7e-41b7-b7ff-cc2ad4ea2f92-kube-api-access-d95vz\") pod \"kuadrant-operator-controller-manager-55467d9f7-n8l6n\" (UID: \"0c6298d1-bf7e-41b7-b7ff-cc2ad4ea2f92\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-n8l6n" Jun 15 02:38:28.098380 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:28.098357 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-n8l6n"] Jun 15 02:38:28.172154 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:28.172074 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8tzrh"] Jun 15 02:38:28.178396 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:28.178376 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8tzrh" Jun 15 02:38:28.189625 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:28.189521 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8tzrh"] Jun 15 02:38:28.280508 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:28.280472 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ef684a3a-f3a0-47ba-bf1c-0b2249816ef4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-8tzrh\" (UID: \"ef684a3a-f3a0-47ba-bf1c-0b2249816ef4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8tzrh" Jun 15 02:38:28.280681 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:28.280520 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wgds\" (UniqueName: \"kubernetes.io/projected/ef684a3a-f3a0-47ba-bf1c-0b2249816ef4-kube-api-access-9wgds\") pod \"kuadrant-operator-controller-manager-86878c769f-8tzrh\" (UID: \"ef684a3a-f3a0-47ba-bf1c-0b2249816ef4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8tzrh" Jun 15 02:38:28.381964 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:28.381924 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ef684a3a-f3a0-47ba-bf1c-0b2249816ef4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-8tzrh\" (UID: \"ef684a3a-f3a0-47ba-bf1c-0b2249816ef4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8tzrh" Jun 15 02:38:28.382148 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:28.381975 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wgds\" (UniqueName: \"kubernetes.io/projected/ef684a3a-f3a0-47ba-bf1c-0b2249816ef4-kube-api-access-9wgds\") pod \"kuadrant-operator-controller-manager-86878c769f-8tzrh\" (UID: \"ef684a3a-f3a0-47ba-bf1c-0b2249816ef4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8tzrh" Jun 15 02:38:28.382338 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:28.382317 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ef684a3a-f3a0-47ba-bf1c-0b2249816ef4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-8tzrh\" (UID: \"ef684a3a-f3a0-47ba-bf1c-0b2249816ef4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8tzrh" Jun 15 02:38:28.397748 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:28.397722 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wgds\" (UniqueName: \"kubernetes.io/projected/ef684a3a-f3a0-47ba-bf1c-0b2249816ef4-kube-api-access-9wgds\") pod \"kuadrant-operator-controller-manager-86878c769f-8tzrh\" (UID: \"ef684a3a-f3a0-47ba-bf1c-0b2249816ef4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8tzrh" Jun 15 02:38:28.492825 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:28.492793 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8tzrh" Jun 15 02:38:28.960612 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:28.960540 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-n8l6n" Jun 15 02:38:28.962590 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:38:28.962539 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-n8l6n\" is forbidden: User \"system:node:ip-10-0-143-51.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-51.ec2.internal' and this object" podUID="0c6298d1-bf7e-41b7-b7ff-cc2ad4ea2f92" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-n8l6n" Jun 15 02:38:28.973791 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:28.973766 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-n8l6n" Jun 15 02:38:28.975828 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:38:28.975801 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-n8l6n\" is forbidden: User \"system:node:ip-10-0-143-51.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-51.ec2.internal' and this object" podUID="0c6298d1-bf7e-41b7-b7ff-cc2ad4ea2f92" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-n8l6n" Jun 15 02:38:29.088271 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:29.088211 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0c6298d1-bf7e-41b7-b7ff-cc2ad4ea2f92-extensions-socket-volume\") pod \"0c6298d1-bf7e-41b7-b7ff-cc2ad4ea2f92\" (UID: \"0c6298d1-bf7e-41b7-b7ff-cc2ad4ea2f92\") " Jun 15 02:38:29.088450 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:29.088301 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d95vz\" (UniqueName: \"kubernetes.io/projected/0c6298d1-bf7e-41b7-b7ff-cc2ad4ea2f92-kube-api-access-d95vz\") pod \"0c6298d1-bf7e-41b7-b7ff-cc2ad4ea2f92\" (UID: \"0c6298d1-bf7e-41b7-b7ff-cc2ad4ea2f92\") " Jun 15 02:38:29.088450 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:29.088442 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c6298d1-bf7e-41b7-b7ff-cc2ad4ea2f92-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "0c6298d1-bf7e-41b7-b7ff-cc2ad4ea2f92" (UID: "0c6298d1-bf7e-41b7-b7ff-cc2ad4ea2f92"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 02:38:29.088586 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:29.088569 2579 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0c6298d1-bf7e-41b7-b7ff-cc2ad4ea2f92-extensions-socket-volume\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:38:29.090627 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:29.090592 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c6298d1-bf7e-41b7-b7ff-cc2ad4ea2f92-kube-api-access-d95vz" (OuterVolumeSpecName: "kube-api-access-d95vz") pod "0c6298d1-bf7e-41b7-b7ff-cc2ad4ea2f92" (UID: "0c6298d1-bf7e-41b7-b7ff-cc2ad4ea2f92"). InnerVolumeSpecName "kube-api-access-d95vz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 02:38:29.189994 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:29.189957 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d95vz\" (UniqueName: \"kubernetes.io/projected/0c6298d1-bf7e-41b7-b7ff-cc2ad4ea2f92-kube-api-access-d95vz\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:38:29.494337 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:29.494295 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c6298d1-bf7e-41b7-b7ff-cc2ad4ea2f92" path="/var/lib/kubelet/pods/0c6298d1-bf7e-41b7-b7ff-cc2ad4ea2f92/volumes" Jun 15 02:38:29.967101 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:29.967035 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-n8l6n" Jun 15 02:38:29.971860 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:38:29.971825 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-n8l6n\" is forbidden: User \"system:node:ip-10-0-143-51.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-51.ec2.internal' and this object" podUID="0c6298d1-bf7e-41b7-b7ff-cc2ad4ea2f92" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-n8l6n" Jun 15 02:38:41.740244 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:41.740214 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-w4m86_fb6051d8-9291-4309-9047-9e8d0b5d2e48/console-operator/1.log" Jun 15 02:38:41.740669 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:41.740650 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-w4m86_fb6051d8-9291-4309-9047-9e8d0b5d2e48/console-operator/1.log" Jun 15 02:38:41.842967 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:41.842936 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8tzrh"] Jun 15 02:38:41.844081 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:38:41.844058 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef684a3a_f3a0_47ba_bf1c_0b2249816ef4.slice/crio-cff11eaa4831cb431c7e9bac3a360deb63648bd961399473c611e9167848e024 WatchSource:0}: Error finding container cff11eaa4831cb431c7e9bac3a360deb63648bd961399473c611e9167848e024: Status 404 returned error can't find the container with id cff11eaa4831cb431c7e9bac3a360deb63648bd961399473c611e9167848e024 Jun 15 02:38:42.040941 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:42.040905 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8tzrh" event={"ID":"ef684a3a-f3a0-47ba-bf1c-0b2249816ef4","Type":"ContainerStarted","Data":"cff11eaa4831cb431c7e9bac3a360deb63648bd961399473c611e9167848e024"} Jun 15 02:38:43.047827 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:43.047790 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-m4kcm" event={"ID":"c09566f4-206a-4394-934d-6acb05947718","Type":"ContainerStarted","Data":"30760a3c1f17f1e8cfc2941f194b394b6cc3033b5434d2ff481e40fb42995fd7"} Jun 15 02:38:43.063120 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:43.063065 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-m4kcm" podStartSLOduration=1.978911492 podStartE2EDuration="26.06305137s" totalImagesPullingTime="24.084139878s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 02:38:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 02:38:43.061692928 +0000 UTC m=+615.929477673" watchObservedRunningTime="2026-06-15 02:38:43.06305137 +0000 UTC m=+615.930836116" Jun 15 02:38:50.088856 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:50.088811 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8tzrh" event={"ID":"ef684a3a-f3a0-47ba-bf1c-0b2249816ef4","Type":"ContainerStarted","Data":"adf1acfebe9821e62674273de286e20966e70f8b9e87ee7dd116416f9f7cc338"} Jun 15 02:38:50.089320 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:50.088876 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8tzrh" Jun 15 02:38:50.108845 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:38:50.108787 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8tzrh" podStartSLOduration=14.279909889 podStartE2EDuration="22.108772223s" totalImagesPullingTime="7.828862334s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 02:38:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 02:38:50.107505131 +0000 UTC m=+622.975289877" watchObservedRunningTime="2026-06-15 02:38:50.108772223 +0000 UTC m=+622.976557027" Jun 15 02:39:01.095612 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:01.095582 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8tzrh" Jun 15 02:39:20.618928 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:20.618892 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-kkdvq"] Jun 15 02:39:20.625774 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:20.625751 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-kkdvq" Jun 15 02:39:20.627898 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:20.627873 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-8fwm6\"" Jun 15 02:39:20.629067 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:20.629045 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-kkdvq"] Jun 15 02:39:20.708629 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:20.708605 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hj5h6\" (UniqueName: \"kubernetes.io/projected/c5956ba2-b1c3-40dc-b6ae-e20558ab23c6-kube-api-access-hj5h6\") pod \"authorino-58cd955675-kkdvq\" (UID: \"c5956ba2-b1c3-40dc-b6ae-e20558ab23c6\") " pod="kuadrant-system/authorino-58cd955675-kkdvq" Jun 15 02:39:20.726424 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:20.726393 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-z8s5z"] Jun 15 02:39:20.731020 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:20.731001 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-z8s5z" Jun 15 02:39:20.734188 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:20.734164 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-z8s5z"] Jun 15 02:39:20.809231 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:20.809209 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bb2mx\" (UniqueName: \"kubernetes.io/projected/42eab444-c243-4e09-b171-9df373d22bcd-kube-api-access-bb2mx\") pod \"authorino-75d556bd77-z8s5z\" (UID: \"42eab444-c243-4e09-b171-9df373d22bcd\") " pod="kuadrant-system/authorino-75d556bd77-z8s5z" Jun 15 02:39:20.809374 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:20.809244 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hj5h6\" (UniqueName: \"kubernetes.io/projected/c5956ba2-b1c3-40dc-b6ae-e20558ab23c6-kube-api-access-hj5h6\") pod \"authorino-58cd955675-kkdvq\" (UID: \"c5956ba2-b1c3-40dc-b6ae-e20558ab23c6\") " pod="kuadrant-system/authorino-58cd955675-kkdvq" Jun 15 02:39:20.816943 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:20.816914 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hj5h6\" (UniqueName: \"kubernetes.io/projected/c5956ba2-b1c3-40dc-b6ae-e20558ab23c6-kube-api-access-hj5h6\") pod \"authorino-58cd955675-kkdvq\" (UID: \"c5956ba2-b1c3-40dc-b6ae-e20558ab23c6\") " pod="kuadrant-system/authorino-58cd955675-kkdvq" Jun 15 02:39:20.910628 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:20.910574 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bb2mx\" (UniqueName: \"kubernetes.io/projected/42eab444-c243-4e09-b171-9df373d22bcd-kube-api-access-bb2mx\") pod \"authorino-75d556bd77-z8s5z\" (UID: \"42eab444-c243-4e09-b171-9df373d22bcd\") " pod="kuadrant-system/authorino-75d556bd77-z8s5z" Jun 15 02:39:20.918052 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:20.918025 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bb2mx\" (UniqueName: \"kubernetes.io/projected/42eab444-c243-4e09-b171-9df373d22bcd-kube-api-access-bb2mx\") pod \"authorino-75d556bd77-z8s5z\" (UID: \"42eab444-c243-4e09-b171-9df373d22bcd\") " pod="kuadrant-system/authorino-75d556bd77-z8s5z" Jun 15 02:39:20.935809 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:20.935790 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-kkdvq" Jun 15 02:39:21.039908 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:21.039879 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-z8s5z" Jun 15 02:39:21.061135 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:21.061109 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-kkdvq"] Jun 15 02:39:21.061682 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:39:21.061657 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc5956ba2_b1c3_40dc_b6ae_e20558ab23c6.slice/crio-2c8e50d4584e29428225200859706cc0b63e1a6a90221a55697df0732be1aefe WatchSource:0}: Error finding container 2c8e50d4584e29428225200859706cc0b63e1a6a90221a55697df0732be1aefe: Status 404 returned error can't find the container with id 2c8e50d4584e29428225200859706cc0b63e1a6a90221a55697df0732be1aefe Jun 15 02:39:21.163959 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:21.163934 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-z8s5z"] Jun 15 02:39:21.165816 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:39:21.165791 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod42eab444_c243_4e09_b171_9df373d22bcd.slice/crio-73667020574bca2f26ff831e16e5ff5df9beb4a058b0d6793b25b128d7aacfb5 WatchSource:0}: Error finding container 73667020574bca2f26ff831e16e5ff5df9beb4a058b0d6793b25b128d7aacfb5: Status 404 returned error can't find the container with id 73667020574bca2f26ff831e16e5ff5df9beb4a058b0d6793b25b128d7aacfb5 Jun 15 02:39:21.256485 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:21.256457 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-kkdvq" event={"ID":"c5956ba2-b1c3-40dc-b6ae-e20558ab23c6","Type":"ContainerStarted","Data":"2c8e50d4584e29428225200859706cc0b63e1a6a90221a55697df0732be1aefe"} Jun 15 02:39:21.257590 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:21.257567 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-z8s5z" event={"ID":"42eab444-c243-4e09-b171-9df373d22bcd","Type":"ContainerStarted","Data":"73667020574bca2f26ff831e16e5ff5df9beb4a058b0d6793b25b128d7aacfb5"} Jun 15 02:39:25.285304 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:25.285256 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-z8s5z" event={"ID":"42eab444-c243-4e09-b171-9df373d22bcd","Type":"ContainerStarted","Data":"07571a33b942648b646a199b62db8900f9f7450dbc6abea61408797e8c8470f3"} Jun 15 02:39:25.286766 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:25.286742 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-kkdvq" event={"ID":"c5956ba2-b1c3-40dc-b6ae-e20558ab23c6","Type":"ContainerStarted","Data":"f2a51a3e196e03bfd3a9f241ed254dd85591c2408b2deb3ab75617debf2a5865"} Jun 15 02:39:25.298468 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:25.298424 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-z8s5z" podStartSLOduration=1.343812161 podStartE2EDuration="5.298411478s" totalImagesPullingTime="3.954599317s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 02:39:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 02:39:25.29776468 +0000 UTC m=+658.165549425" watchObservedRunningTime="2026-06-15 02:39:25.298411478 +0000 UTC m=+658.166196222" Jun 15 02:39:25.311054 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:25.311009 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-kkdvq" podStartSLOduration=1.24065436 podStartE2EDuration="5.31099584s" totalImagesPullingTime="4.07034148s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 02:39:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 02:39:25.309238608 +0000 UTC m=+658.177023354" watchObservedRunningTime="2026-06-15 02:39:25.31099584 +0000 UTC m=+658.178780585" Jun 15 02:39:25.335204 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:25.335169 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-kkdvq"] Jun 15 02:39:27.298744 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:27.298699 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-kkdvq" podUID="c5956ba2-b1c3-40dc-b6ae-e20558ab23c6" containerName="authorino" containerID="cri-o://f2a51a3e196e03bfd3a9f241ed254dd85591c2408b2deb3ab75617debf2a5865" gracePeriod=30 Jun 15 02:39:27.550244 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:27.550191 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-kkdvq" Jun 15 02:39:27.664744 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:27.664721 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hj5h6\" (UniqueName: \"kubernetes.io/projected/c5956ba2-b1c3-40dc-b6ae-e20558ab23c6-kube-api-access-hj5h6\") pod \"c5956ba2-b1c3-40dc-b6ae-e20558ab23c6\" (UID: \"c5956ba2-b1c3-40dc-b6ae-e20558ab23c6\") " Jun 15 02:39:27.666665 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:27.666626 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5956ba2-b1c3-40dc-b6ae-e20558ab23c6-kube-api-access-hj5h6" (OuterVolumeSpecName: "kube-api-access-hj5h6") pod "c5956ba2-b1c3-40dc-b6ae-e20558ab23c6" (UID: "c5956ba2-b1c3-40dc-b6ae-e20558ab23c6"). InnerVolumeSpecName "kube-api-access-hj5h6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 02:39:27.766022 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:27.765998 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hj5h6\" (UniqueName: \"kubernetes.io/projected/c5956ba2-b1c3-40dc-b6ae-e20558ab23c6-kube-api-access-hj5h6\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:39:28.304768 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:28.304738 2579 generic.go:356] "Generic (PLEG): container finished" podID="c5956ba2-b1c3-40dc-b6ae-e20558ab23c6" containerID="f2a51a3e196e03bfd3a9f241ed254dd85591c2408b2deb3ab75617debf2a5865" exitCode=0 Jun 15 02:39:28.305191 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:28.304775 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-kkdvq" Jun 15 02:39:28.305191 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:28.304827 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-kkdvq" event={"ID":"c5956ba2-b1c3-40dc-b6ae-e20558ab23c6","Type":"ContainerDied","Data":"f2a51a3e196e03bfd3a9f241ed254dd85591c2408b2deb3ab75617debf2a5865"} Jun 15 02:39:28.305191 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:28.304874 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-kkdvq" event={"ID":"c5956ba2-b1c3-40dc-b6ae-e20558ab23c6","Type":"ContainerDied","Data":"2c8e50d4584e29428225200859706cc0b63e1a6a90221a55697df0732be1aefe"} Jun 15 02:39:28.305191 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:28.304899 2579 scope.go:117] "RemoveContainer" containerID="f2a51a3e196e03bfd3a9f241ed254dd85591c2408b2deb3ab75617debf2a5865" Jun 15 02:39:28.313795 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:28.313771 2579 scope.go:117] "RemoveContainer" containerID="f2a51a3e196e03bfd3a9f241ed254dd85591c2408b2deb3ab75617debf2a5865" Jun 15 02:39:28.314034 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:39:28.314017 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2a51a3e196e03bfd3a9f241ed254dd85591c2408b2deb3ab75617debf2a5865\": container with ID starting with f2a51a3e196e03bfd3a9f241ed254dd85591c2408b2deb3ab75617debf2a5865 not found: ID does not exist" containerID="f2a51a3e196e03bfd3a9f241ed254dd85591c2408b2deb3ab75617debf2a5865" Jun 15 02:39:28.314099 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:28.314042 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2a51a3e196e03bfd3a9f241ed254dd85591c2408b2deb3ab75617debf2a5865"} err="failed to get container status \"f2a51a3e196e03bfd3a9f241ed254dd85591c2408b2deb3ab75617debf2a5865\": rpc error: code = NotFound desc = could not find container \"f2a51a3e196e03bfd3a9f241ed254dd85591c2408b2deb3ab75617debf2a5865\": container with ID starting with f2a51a3e196e03bfd3a9f241ed254dd85591c2408b2deb3ab75617debf2a5865 not found: ID does not exist" Jun 15 02:39:28.324424 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:28.324403 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-kkdvq"] Jun 15 02:39:28.331050 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:28.331030 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-kkdvq"] Jun 15 02:39:29.492340 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:29.492299 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5956ba2-b1c3-40dc-b6ae-e20558ab23c6" path="/var/lib/kubelet/pods/c5956ba2-b1c3-40dc-b6ae-e20558ab23c6/volumes" Jun 15 02:39:34.370424 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:34.370388 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-b5d5ff784-9m5g7"] Jun 15 02:39:34.370939 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:34.370919 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5956ba2-b1c3-40dc-b6ae-e20558ab23c6" containerName="authorino" Jun 15 02:39:34.371034 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:34.370955 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5956ba2-b1c3-40dc-b6ae-e20558ab23c6" containerName="authorino" Jun 15 02:39:34.371100 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:34.371077 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5956ba2-b1c3-40dc-b6ae-e20558ab23c6" containerName="authorino" Jun 15 02:39:34.375595 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:34.375573 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b5d5ff784-9m5g7" Jun 15 02:39:34.378511 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:34.378485 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jun 15 02:39:34.378511 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:34.378502 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jun 15 02:39:34.378710 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:34.378516 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-h8zx5\"" Jun 15 02:39:34.378710 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:34.378486 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jun 15 02:39:34.378710 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:34.378550 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jun 15 02:39:34.378710 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:34.378591 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jun 15 02:39:34.378710 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:34.378491 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jun 15 02:39:34.378710 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:34.378491 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jun 15 02:39:34.382191 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:34.382169 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jun 15 02:39:34.389332 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:34.389308 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b5d5ff784-9m5g7"] Jun 15 02:39:34.416523 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:34.416498 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b655b\" (UniqueName: \"kubernetes.io/projected/57191038-ba31-4483-9d09-dc0a07f99dbd-kube-api-access-b655b\") pod \"console-b5d5ff784-9m5g7\" (UID: \"57191038-ba31-4483-9d09-dc0a07f99dbd\") " pod="openshift-console/console-b5d5ff784-9m5g7" Jun 15 02:39:34.416614 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:34.416538 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/57191038-ba31-4483-9d09-dc0a07f99dbd-console-oauth-config\") pod \"console-b5d5ff784-9m5g7\" (UID: \"57191038-ba31-4483-9d09-dc0a07f99dbd\") " pod="openshift-console/console-b5d5ff784-9m5g7" Jun 15 02:39:34.416614 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:34.416562 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57191038-ba31-4483-9d09-dc0a07f99dbd-trusted-ca-bundle\") pod \"console-b5d5ff784-9m5g7\" (UID: \"57191038-ba31-4483-9d09-dc0a07f99dbd\") " pod="openshift-console/console-b5d5ff784-9m5g7" Jun 15 02:39:34.416725 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:34.416628 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/57191038-ba31-4483-9d09-dc0a07f99dbd-console-serving-cert\") pod \"console-b5d5ff784-9m5g7\" (UID: \"57191038-ba31-4483-9d09-dc0a07f99dbd\") " pod="openshift-console/console-b5d5ff784-9m5g7" Jun 15 02:39:34.416725 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:34.416672 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/57191038-ba31-4483-9d09-dc0a07f99dbd-oauth-serving-cert\") pod \"console-b5d5ff784-9m5g7\" (UID: \"57191038-ba31-4483-9d09-dc0a07f99dbd\") " pod="openshift-console/console-b5d5ff784-9m5g7" Jun 15 02:39:34.416725 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:34.416714 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/57191038-ba31-4483-9d09-dc0a07f99dbd-console-config\") pod \"console-b5d5ff784-9m5g7\" (UID: \"57191038-ba31-4483-9d09-dc0a07f99dbd\") " pod="openshift-console/console-b5d5ff784-9m5g7" Jun 15 02:39:34.416866 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:34.416775 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/57191038-ba31-4483-9d09-dc0a07f99dbd-service-ca\") pod \"console-b5d5ff784-9m5g7\" (UID: \"57191038-ba31-4483-9d09-dc0a07f99dbd\") " pod="openshift-console/console-b5d5ff784-9m5g7" Jun 15 02:39:34.517964 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:34.517941 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b655b\" (UniqueName: \"kubernetes.io/projected/57191038-ba31-4483-9d09-dc0a07f99dbd-kube-api-access-b655b\") pod \"console-b5d5ff784-9m5g7\" (UID: \"57191038-ba31-4483-9d09-dc0a07f99dbd\") " pod="openshift-console/console-b5d5ff784-9m5g7" Jun 15 02:39:34.518080 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:34.517984 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/57191038-ba31-4483-9d09-dc0a07f99dbd-console-oauth-config\") pod \"console-b5d5ff784-9m5g7\" (UID: \"57191038-ba31-4483-9d09-dc0a07f99dbd\") " pod="openshift-console/console-b5d5ff784-9m5g7" Jun 15 02:39:34.518080 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:34.518010 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57191038-ba31-4483-9d09-dc0a07f99dbd-trusted-ca-bundle\") pod \"console-b5d5ff784-9m5g7\" (UID: \"57191038-ba31-4483-9d09-dc0a07f99dbd\") " pod="openshift-console/console-b5d5ff784-9m5g7" Jun 15 02:39:34.518080 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:34.518045 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/57191038-ba31-4483-9d09-dc0a07f99dbd-console-serving-cert\") pod \"console-b5d5ff784-9m5g7\" (UID: \"57191038-ba31-4483-9d09-dc0a07f99dbd\") " pod="openshift-console/console-b5d5ff784-9m5g7" Jun 15 02:39:34.518080 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:34.518071 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/57191038-ba31-4483-9d09-dc0a07f99dbd-oauth-serving-cert\") pod \"console-b5d5ff784-9m5g7\" (UID: \"57191038-ba31-4483-9d09-dc0a07f99dbd\") " pod="openshift-console/console-b5d5ff784-9m5g7" Jun 15 02:39:34.518331 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:34.518313 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/57191038-ba31-4483-9d09-dc0a07f99dbd-console-config\") pod \"console-b5d5ff784-9m5g7\" (UID: \"57191038-ba31-4483-9d09-dc0a07f99dbd\") " pod="openshift-console/console-b5d5ff784-9m5g7" Jun 15 02:39:34.518405 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:34.518375 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/57191038-ba31-4483-9d09-dc0a07f99dbd-service-ca\") pod \"console-b5d5ff784-9m5g7\" (UID: \"57191038-ba31-4483-9d09-dc0a07f99dbd\") " pod="openshift-console/console-b5d5ff784-9m5g7" Jun 15 02:39:34.518973 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:34.518946 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/57191038-ba31-4483-9d09-dc0a07f99dbd-oauth-serving-cert\") pod \"console-b5d5ff784-9m5g7\" (UID: \"57191038-ba31-4483-9d09-dc0a07f99dbd\") " pod="openshift-console/console-b5d5ff784-9m5g7" Jun 15 02:39:34.519081 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:34.518980 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/57191038-ba31-4483-9d09-dc0a07f99dbd-console-config\") pod \"console-b5d5ff784-9m5g7\" (UID: \"57191038-ba31-4483-9d09-dc0a07f99dbd\") " pod="openshift-console/console-b5d5ff784-9m5g7" Jun 15 02:39:34.519081 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:34.518982 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/57191038-ba31-4483-9d09-dc0a07f99dbd-service-ca\") pod \"console-b5d5ff784-9m5g7\" (UID: \"57191038-ba31-4483-9d09-dc0a07f99dbd\") " pod="openshift-console/console-b5d5ff784-9m5g7" Jun 15 02:39:34.519202 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:34.519086 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57191038-ba31-4483-9d09-dc0a07f99dbd-trusted-ca-bundle\") pod \"console-b5d5ff784-9m5g7\" (UID: \"57191038-ba31-4483-9d09-dc0a07f99dbd\") " pod="openshift-console/console-b5d5ff784-9m5g7" Jun 15 02:39:34.520442 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:34.520420 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/57191038-ba31-4483-9d09-dc0a07f99dbd-console-oauth-config\") pod \"console-b5d5ff784-9m5g7\" (UID: \"57191038-ba31-4483-9d09-dc0a07f99dbd\") " pod="openshift-console/console-b5d5ff784-9m5g7" Jun 15 02:39:34.520607 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:34.520590 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/57191038-ba31-4483-9d09-dc0a07f99dbd-console-serving-cert\") pod \"console-b5d5ff784-9m5g7\" (UID: \"57191038-ba31-4483-9d09-dc0a07f99dbd\") " pod="openshift-console/console-b5d5ff784-9m5g7" Jun 15 02:39:34.524934 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:34.524911 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b655b\" (UniqueName: \"kubernetes.io/projected/57191038-ba31-4483-9d09-dc0a07f99dbd-kube-api-access-b655b\") pod \"console-b5d5ff784-9m5g7\" (UID: \"57191038-ba31-4483-9d09-dc0a07f99dbd\") " pod="openshift-console/console-b5d5ff784-9m5g7" Jun 15 02:39:34.686654 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:34.686579 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b5d5ff784-9m5g7" Jun 15 02:39:34.821049 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:34.821023 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b5d5ff784-9m5g7"] Jun 15 02:39:34.823015 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:39:34.822985 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57191038_ba31_4483_9d09_dc0a07f99dbd.slice/crio-4fbee6889fa6e1455caa6db5d7abadfcb3e850a94e447be4be7624e2105d97f5 WatchSource:0}: Error finding container 4fbee6889fa6e1455caa6db5d7abadfcb3e850a94e447be4be7624e2105d97f5: Status 404 returned error can't find the container with id 4fbee6889fa6e1455caa6db5d7abadfcb3e850a94e447be4be7624e2105d97f5 Jun 15 02:39:35.352516 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:35.352476 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b5d5ff784-9m5g7" event={"ID":"57191038-ba31-4483-9d09-dc0a07f99dbd","Type":"ContainerStarted","Data":"60feea44456916907d28d92c194b58100e23e5742ca6162bed610c065d86b7ed"} Jun 15 02:39:35.352708 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:35.352553 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b5d5ff784-9m5g7" event={"ID":"57191038-ba31-4483-9d09-dc0a07f99dbd","Type":"ContainerStarted","Data":"4fbee6889fa6e1455caa6db5d7abadfcb3e850a94e447be4be7624e2105d97f5"} Jun 15 02:39:35.369601 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:35.369556 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-b5d5ff784-9m5g7" podStartSLOduration=1.369542719 podStartE2EDuration="1.369542719s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 02:39:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 02:39:35.367418694 +0000 UTC m=+668.235203439" watchObservedRunningTime="2026-06-15 02:39:35.369542719 +0000 UTC m=+668.237327464" Jun 15 02:39:44.686962 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:44.686928 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-b5d5ff784-9m5g7" Jun 15 02:39:44.687339 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:44.686982 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-b5d5ff784-9m5g7" Jun 15 02:39:44.691582 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:44.691558 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-b5d5ff784-9m5g7" Jun 15 02:39:45.406882 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:45.406856 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-b5d5ff784-9m5g7" Jun 15 02:39:47.434966 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:47.434931 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748c4p2c"] Jun 15 02:39:47.438373 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:47.438356 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748c4p2c" Jun 15 02:39:47.440551 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:47.440525 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 15 02:39:47.441395 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:47.441377 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 15 02:39:47.441497 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:47.441382 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-mkns2\"" Jun 15 02:39:47.445740 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:47.445712 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748c4p2c"] Jun 15 02:39:47.520879 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:47.520855 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/671eac45-8d2a-4bf0-9a34-053a88932bb7-util\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748c4p2c\" (UID: \"671eac45-8d2a-4bf0-9a34-053a88932bb7\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748c4p2c" Jun 15 02:39:47.520993 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:47.520884 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/671eac45-8d2a-4bf0-9a34-053a88932bb7-bundle\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748c4p2c\" (UID: \"671eac45-8d2a-4bf0-9a34-053a88932bb7\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748c4p2c" Jun 15 02:39:47.520993 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:47.520918 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwh5q\" (UniqueName: \"kubernetes.io/projected/671eac45-8d2a-4bf0-9a34-053a88932bb7-kube-api-access-xwh5q\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748c4p2c\" (UID: \"671eac45-8d2a-4bf0-9a34-053a88932bb7\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748c4p2c" Jun 15 02:39:47.621237 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:47.621213 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/671eac45-8d2a-4bf0-9a34-053a88932bb7-util\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748c4p2c\" (UID: \"671eac45-8d2a-4bf0-9a34-053a88932bb7\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748c4p2c" Jun 15 02:39:47.621331 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:47.621243 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/671eac45-8d2a-4bf0-9a34-053a88932bb7-bundle\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748c4p2c\" (UID: \"671eac45-8d2a-4bf0-9a34-053a88932bb7\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748c4p2c" Jun 15 02:39:47.621331 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:47.621273 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwh5q\" (UniqueName: \"kubernetes.io/projected/671eac45-8d2a-4bf0-9a34-053a88932bb7-kube-api-access-xwh5q\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748c4p2c\" (UID: \"671eac45-8d2a-4bf0-9a34-053a88932bb7\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748c4p2c" Jun 15 02:39:47.621547 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:47.621526 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/671eac45-8d2a-4bf0-9a34-053a88932bb7-util\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748c4p2c\" (UID: \"671eac45-8d2a-4bf0-9a34-053a88932bb7\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748c4p2c" Jun 15 02:39:47.621614 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:47.621572 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/671eac45-8d2a-4bf0-9a34-053a88932bb7-bundle\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748c4p2c\" (UID: \"671eac45-8d2a-4bf0-9a34-053a88932bb7\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748c4p2c" Jun 15 02:39:47.628899 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:47.628877 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwh5q\" (UniqueName: \"kubernetes.io/projected/671eac45-8d2a-4bf0-9a34-053a88932bb7-kube-api-access-xwh5q\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748c4p2c\" (UID: \"671eac45-8d2a-4bf0-9a34-053a88932bb7\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748c4p2c" Jun 15 02:39:47.749522 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:47.749503 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748c4p2c" Jun 15 02:39:47.877022 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:47.876998 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748c4p2c"] Jun 15 02:39:47.878469 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:39:47.878436 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod671eac45_8d2a_4bf0_9a34_053a88932bb7.slice/crio-1705fd553e7016c25c8117dad13ff0931e7ea578531708d2148afebf81168a6d WatchSource:0}: Error finding container 1705fd553e7016c25c8117dad13ff0931e7ea578531708d2148afebf81168a6d: Status 404 returned error can't find the container with id 1705fd553e7016c25c8117dad13ff0931e7ea578531708d2148afebf81168a6d Jun 15 02:39:48.427517 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:48.427485 2579 generic.go:356] "Generic (PLEG): container finished" podID="671eac45-8d2a-4bf0-9a34-053a88932bb7" containerID="6068a7375c582e440cb367c0705f23d736b369562c72a7178f15809bb09d7d51" exitCode=0 Jun 15 02:39:48.427700 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:48.427570 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748c4p2c" event={"ID":"671eac45-8d2a-4bf0-9a34-053a88932bb7","Type":"ContainerDied","Data":"6068a7375c582e440cb367c0705f23d736b369562c72a7178f15809bb09d7d51"} Jun 15 02:39:48.427700 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:48.427616 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748c4p2c" event={"ID":"671eac45-8d2a-4bf0-9a34-053a88932bb7","Type":"ContainerStarted","Data":"1705fd553e7016c25c8117dad13ff0931e7ea578531708d2148afebf81168a6d"} Jun 15 02:39:48.428494 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:48.428478 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 15 02:39:49.107499 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:39:49.107472 2579 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod671eac45_8d2a_4bf0_9a34_053a88932bb7.slice/crio-conmon-61a4f6e3fd907ce0a890a717fb02b2474345e400e2a4bb5d8a785e72fd4dbe4a.scope\": RecentStats: unable to find data in memory cache]" Jun 15 02:39:49.434918 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:49.434832 2579 generic.go:356] "Generic (PLEG): container finished" podID="671eac45-8d2a-4bf0-9a34-053a88932bb7" containerID="61a4f6e3fd907ce0a890a717fb02b2474345e400e2a4bb5d8a785e72fd4dbe4a" exitCode=0 Jun 15 02:39:49.435062 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:49.434916 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748c4p2c" event={"ID":"671eac45-8d2a-4bf0-9a34-053a88932bb7","Type":"ContainerDied","Data":"61a4f6e3fd907ce0a890a717fb02b2474345e400e2a4bb5d8a785e72fd4dbe4a"} Jun 15 02:39:50.442677 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:50.442624 2579 generic.go:356] "Generic (PLEG): container finished" podID="671eac45-8d2a-4bf0-9a34-053a88932bb7" containerID="002f2b221519bf8e3fa57cec9836417ad0cfee606564793fea9cd99806a41701" exitCode=0 Jun 15 02:39:50.443082 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:50.442698 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748c4p2c" event={"ID":"671eac45-8d2a-4bf0-9a34-053a88932bb7","Type":"ContainerDied","Data":"002f2b221519bf8e3fa57cec9836417ad0cfee606564793fea9cd99806a41701"} Jun 15 02:39:51.573416 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:51.573393 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748c4p2c" Jun 15 02:39:51.654774 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:51.654747 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/671eac45-8d2a-4bf0-9a34-053a88932bb7-util\") pod \"671eac45-8d2a-4bf0-9a34-053a88932bb7\" (UID: \"671eac45-8d2a-4bf0-9a34-053a88932bb7\") " Jun 15 02:39:51.654888 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:51.654779 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xwh5q\" (UniqueName: \"kubernetes.io/projected/671eac45-8d2a-4bf0-9a34-053a88932bb7-kube-api-access-xwh5q\") pod \"671eac45-8d2a-4bf0-9a34-053a88932bb7\" (UID: \"671eac45-8d2a-4bf0-9a34-053a88932bb7\") " Jun 15 02:39:51.654888 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:51.654853 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/671eac45-8d2a-4bf0-9a34-053a88932bb7-bundle\") pod \"671eac45-8d2a-4bf0-9a34-053a88932bb7\" (UID: \"671eac45-8d2a-4bf0-9a34-053a88932bb7\") " Jun 15 02:39:51.655464 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:51.655437 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/671eac45-8d2a-4bf0-9a34-053a88932bb7-bundle" (OuterVolumeSpecName: "bundle") pod "671eac45-8d2a-4bf0-9a34-053a88932bb7" (UID: "671eac45-8d2a-4bf0-9a34-053a88932bb7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 02:39:51.656867 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:51.656844 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/671eac45-8d2a-4bf0-9a34-053a88932bb7-kube-api-access-xwh5q" (OuterVolumeSpecName: "kube-api-access-xwh5q") pod "671eac45-8d2a-4bf0-9a34-053a88932bb7" (UID: "671eac45-8d2a-4bf0-9a34-053a88932bb7"). InnerVolumeSpecName "kube-api-access-xwh5q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 02:39:51.660520 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:51.660502 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/671eac45-8d2a-4bf0-9a34-053a88932bb7-util" (OuterVolumeSpecName: "util") pod "671eac45-8d2a-4bf0-9a34-053a88932bb7" (UID: "671eac45-8d2a-4bf0-9a34-053a88932bb7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 02:39:51.755997 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:51.755977 2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/671eac45-8d2a-4bf0-9a34-053a88932bb7-util\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:39:51.755997 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:51.755997 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xwh5q\" (UniqueName: \"kubernetes.io/projected/671eac45-8d2a-4bf0-9a34-053a88932bb7-kube-api-access-xwh5q\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:39:51.756130 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:51.756007 2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/671eac45-8d2a-4bf0-9a34-053a88932bb7-bundle\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:39:52.455905 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:52.455879 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748c4p2c" Jun 15 02:39:52.456062 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:52.455909 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748c4p2c" event={"ID":"671eac45-8d2a-4bf0-9a34-053a88932bb7","Type":"ContainerDied","Data":"1705fd553e7016c25c8117dad13ff0931e7ea578531708d2148afebf81168a6d"} Jun 15 02:39:52.456062 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:39:52.455954 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1705fd553e7016c25c8117dad13ff0931e7ea578531708d2148afebf81168a6d" Jun 15 02:40:40.300020 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:40.299987 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-snk46"] Jun 15 02:40:40.300472 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:40.300328 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="671eac45-8d2a-4bf0-9a34-053a88932bb7" containerName="pull" Jun 15 02:40:40.300472 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:40.300346 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="671eac45-8d2a-4bf0-9a34-053a88932bb7" containerName="pull" Jun 15 02:40:40.300472 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:40.300376 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="671eac45-8d2a-4bf0-9a34-053a88932bb7" containerName="util" Jun 15 02:40:40.300472 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:40.300384 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="671eac45-8d2a-4bf0-9a34-053a88932bb7" containerName="util" Jun 15 02:40:40.300472 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:40.300393 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="671eac45-8d2a-4bf0-9a34-053a88932bb7" containerName="extract" Jun 15 02:40:40.300472 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:40.300404 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="671eac45-8d2a-4bf0-9a34-053a88932bb7" containerName="extract" Jun 15 02:40:40.300703 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:40.300511 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="671eac45-8d2a-4bf0-9a34-053a88932bb7" containerName="extract" Jun 15 02:40:40.305067 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:40.305040 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-snk46" Jun 15 02:40:40.307986 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:40.307957 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-snk46"] Jun 15 02:40:40.400263 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:40.400240 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dpkt\" (UniqueName: \"kubernetes.io/projected/c888dea1-aa61-4b57-b3c9-e39c5f7ad060-kube-api-access-2dpkt\") pod \"authorino-5d4b67c446-snk46\" (UID: \"c888dea1-aa61-4b57-b3c9-e39c5f7ad060\") " pod="kuadrant-system/authorino-5d4b67c446-snk46" Jun 15 02:40:40.500835 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:40.500810 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dpkt\" (UniqueName: \"kubernetes.io/projected/c888dea1-aa61-4b57-b3c9-e39c5f7ad060-kube-api-access-2dpkt\") pod \"authorino-5d4b67c446-snk46\" (UID: \"c888dea1-aa61-4b57-b3c9-e39c5f7ad060\") " pod="kuadrant-system/authorino-5d4b67c446-snk46" Jun 15 02:40:40.511113 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:40.511088 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dpkt\" (UniqueName: \"kubernetes.io/projected/c888dea1-aa61-4b57-b3c9-e39c5f7ad060-kube-api-access-2dpkt\") pod \"authorino-5d4b67c446-snk46\" (UID: \"c888dea1-aa61-4b57-b3c9-e39c5f7ad060\") " pod="kuadrant-system/authorino-5d4b67c446-snk46" Jun 15 02:40:40.530355 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:40.530330 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-snk46"] Jun 15 02:40:40.530536 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:40.530523 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-snk46" Jun 15 02:40:40.555034 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:40.554975 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-55489d8f4c-kxrtn"] Jun 15 02:40:40.558669 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:40.558650 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-55489d8f4c-kxrtn" Jun 15 02:40:40.569425 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:40.569404 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-55489d8f4c-kxrtn"] Jun 15 02:40:40.601513 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:40.601442 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htsnw\" (UniqueName: \"kubernetes.io/projected/758afe47-fee0-4e7a-8a4d-1f524ec62456-kube-api-access-htsnw\") pod \"authorino-55489d8f4c-kxrtn\" (UID: \"758afe47-fee0-4e7a-8a4d-1f524ec62456\") " pod="kuadrant-system/authorino-55489d8f4c-kxrtn" Jun 15 02:40:40.651531 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:40.651503 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-55489d8f4c-kxrtn"] Jun 15 02:40:40.651802 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:40:40.651781 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-htsnw], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-55489d8f4c-kxrtn" podUID="758afe47-fee0-4e7a-8a4d-1f524ec62456" Jun 15 02:40:40.662981 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:40.662961 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-snk46"] Jun 15 02:40:40.665308 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:40:40.665284 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc888dea1_aa61_4b57_b3c9_e39c5f7ad060.slice/crio-d1cde1a88be51b7c4cf720bd9e78a471f550e33d7fa474bdf1ab4b212d7f4a01 WatchSource:0}: Error finding container d1cde1a88be51b7c4cf720bd9e78a471f550e33d7fa474bdf1ab4b212d7f4a01: Status 404 returned error can't find the container with id d1cde1a88be51b7c4cf720bd9e78a471f550e33d7fa474bdf1ab4b212d7f4a01 Jun 15 02:40:40.680942 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:40.680916 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-778cd84b87-7nhtj"] Jun 15 02:40:40.684369 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:40.684353 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-778cd84b87-7nhtj" Jun 15 02:40:40.687589 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:40.687572 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jun 15 02:40:40.702197 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:40.702173 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-778cd84b87-7nhtj"] Jun 15 02:40:40.702469 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:40.702451 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/68fbb99e-9bbb-457d-b3e9-4c584ba5a07f-tls-cert\") pod \"authorino-778cd84b87-7nhtj\" (UID: \"68fbb99e-9bbb-457d-b3e9-4c584ba5a07f\") " pod="kuadrant-system/authorino-778cd84b87-7nhtj" Jun 15 02:40:40.702562 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:40.702501 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htsnw\" (UniqueName: \"kubernetes.io/projected/758afe47-fee0-4e7a-8a4d-1f524ec62456-kube-api-access-htsnw\") pod \"authorino-55489d8f4c-kxrtn\" (UID: \"758afe47-fee0-4e7a-8a4d-1f524ec62456\") " pod="kuadrant-system/authorino-55489d8f4c-kxrtn" Jun 15 02:40:40.702624 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:40.702578 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qhqb\" (UniqueName: \"kubernetes.io/projected/68fbb99e-9bbb-457d-b3e9-4c584ba5a07f-kube-api-access-8qhqb\") pod \"authorino-778cd84b87-7nhtj\" (UID: \"68fbb99e-9bbb-457d-b3e9-4c584ba5a07f\") " pod="kuadrant-system/authorino-778cd84b87-7nhtj" Jun 15 02:40:40.704424 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:40.704404 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-55489d8f4c-kxrtn" Jun 15 02:40:40.704523 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:40.704420 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-snk46" event={"ID":"c888dea1-aa61-4b57-b3c9-e39c5f7ad060","Type":"ContainerStarted","Data":"d1cde1a88be51b7c4cf720bd9e78a471f550e33d7fa474bdf1ab4b212d7f4a01"} Jun 15 02:40:40.710535 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:40.710498 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-55489d8f4c-kxrtn" Jun 15 02:40:40.717786 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:40.717766 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-htsnw\" (UniqueName: \"kubernetes.io/projected/758afe47-fee0-4e7a-8a4d-1f524ec62456-kube-api-access-htsnw\") pod \"authorino-55489d8f4c-kxrtn\" (UID: \"758afe47-fee0-4e7a-8a4d-1f524ec62456\") " pod="kuadrant-system/authorino-55489d8f4c-kxrtn" Jun 15 02:40:40.803419 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:40.803401 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-htsnw\" (UniqueName: \"kubernetes.io/projected/758afe47-fee0-4e7a-8a4d-1f524ec62456-kube-api-access-htsnw\") pod \"758afe47-fee0-4e7a-8a4d-1f524ec62456\" (UID: \"758afe47-fee0-4e7a-8a4d-1f524ec62456\") " Jun 15 02:40:40.803594 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:40.803576 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/68fbb99e-9bbb-457d-b3e9-4c584ba5a07f-tls-cert\") pod \"authorino-778cd84b87-7nhtj\" (UID: \"68fbb99e-9bbb-457d-b3e9-4c584ba5a07f\") " pod="kuadrant-system/authorino-778cd84b87-7nhtj" Jun 15 02:40:40.803698 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:40.803682 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qhqb\" (UniqueName: \"kubernetes.io/projected/68fbb99e-9bbb-457d-b3e9-4c584ba5a07f-kube-api-access-8qhqb\") pod \"authorino-778cd84b87-7nhtj\" (UID: \"68fbb99e-9bbb-457d-b3e9-4c584ba5a07f\") " pod="kuadrant-system/authorino-778cd84b87-7nhtj" Jun 15 02:40:40.805491 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:40.805430 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/758afe47-fee0-4e7a-8a4d-1f524ec62456-kube-api-access-htsnw" (OuterVolumeSpecName: "kube-api-access-htsnw") pod "758afe47-fee0-4e7a-8a4d-1f524ec62456" (UID: "758afe47-fee0-4e7a-8a4d-1f524ec62456"). InnerVolumeSpecName "kube-api-access-htsnw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 02:40:40.805872 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:40.805856 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/68fbb99e-9bbb-457d-b3e9-4c584ba5a07f-tls-cert\") pod \"authorino-778cd84b87-7nhtj\" (UID: \"68fbb99e-9bbb-457d-b3e9-4c584ba5a07f\") " pod="kuadrant-system/authorino-778cd84b87-7nhtj" Jun 15 02:40:40.811301 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:40.811282 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qhqb\" (UniqueName: \"kubernetes.io/projected/68fbb99e-9bbb-457d-b3e9-4c584ba5a07f-kube-api-access-8qhqb\") pod \"authorino-778cd84b87-7nhtj\" (UID: \"68fbb99e-9bbb-457d-b3e9-4c584ba5a07f\") " pod="kuadrant-system/authorino-778cd84b87-7nhtj" Jun 15 02:40:40.904560 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:40.904532 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-htsnw\" (UniqueName: \"kubernetes.io/projected/758afe47-fee0-4e7a-8a4d-1f524ec62456-kube-api-access-htsnw\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:40:40.993902 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:40.993876 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-778cd84b87-7nhtj" Jun 15 02:40:41.124855 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:41.124829 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-778cd84b87-7nhtj"] Jun 15 02:40:41.127376 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:40:41.127349 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod68fbb99e_9bbb_457d_b3e9_4c584ba5a07f.slice/crio-de058e5cf85d4ec84018668c4e135fabe02022f037a93ea499ad7e0577507329 WatchSource:0}: Error finding container de058e5cf85d4ec84018668c4e135fabe02022f037a93ea499ad7e0577507329: Status 404 returned error can't find the container with id de058e5cf85d4ec84018668c4e135fabe02022f037a93ea499ad7e0577507329 Jun 15 02:40:41.712195 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:41.712111 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-778cd84b87-7nhtj" event={"ID":"68fbb99e-9bbb-457d-b3e9-4c584ba5a07f","Type":"ContainerStarted","Data":"5eb75b7e95d9a2bf01303bea56242dec0bf1ea6715953d8d87ac26ad223cfcfd"} Jun 15 02:40:41.712195 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:41.712156 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-778cd84b87-7nhtj" event={"ID":"68fbb99e-9bbb-457d-b3e9-4c584ba5a07f","Type":"ContainerStarted","Data":"de058e5cf85d4ec84018668c4e135fabe02022f037a93ea499ad7e0577507329"} Jun 15 02:40:41.713560 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:41.713538 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-55489d8f4c-kxrtn" Jun 15 02:40:41.713716 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:41.713578 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-snk46" podUID="c888dea1-aa61-4b57-b3c9-e39c5f7ad060" containerName="authorino" containerID="cri-o://f4297a3ee91fae174ffe96ac78e096afebacac060b665f118ce43e4cd71ba52d" gracePeriod=30 Jun 15 02:40:41.713716 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:41.713540 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-snk46" event={"ID":"c888dea1-aa61-4b57-b3c9-e39c5f7ad060","Type":"ContainerStarted","Data":"f4297a3ee91fae174ffe96ac78e096afebacac060b665f118ce43e4cd71ba52d"} Jun 15 02:40:41.727886 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:41.727837 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-778cd84b87-7nhtj" podStartSLOduration=1.439099264 podStartE2EDuration="1.727820784s" totalImagesPullingTime="288.72152ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 02:40:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 02:40:41.726463454 +0000 UTC m=+734.594248200" watchObservedRunningTime="2026-06-15 02:40:41.727820784 +0000 UTC m=+734.595605533" Jun 15 02:40:41.741175 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:41.741134 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-snk46" podStartSLOduration=1.375309252 podStartE2EDuration="1.741118423s" totalImagesPullingTime="365.809171ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 02:40:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 02:40:41.73935293 +0000 UTC m=+734.607137679" watchObservedRunningTime="2026-06-15 02:40:41.741118423 +0000 UTC m=+734.608903172" Jun 15 02:40:41.752662 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:41.752617 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-z8s5z"] Jun 15 02:40:41.752799 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:41.752780 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-z8s5z" podUID="42eab444-c243-4e09-b171-9df373d22bcd" containerName="authorino" containerID="cri-o://07571a33b942648b646a199b62db8900f9f7450dbc6abea61408797e8c8470f3" gracePeriod=30 Jun 15 02:40:41.764824 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:41.764790 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-55489d8f4c-kxrtn"] Jun 15 02:40:41.767329 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:41.767307 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-55489d8f4c-kxrtn"] Jun 15 02:40:42.009832 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:42.009808 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-snk46" Jun 15 02:40:42.012781 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:42.012762 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-z8s5z" Jun 15 02:40:42.114375 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:42.114347 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2dpkt\" (UniqueName: \"kubernetes.io/projected/c888dea1-aa61-4b57-b3c9-e39c5f7ad060-kube-api-access-2dpkt\") pod \"c888dea1-aa61-4b57-b3c9-e39c5f7ad060\" (UID: \"c888dea1-aa61-4b57-b3c9-e39c5f7ad060\") " Jun 15 02:40:42.114532 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:42.114386 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bb2mx\" (UniqueName: \"kubernetes.io/projected/42eab444-c243-4e09-b171-9df373d22bcd-kube-api-access-bb2mx\") pod \"42eab444-c243-4e09-b171-9df373d22bcd\" (UID: \"42eab444-c243-4e09-b171-9df373d22bcd\") " Jun 15 02:40:42.116411 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:42.116381 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c888dea1-aa61-4b57-b3c9-e39c5f7ad060-kube-api-access-2dpkt" (OuterVolumeSpecName: "kube-api-access-2dpkt") pod "c888dea1-aa61-4b57-b3c9-e39c5f7ad060" (UID: "c888dea1-aa61-4b57-b3c9-e39c5f7ad060"). InnerVolumeSpecName "kube-api-access-2dpkt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 02:40:42.116411 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:42.116390 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42eab444-c243-4e09-b171-9df373d22bcd-kube-api-access-bb2mx" (OuterVolumeSpecName: "kube-api-access-bb2mx") pod "42eab444-c243-4e09-b171-9df373d22bcd" (UID: "42eab444-c243-4e09-b171-9df373d22bcd"). InnerVolumeSpecName "kube-api-access-bb2mx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 02:40:42.215848 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:42.215822 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2dpkt\" (UniqueName: \"kubernetes.io/projected/c888dea1-aa61-4b57-b3c9-e39c5f7ad060-kube-api-access-2dpkt\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:40:42.215848 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:42.215848 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bb2mx\" (UniqueName: \"kubernetes.io/projected/42eab444-c243-4e09-b171-9df373d22bcd-kube-api-access-bb2mx\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:40:42.720070 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:42.720040 2579 generic.go:356] "Generic (PLEG): container finished" podID="c888dea1-aa61-4b57-b3c9-e39c5f7ad060" containerID="f4297a3ee91fae174ffe96ac78e096afebacac060b665f118ce43e4cd71ba52d" exitCode=0 Jun 15 02:40:42.720070 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:42.720058 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-snk46" event={"ID":"c888dea1-aa61-4b57-b3c9-e39c5f7ad060","Type":"ContainerDied","Data":"f4297a3ee91fae174ffe96ac78e096afebacac060b665f118ce43e4cd71ba52d"} Jun 15 02:40:42.720572 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:42.720118 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-snk46" event={"ID":"c888dea1-aa61-4b57-b3c9-e39c5f7ad060","Type":"ContainerDied","Data":"d1cde1a88be51b7c4cf720bd9e78a471f550e33d7fa474bdf1ab4b212d7f4a01"} Jun 15 02:40:42.720572 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:42.720070 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-snk46" Jun 15 02:40:42.720572 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:42.720140 2579 scope.go:117] "RemoveContainer" containerID="f4297a3ee91fae174ffe96ac78e096afebacac060b665f118ce43e4cd71ba52d" Jun 15 02:40:42.721582 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:42.721405 2579 generic.go:356] "Generic (PLEG): container finished" podID="42eab444-c243-4e09-b171-9df373d22bcd" containerID="07571a33b942648b646a199b62db8900f9f7450dbc6abea61408797e8c8470f3" exitCode=0 Jun 15 02:40:42.721582 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:42.721444 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-z8s5z" Jun 15 02:40:42.721582 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:42.721487 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-z8s5z" event={"ID":"42eab444-c243-4e09-b171-9df373d22bcd","Type":"ContainerDied","Data":"07571a33b942648b646a199b62db8900f9f7450dbc6abea61408797e8c8470f3"} Jun 15 02:40:42.721582 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:42.721540 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-z8s5z" event={"ID":"42eab444-c243-4e09-b171-9df373d22bcd","Type":"ContainerDied","Data":"73667020574bca2f26ff831e16e5ff5df9beb4a058b0d6793b25b128d7aacfb5"} Jun 15 02:40:42.730554 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:42.730538 2579 scope.go:117] "RemoveContainer" containerID="f4297a3ee91fae174ffe96ac78e096afebacac060b665f118ce43e4cd71ba52d" Jun 15 02:40:42.730875 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:40:42.730848 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4297a3ee91fae174ffe96ac78e096afebacac060b665f118ce43e4cd71ba52d\": container with ID starting with f4297a3ee91fae174ffe96ac78e096afebacac060b665f118ce43e4cd71ba52d not found: ID does not exist" containerID="f4297a3ee91fae174ffe96ac78e096afebacac060b665f118ce43e4cd71ba52d" Jun 15 02:40:42.730983 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:42.730887 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4297a3ee91fae174ffe96ac78e096afebacac060b665f118ce43e4cd71ba52d"} err="failed to get container status \"f4297a3ee91fae174ffe96ac78e096afebacac060b665f118ce43e4cd71ba52d\": rpc error: code = NotFound desc = could not find container \"f4297a3ee91fae174ffe96ac78e096afebacac060b665f118ce43e4cd71ba52d\": container with ID starting with f4297a3ee91fae174ffe96ac78e096afebacac060b665f118ce43e4cd71ba52d not found: ID does not exist" Jun 15 02:40:42.730983 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:42.730909 2579 scope.go:117] "RemoveContainer" containerID="07571a33b942648b646a199b62db8900f9f7450dbc6abea61408797e8c8470f3" Jun 15 02:40:42.738812 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:42.738795 2579 scope.go:117] "RemoveContainer" containerID="07571a33b942648b646a199b62db8900f9f7450dbc6abea61408797e8c8470f3" Jun 15 02:40:42.739053 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:40:42.739033 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07571a33b942648b646a199b62db8900f9f7450dbc6abea61408797e8c8470f3\": container with ID starting with 07571a33b942648b646a199b62db8900f9f7450dbc6abea61408797e8c8470f3 not found: ID does not exist" containerID="07571a33b942648b646a199b62db8900f9f7450dbc6abea61408797e8c8470f3" Jun 15 02:40:42.739118 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:42.739063 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07571a33b942648b646a199b62db8900f9f7450dbc6abea61408797e8c8470f3"} err="failed to get container status \"07571a33b942648b646a199b62db8900f9f7450dbc6abea61408797e8c8470f3\": rpc error: code = NotFound desc = could not find container \"07571a33b942648b646a199b62db8900f9f7450dbc6abea61408797e8c8470f3\": container with ID starting with 07571a33b942648b646a199b62db8900f9f7450dbc6abea61408797e8c8470f3 not found: ID does not exist" Jun 15 02:40:42.741102 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:42.741083 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-snk46"] Jun 15 02:40:42.743323 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:42.743306 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-snk46"] Jun 15 02:40:42.753406 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:42.753389 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-z8s5z"] Jun 15 02:40:42.756347 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:42.756327 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-z8s5z"] Jun 15 02:40:43.492783 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:43.492755 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42eab444-c243-4e09-b171-9df373d22bcd" path="/var/lib/kubelet/pods/42eab444-c243-4e09-b171-9df373d22bcd/volumes" Jun 15 02:40:43.493067 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:43.493055 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="758afe47-fee0-4e7a-8a4d-1f524ec62456" path="/var/lib/kubelet/pods/758afe47-fee0-4e7a-8a4d-1f524ec62456/volumes" Jun 15 02:40:43.493250 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:43.493239 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c888dea1-aa61-4b57-b3c9-e39c5f7ad060" path="/var/lib/kubelet/pods/c888dea1-aa61-4b57-b3c9-e39c5f7ad060/volumes" Jun 15 02:40:45.230729 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:45.230699 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-58978d5c5f-dlk6q"] Jun 15 02:40:45.231103 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:45.231023 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c888dea1-aa61-4b57-b3c9-e39c5f7ad060" containerName="authorino" Jun 15 02:40:45.231103 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:45.231034 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c888dea1-aa61-4b57-b3c9-e39c5f7ad060" containerName="authorino" Jun 15 02:40:45.231103 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:45.231044 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42eab444-c243-4e09-b171-9df373d22bcd" containerName="authorino" Jun 15 02:40:45.231103 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:45.231049 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="42eab444-c243-4e09-b171-9df373d22bcd" containerName="authorino" Jun 15 02:40:45.231469 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:45.231105 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="42eab444-c243-4e09-b171-9df373d22bcd" containerName="authorino" Jun 15 02:40:45.231469 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:45.231123 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c888dea1-aa61-4b57-b3c9-e39c5f7ad060" containerName="authorino" Jun 15 02:40:45.234558 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:45.234541 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58978d5c5f-dlk6q" Jun 15 02:40:45.237404 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:45.237382 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-mqtvz\"" Jun 15 02:40:45.237404 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:45.237402 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 15 02:40:45.237519 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:45.237445 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jun 15 02:40:45.250872 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:45.250837 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-58978d5c5f-dlk6q"] Jun 15 02:40:45.337498 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:45.337468 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzf77\" (UniqueName: \"kubernetes.io/projected/06da59a0-e47e-42bd-bafb-f3e51de30541-kube-api-access-lzf77\") pod \"maas-controller-58978d5c5f-dlk6q\" (UID: \"06da59a0-e47e-42bd-bafb-f3e51de30541\") " pod="opendatahub/maas-controller-58978d5c5f-dlk6q" Jun 15 02:40:45.337677 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:45.337521 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/06da59a0-e47e-42bd-bafb-f3e51de30541-webhook-cert\") pod \"maas-controller-58978d5c5f-dlk6q\" (UID: \"06da59a0-e47e-42bd-bafb-f3e51de30541\") " pod="opendatahub/maas-controller-58978d5c5f-dlk6q" Jun 15 02:40:45.401663 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:45.401621 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-ddf7649c7-jxwp6"] Jun 15 02:40:45.404919 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:45.404897 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-ddf7649c7-jxwp6" Jun 15 02:40:45.414325 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:45.414304 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-ddf7649c7-jxwp6"] Jun 15 02:40:45.438691 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:45.438668 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/06da59a0-e47e-42bd-bafb-f3e51de30541-webhook-cert\") pod \"maas-controller-58978d5c5f-dlk6q\" (UID: \"06da59a0-e47e-42bd-bafb-f3e51de30541\") " pod="opendatahub/maas-controller-58978d5c5f-dlk6q" Jun 15 02:40:45.438798 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:45.438755 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/30e800c7-b85f-4404-be63-e9ad90b8dee1-webhook-cert\") pod \"maas-controller-ddf7649c7-jxwp6\" (UID: \"30e800c7-b85f-4404-be63-e9ad90b8dee1\") " pod="opendatahub/maas-controller-ddf7649c7-jxwp6" Jun 15 02:40:45.438798 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:45.438793 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrhzc\" (UniqueName: \"kubernetes.io/projected/30e800c7-b85f-4404-be63-e9ad90b8dee1-kube-api-access-mrhzc\") pod \"maas-controller-ddf7649c7-jxwp6\" (UID: \"30e800c7-b85f-4404-be63-e9ad90b8dee1\") " pod="opendatahub/maas-controller-ddf7649c7-jxwp6" Jun 15 02:40:45.438918 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:45.438848 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzf77\" (UniqueName: \"kubernetes.io/projected/06da59a0-e47e-42bd-bafb-f3e51de30541-kube-api-access-lzf77\") pod \"maas-controller-58978d5c5f-dlk6q\" (UID: \"06da59a0-e47e-42bd-bafb-f3e51de30541\") " pod="opendatahub/maas-controller-58978d5c5f-dlk6q" Jun 15 02:40:45.440959 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:45.440942 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/06da59a0-e47e-42bd-bafb-f3e51de30541-webhook-cert\") pod \"maas-controller-58978d5c5f-dlk6q\" (UID: \"06da59a0-e47e-42bd-bafb-f3e51de30541\") " pod="opendatahub/maas-controller-58978d5c5f-dlk6q" Jun 15 02:40:45.446237 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:45.446216 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzf77\" (UniqueName: \"kubernetes.io/projected/06da59a0-e47e-42bd-bafb-f3e51de30541-kube-api-access-lzf77\") pod \"maas-controller-58978d5c5f-dlk6q\" (UID: \"06da59a0-e47e-42bd-bafb-f3e51de30541\") " pod="opendatahub/maas-controller-58978d5c5f-dlk6q" Jun 15 02:40:45.540103 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:45.540040 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/30e800c7-b85f-4404-be63-e9ad90b8dee1-webhook-cert\") pod \"maas-controller-ddf7649c7-jxwp6\" (UID: \"30e800c7-b85f-4404-be63-e9ad90b8dee1\") " pod="opendatahub/maas-controller-ddf7649c7-jxwp6" Jun 15 02:40:45.540103 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:45.540072 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrhzc\" (UniqueName: \"kubernetes.io/projected/30e800c7-b85f-4404-be63-e9ad90b8dee1-kube-api-access-mrhzc\") pod \"maas-controller-ddf7649c7-jxwp6\" (UID: \"30e800c7-b85f-4404-be63-e9ad90b8dee1\") " pod="opendatahub/maas-controller-ddf7649c7-jxwp6" Jun 15 02:40:45.542324 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:45.542300 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/30e800c7-b85f-4404-be63-e9ad90b8dee1-webhook-cert\") pod \"maas-controller-ddf7649c7-jxwp6\" (UID: \"30e800c7-b85f-4404-be63-e9ad90b8dee1\") " pod="opendatahub/maas-controller-ddf7649c7-jxwp6" Jun 15 02:40:45.545779 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:45.545758 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58978d5c5f-dlk6q" Jun 15 02:40:45.547815 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:45.547797 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrhzc\" (UniqueName: \"kubernetes.io/projected/30e800c7-b85f-4404-be63-e9ad90b8dee1-kube-api-access-mrhzc\") pod \"maas-controller-ddf7649c7-jxwp6\" (UID: \"30e800c7-b85f-4404-be63-e9ad90b8dee1\") " pod="opendatahub/maas-controller-ddf7649c7-jxwp6" Jun 15 02:40:45.679024 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:45.678996 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-58978d5c5f-dlk6q"] Jun 15 02:40:45.680067 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:40:45.680049 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06da59a0_e47e_42bd_bafb_f3e51de30541.slice/crio-754d2c4732108bd0dd0951d1e6360ce56916de4d96c5066ba459445f7137e17c WatchSource:0}: Error finding container 754d2c4732108bd0dd0951d1e6360ce56916de4d96c5066ba459445f7137e17c: Status 404 returned error can't find the container with id 754d2c4732108bd0dd0951d1e6360ce56916de4d96c5066ba459445f7137e17c Jun 15 02:40:45.715413 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:45.715386 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-ddf7649c7-jxwp6" Jun 15 02:40:45.740129 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:45.740102 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58978d5c5f-dlk6q" event={"ID":"06da59a0-e47e-42bd-bafb-f3e51de30541","Type":"ContainerStarted","Data":"754d2c4732108bd0dd0951d1e6360ce56916de4d96c5066ba459445f7137e17c"} Jun 15 02:40:45.845042 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:45.845015 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-ddf7649c7-jxwp6"] Jun 15 02:40:45.846376 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:40:45.846350 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30e800c7_b85f_4404_be63_e9ad90b8dee1.slice/crio-a2ad9b8d88fd3a55bd96386b8df14a78942424eba74808c387e1ea3561af8ccc WatchSource:0}: Error finding container a2ad9b8d88fd3a55bd96386b8df14a78942424eba74808c387e1ea3561af8ccc: Status 404 returned error can't find the container with id a2ad9b8d88fd3a55bd96386b8df14a78942424eba74808c387e1ea3561af8ccc Jun 15 02:40:46.749118 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:46.749076 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-ddf7649c7-jxwp6" event={"ID":"30e800c7-b85f-4404-be63-e9ad90b8dee1","Type":"ContainerStarted","Data":"a2ad9b8d88fd3a55bd96386b8df14a78942424eba74808c387e1ea3561af8ccc"} Jun 15 02:40:47.757484 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:47.757393 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58978d5c5f-dlk6q" event={"ID":"06da59a0-e47e-42bd-bafb-f3e51de30541","Type":"ContainerStarted","Data":"064e148dfbbc6831b72ed96f160b452cfec588b7ec2ea4cbd12d16150abdeab4"} Jun 15 02:40:47.757484 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:47.757453 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-58978d5c5f-dlk6q" Jun 15 02:40:47.759026 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:47.758999 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-ddf7649c7-jxwp6" event={"ID":"30e800c7-b85f-4404-be63-e9ad90b8dee1","Type":"ContainerStarted","Data":"409de71dd6dc305b25d4d5dadf6160ac76275c8b56869259f21112af00ad8edc"} Jun 15 02:40:47.759157 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:47.759091 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-ddf7649c7-jxwp6" Jun 15 02:40:47.789917 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:47.789871 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-58978d5c5f-dlk6q" podStartSLOduration=0.967906979 podStartE2EDuration="2.789858468s" totalImagesPullingTime="1.821951489s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 02:40:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 02:40:47.787568859 +0000 UTC m=+740.655353607" watchObservedRunningTime="2026-06-15 02:40:47.789858468 +0000 UTC m=+740.657643213" Jun 15 02:40:47.810084 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:47.810032 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-ddf7649c7-jxwp6" podStartSLOduration=1.157816783 podStartE2EDuration="2.810014348s" totalImagesPullingTime="1.652197565s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 02:40:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 02:40:47.808983743 +0000 UTC m=+740.676768488" watchObservedRunningTime="2026-06-15 02:40:47.810014348 +0000 UTC m=+740.677799095" Jun 15 02:40:58.770052 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:58.770023 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-ddf7649c7-jxwp6" Jun 15 02:40:58.770546 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:58.770260 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-58978d5c5f-dlk6q" Jun 15 02:40:58.832894 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:58.832859 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-58978d5c5f-dlk6q"] Jun 15 02:40:58.833352 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:58.833080 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-58978d5c5f-dlk6q" podUID="06da59a0-e47e-42bd-bafb-f3e51de30541" containerName="manager" containerID="cri-o://064e148dfbbc6831b72ed96f160b452cfec588b7ec2ea4cbd12d16150abdeab4" gracePeriod=30 Jun 15 02:40:59.079533 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:59.079508 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58978d5c5f-dlk6q" Jun 15 02:40:59.143652 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:59.143610 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lzf77\" (UniqueName: \"kubernetes.io/projected/06da59a0-e47e-42bd-bafb-f3e51de30541-kube-api-access-lzf77\") pod \"06da59a0-e47e-42bd-bafb-f3e51de30541\" (UID: \"06da59a0-e47e-42bd-bafb-f3e51de30541\") " Jun 15 02:40:59.143795 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:59.143778 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/06da59a0-e47e-42bd-bafb-f3e51de30541-webhook-cert\") pod \"06da59a0-e47e-42bd-bafb-f3e51de30541\" (UID: \"06da59a0-e47e-42bd-bafb-f3e51de30541\") " Jun 15 02:40:59.145755 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:59.145727 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06da59a0-e47e-42bd-bafb-f3e51de30541-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "06da59a0-e47e-42bd-bafb-f3e51de30541" (UID: "06da59a0-e47e-42bd-bafb-f3e51de30541"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 15 02:40:59.145861 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:59.145765 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06da59a0-e47e-42bd-bafb-f3e51de30541-kube-api-access-lzf77" (OuterVolumeSpecName: "kube-api-access-lzf77") pod "06da59a0-e47e-42bd-bafb-f3e51de30541" (UID: "06da59a0-e47e-42bd-bafb-f3e51de30541"). InnerVolumeSpecName "kube-api-access-lzf77". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 02:40:59.244940 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:59.244916 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lzf77\" (UniqueName: \"kubernetes.io/projected/06da59a0-e47e-42bd-bafb-f3e51de30541-kube-api-access-lzf77\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:40:59.244940 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:59.244937 2579 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/06da59a0-e47e-42bd-bafb-f3e51de30541-webhook-cert\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:40:59.825775 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:59.825689 2579 generic.go:356] "Generic (PLEG): container finished" podID="06da59a0-e47e-42bd-bafb-f3e51de30541" containerID="064e148dfbbc6831b72ed96f160b452cfec588b7ec2ea4cbd12d16150abdeab4" exitCode=0 Jun 15 02:40:59.825775 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:59.825732 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58978d5c5f-dlk6q" Jun 15 02:40:59.825775 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:59.825755 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58978d5c5f-dlk6q" event={"ID":"06da59a0-e47e-42bd-bafb-f3e51de30541","Type":"ContainerDied","Data":"064e148dfbbc6831b72ed96f160b452cfec588b7ec2ea4cbd12d16150abdeab4"} Jun 15 02:40:59.826267 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:59.825806 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58978d5c5f-dlk6q" event={"ID":"06da59a0-e47e-42bd-bafb-f3e51de30541","Type":"ContainerDied","Data":"754d2c4732108bd0dd0951d1e6360ce56916de4d96c5066ba459445f7137e17c"} Jun 15 02:40:59.826267 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:59.825837 2579 scope.go:117] "RemoveContainer" containerID="064e148dfbbc6831b72ed96f160b452cfec588b7ec2ea4cbd12d16150abdeab4" Jun 15 02:40:59.834980 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:59.834758 2579 scope.go:117] "RemoveContainer" containerID="064e148dfbbc6831b72ed96f160b452cfec588b7ec2ea4cbd12d16150abdeab4" Jun 15 02:40:59.835033 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:40:59.835008 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"064e148dfbbc6831b72ed96f160b452cfec588b7ec2ea4cbd12d16150abdeab4\": container with ID starting with 064e148dfbbc6831b72ed96f160b452cfec588b7ec2ea4cbd12d16150abdeab4 not found: ID does not exist" containerID="064e148dfbbc6831b72ed96f160b452cfec588b7ec2ea4cbd12d16150abdeab4" Jun 15 02:40:59.835071 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:59.835033 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"064e148dfbbc6831b72ed96f160b452cfec588b7ec2ea4cbd12d16150abdeab4"} err="failed to get container status \"064e148dfbbc6831b72ed96f160b452cfec588b7ec2ea4cbd12d16150abdeab4\": rpc error: code = NotFound desc = could not find container \"064e148dfbbc6831b72ed96f160b452cfec588b7ec2ea4cbd12d16150abdeab4\": container with ID starting with 064e148dfbbc6831b72ed96f160b452cfec588b7ec2ea4cbd12d16150abdeab4 not found: ID does not exist" Jun 15 02:40:59.841743 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:59.841722 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-58978d5c5f-dlk6q"] Jun 15 02:40:59.849832 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:40:59.849802 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-58978d5c5f-dlk6q"] Jun 15 02:41:01.492995 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:41:01.492963 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06da59a0-e47e-42bd-bafb-f3e51de30541" path="/var/lib/kubelet/pods/06da59a0-e47e-42bd-bafb-f3e51de30541/volumes" Jun 15 02:41:18.833077 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:41:18.833046 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-579785bc64-rngxp"] Jun 15 02:41:18.833510 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:41:18.833370 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06da59a0-e47e-42bd-bafb-f3e51de30541" containerName="manager" Jun 15 02:41:18.833510 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:41:18.833382 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="06da59a0-e47e-42bd-bafb-f3e51de30541" containerName="manager" Jun 15 02:41:18.833510 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:41:18.833444 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="06da59a0-e47e-42bd-bafb-f3e51de30541" containerName="manager" Jun 15 02:41:18.836378 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:41:18.836360 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-579785bc64-rngxp" Jun 15 02:41:18.838771 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:41:18.838751 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jun 15 02:41:18.838980 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:41:18.838966 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-vp6rr\"" Jun 15 02:41:18.844987 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:41:18.844965 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-579785bc64-rngxp"] Jun 15 02:41:19.008976 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:41:19.008939 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnbns\" (UniqueName: \"kubernetes.io/projected/007497bc-5b0b-4320-bda5-4e08f8228820-kube-api-access-gnbns\") pod \"payload-pre-processing-579785bc64-rngxp\" (UID: \"007497bc-5b0b-4320-bda5-4e08f8228820\") " pod="openshift-ingress/payload-pre-processing-579785bc64-rngxp" Jun 15 02:41:19.110352 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:41:19.110254 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnbns\" (UniqueName: \"kubernetes.io/projected/007497bc-5b0b-4320-bda5-4e08f8228820-kube-api-access-gnbns\") pod \"payload-pre-processing-579785bc64-rngxp\" (UID: \"007497bc-5b0b-4320-bda5-4e08f8228820\") " pod="openshift-ingress/payload-pre-processing-579785bc64-rngxp" Jun 15 02:41:19.118019 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:41:19.117987 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnbns\" (UniqueName: \"kubernetes.io/projected/007497bc-5b0b-4320-bda5-4e08f8228820-kube-api-access-gnbns\") pod \"payload-pre-processing-579785bc64-rngxp\" (UID: \"007497bc-5b0b-4320-bda5-4e08f8228820\") " pod="openshift-ingress/payload-pre-processing-579785bc64-rngxp" Jun 15 02:41:19.147481 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:41:19.147455 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-579785bc64-rngxp" Jun 15 02:41:19.278081 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:41:19.278051 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-579785bc64-rngxp"] Jun 15 02:41:19.278585 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:41:19.278558 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod007497bc_5b0b_4320_bda5_4e08f8228820.slice/crio-bcc4f0b3856f33fe8d22c8bd2b160e1b9c0dc070ee5007368d0940b2c2255b89 WatchSource:0}: Error finding container bcc4f0b3856f33fe8d22c8bd2b160e1b9c0dc070ee5007368d0940b2c2255b89: Status 404 returned error can't find the container with id bcc4f0b3856f33fe8d22c8bd2b160e1b9c0dc070ee5007368d0940b2c2255b89 Jun 15 02:41:19.940771 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:41:19.940725 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-579785bc64-rngxp" event={"ID":"007497bc-5b0b-4320-bda5-4e08f8228820","Type":"ContainerStarted","Data":"bcc4f0b3856f33fe8d22c8bd2b160e1b9c0dc070ee5007368d0940b2c2255b89"} Jun 15 02:41:21.958808 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:41:21.958772 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-579785bc64-rngxp" event={"ID":"007497bc-5b0b-4320-bda5-4e08f8228820","Type":"ContainerStarted","Data":"057e0fc6b397fc9ecd5052600afef24296a064f5d18be0946dded38e06725e44"} Jun 15 02:41:21.959273 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:41:21.958912 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-579785bc64-rngxp" Jun 15 02:41:21.976346 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:41:21.976305 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-579785bc64-rngxp" podStartSLOduration=1.8257040839999998 podStartE2EDuration="3.976292952s" totalImagesPullingTime="2.150588868s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 02:41:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 02:41:21.973471942 +0000 UTC m=+774.841256689" watchObservedRunningTime="2026-06-15 02:41:21.976292952 +0000 UTC m=+774.844077697" Jun 15 02:41:32.966203 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:41:32.966169 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-579785bc64-rngxp" Jun 15 02:42:07.711071 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:07.711042 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d8fcddbc5-4v9h5"] Jun 15 02:42:07.714873 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:07.714850 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d8fcddbc5-4v9h5" Jun 15 02:42:07.717075 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:07.717053 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"authorino-oidc-ca\"" Jun 15 02:42:07.728224 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:07.728153 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d8fcddbc5-4v9h5"] Jun 15 02:42:07.778720 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:07.778696 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/20d349ab-b828-46f7-bf08-2152b6dfa9c1-oidc-ca\") pod \"authorino-6d8fcddbc5-4v9h5\" (UID: \"20d349ab-b828-46f7-bf08-2152b6dfa9c1\") " pod="kuadrant-system/authorino-6d8fcddbc5-4v9h5" Jun 15 02:42:07.778815 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:07.778725 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/20d349ab-b828-46f7-bf08-2152b6dfa9c1-tls-cert\") pod \"authorino-6d8fcddbc5-4v9h5\" (UID: \"20d349ab-b828-46f7-bf08-2152b6dfa9c1\") " pod="kuadrant-system/authorino-6d8fcddbc5-4v9h5" Jun 15 02:42:07.778815 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:07.778756 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skqxg\" (UniqueName: \"kubernetes.io/projected/20d349ab-b828-46f7-bf08-2152b6dfa9c1-kube-api-access-skqxg\") pod \"authorino-6d8fcddbc5-4v9h5\" (UID: \"20d349ab-b828-46f7-bf08-2152b6dfa9c1\") " pod="kuadrant-system/authorino-6d8fcddbc5-4v9h5" Jun 15 02:42:07.879437 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:07.879412 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/20d349ab-b828-46f7-bf08-2152b6dfa9c1-tls-cert\") pod \"authorino-6d8fcddbc5-4v9h5\" (UID: \"20d349ab-b828-46f7-bf08-2152b6dfa9c1\") " pod="kuadrant-system/authorino-6d8fcddbc5-4v9h5" Jun 15 02:42:07.879532 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:07.879450 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skqxg\" (UniqueName: \"kubernetes.io/projected/20d349ab-b828-46f7-bf08-2152b6dfa9c1-kube-api-access-skqxg\") pod \"authorino-6d8fcddbc5-4v9h5\" (UID: \"20d349ab-b828-46f7-bf08-2152b6dfa9c1\") " pod="kuadrant-system/authorino-6d8fcddbc5-4v9h5" Jun 15 02:42:07.879532 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:07.879518 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/20d349ab-b828-46f7-bf08-2152b6dfa9c1-oidc-ca\") pod \"authorino-6d8fcddbc5-4v9h5\" (UID: \"20d349ab-b828-46f7-bf08-2152b6dfa9c1\") " pod="kuadrant-system/authorino-6d8fcddbc5-4v9h5" Jun 15 02:42:07.880135 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:07.880117 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/20d349ab-b828-46f7-bf08-2152b6dfa9c1-oidc-ca\") pod \"authorino-6d8fcddbc5-4v9h5\" (UID: \"20d349ab-b828-46f7-bf08-2152b6dfa9c1\") " pod="kuadrant-system/authorino-6d8fcddbc5-4v9h5" Jun 15 02:42:07.881781 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:07.881763 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/20d349ab-b828-46f7-bf08-2152b6dfa9c1-tls-cert\") pod \"authorino-6d8fcddbc5-4v9h5\" (UID: \"20d349ab-b828-46f7-bf08-2152b6dfa9c1\") " pod="kuadrant-system/authorino-6d8fcddbc5-4v9h5" Jun 15 02:42:07.886859 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:07.886840 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-skqxg\" (UniqueName: \"kubernetes.io/projected/20d349ab-b828-46f7-bf08-2152b6dfa9c1-kube-api-access-skqxg\") pod \"authorino-6d8fcddbc5-4v9h5\" (UID: \"20d349ab-b828-46f7-bf08-2152b6dfa9c1\") " pod="kuadrant-system/authorino-6d8fcddbc5-4v9h5" Jun 15 02:42:08.031157 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:08.031131 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d8fcddbc5-4v9h5" Jun 15 02:42:08.162337 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:08.162307 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d8fcddbc5-4v9h5"] Jun 15 02:42:08.163905 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:42:08.163873 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod20d349ab_b828_46f7_bf08_2152b6dfa9c1.slice/crio-e3453b0f26f7eb7993f4407171aba56e0a2b309bd20ce48440e002efd1637988 WatchSource:0}: Error finding container e3453b0f26f7eb7993f4407171aba56e0a2b309bd20ce48440e002efd1637988: Status 404 returned error can't find the container with id e3453b0f26f7eb7993f4407171aba56e0a2b309bd20ce48440e002efd1637988 Jun 15 02:42:08.200430 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:08.200402 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6d8fcddbc5-4v9h5" event={"ID":"20d349ab-b828-46f7-bf08-2152b6dfa9c1","Type":"ContainerStarted","Data":"e3453b0f26f7eb7993f4407171aba56e0a2b309bd20ce48440e002efd1637988"} Jun 15 02:42:09.208003 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:09.207969 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6d8fcddbc5-4v9h5" event={"ID":"20d349ab-b828-46f7-bf08-2152b6dfa9c1","Type":"ContainerStarted","Data":"fb704c36e27aba2fb5cedad739d2fd141a106eec8da48d53e8134e48707629b1"} Jun 15 02:42:09.225482 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:09.225438 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6d8fcddbc5-4v9h5" podStartSLOduration=1.808546367 podStartE2EDuration="2.225424671s" totalImagesPullingTime="416.878304ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 02:42:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 02:42:09.223968701 +0000 UTC m=+822.091753448" watchObservedRunningTime="2026-06-15 02:42:09.225424671 +0000 UTC m=+822.093209416" Jun 15 02:42:09.250332 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:09.250304 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-778cd84b87-7nhtj"] Jun 15 02:42:09.250487 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:09.250466 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-778cd84b87-7nhtj" podUID="68fbb99e-9bbb-457d-b3e9-4c584ba5a07f" containerName="authorino" containerID="cri-o://5eb75b7e95d9a2bf01303bea56242dec0bf1ea6715953d8d87ac26ad223cfcfd" gracePeriod=30 Jun 15 02:42:09.498336 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:09.498312 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-778cd84b87-7nhtj" Jun 15 02:42:09.593394 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:09.593367 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8qhqb\" (UniqueName: \"kubernetes.io/projected/68fbb99e-9bbb-457d-b3e9-4c584ba5a07f-kube-api-access-8qhqb\") pod \"68fbb99e-9bbb-457d-b3e9-4c584ba5a07f\" (UID: \"68fbb99e-9bbb-457d-b3e9-4c584ba5a07f\") " Jun 15 02:42:09.593542 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:09.593423 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/68fbb99e-9bbb-457d-b3e9-4c584ba5a07f-tls-cert\") pod \"68fbb99e-9bbb-457d-b3e9-4c584ba5a07f\" (UID: \"68fbb99e-9bbb-457d-b3e9-4c584ba5a07f\") " Jun 15 02:42:09.595411 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:09.595382 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68fbb99e-9bbb-457d-b3e9-4c584ba5a07f-kube-api-access-8qhqb" (OuterVolumeSpecName: "kube-api-access-8qhqb") pod "68fbb99e-9bbb-457d-b3e9-4c584ba5a07f" (UID: "68fbb99e-9bbb-457d-b3e9-4c584ba5a07f"). InnerVolumeSpecName "kube-api-access-8qhqb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 02:42:09.603450 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:09.603426 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68fbb99e-9bbb-457d-b3e9-4c584ba5a07f-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "68fbb99e-9bbb-457d-b3e9-4c584ba5a07f" (UID: "68fbb99e-9bbb-457d-b3e9-4c584ba5a07f"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 15 02:42:09.694837 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:09.694812 2579 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/68fbb99e-9bbb-457d-b3e9-4c584ba5a07f-tls-cert\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:42:09.694837 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:09.694836 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8qhqb\" (UniqueName: \"kubernetes.io/projected/68fbb99e-9bbb-457d-b3e9-4c584ba5a07f-kube-api-access-8qhqb\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:42:10.215188 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:10.215105 2579 generic.go:356] "Generic (PLEG): container finished" podID="68fbb99e-9bbb-457d-b3e9-4c584ba5a07f" containerID="5eb75b7e95d9a2bf01303bea56242dec0bf1ea6715953d8d87ac26ad223cfcfd" exitCode=0 Jun 15 02:42:10.215188 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:10.215152 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-778cd84b87-7nhtj" Jun 15 02:42:10.215670 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:10.215190 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-778cd84b87-7nhtj" event={"ID":"68fbb99e-9bbb-457d-b3e9-4c584ba5a07f","Type":"ContainerDied","Data":"5eb75b7e95d9a2bf01303bea56242dec0bf1ea6715953d8d87ac26ad223cfcfd"} Jun 15 02:42:10.215670 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:10.215234 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-778cd84b87-7nhtj" event={"ID":"68fbb99e-9bbb-457d-b3e9-4c584ba5a07f","Type":"ContainerDied","Data":"de058e5cf85d4ec84018668c4e135fabe02022f037a93ea499ad7e0577507329"} Jun 15 02:42:10.215670 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:10.215252 2579 scope.go:117] "RemoveContainer" containerID="5eb75b7e95d9a2bf01303bea56242dec0bf1ea6715953d8d87ac26ad223cfcfd" Jun 15 02:42:10.224853 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:10.224837 2579 scope.go:117] "RemoveContainer" containerID="5eb75b7e95d9a2bf01303bea56242dec0bf1ea6715953d8d87ac26ad223cfcfd" Jun 15 02:42:10.225084 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:42:10.225066 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5eb75b7e95d9a2bf01303bea56242dec0bf1ea6715953d8d87ac26ad223cfcfd\": container with ID starting with 5eb75b7e95d9a2bf01303bea56242dec0bf1ea6715953d8d87ac26ad223cfcfd not found: ID does not exist" containerID="5eb75b7e95d9a2bf01303bea56242dec0bf1ea6715953d8d87ac26ad223cfcfd" Jun 15 02:42:10.225132 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:10.225093 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5eb75b7e95d9a2bf01303bea56242dec0bf1ea6715953d8d87ac26ad223cfcfd"} err="failed to get container status \"5eb75b7e95d9a2bf01303bea56242dec0bf1ea6715953d8d87ac26ad223cfcfd\": rpc error: code = NotFound desc = could not find container \"5eb75b7e95d9a2bf01303bea56242dec0bf1ea6715953d8d87ac26ad223cfcfd\": container with ID starting with 5eb75b7e95d9a2bf01303bea56242dec0bf1ea6715953d8d87ac26ad223cfcfd not found: ID does not exist" Jun 15 02:42:10.235384 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:10.235363 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-778cd84b87-7nhtj"] Jun 15 02:42:10.241318 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:10.241300 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-778cd84b87-7nhtj"] Jun 15 02:42:11.497315 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:11.497280 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68fbb99e-9bbb-457d-b3e9-4c584ba5a07f" path="/var/lib/kubelet/pods/68fbb99e-9bbb-457d-b3e9-4c584ba5a07f/volumes" Jun 15 02:42:19.794717 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:19.794681 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-wbmcv"] Jun 15 02:42:19.795075 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:19.795042 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68fbb99e-9bbb-457d-b3e9-4c584ba5a07f" containerName="authorino" Jun 15 02:42:19.795075 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:19.795053 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="68fbb99e-9bbb-457d-b3e9-4c584ba5a07f" containerName="authorino" Jun 15 02:42:19.795164 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:19.795109 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="68fbb99e-9bbb-457d-b3e9-4c584ba5a07f" containerName="authorino" Jun 15 02:42:19.798383 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:19.798366 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-wbmcv" Jun 15 02:42:19.801556 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:19.801531 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jun 15 02:42:19.801710 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:19.801572 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jun 15 02:42:19.801710 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:19.801588 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-xpbk6\"" Jun 15 02:42:19.801710 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:19.801654 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jun 15 02:42:19.809472 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:19.809448 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-wbmcv"] Jun 15 02:42:19.877134 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:19.877108 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1be8db44-3fb9-4cd4-b3af-ffaec8a44636-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-wbmcv\" (UID: \"1be8db44-3fb9-4cd4-b3af-ffaec8a44636\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-wbmcv" Jun 15 02:42:19.877273 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:19.877140 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1be8db44-3fb9-4cd4-b3af-ffaec8a44636-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-wbmcv\" (UID: \"1be8db44-3fb9-4cd4-b3af-ffaec8a44636\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-wbmcv" Jun 15 02:42:19.877273 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:19.877159 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtv5p\" (UniqueName: \"kubernetes.io/projected/1be8db44-3fb9-4cd4-b3af-ffaec8a44636-kube-api-access-vtv5p\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-wbmcv\" (UID: \"1be8db44-3fb9-4cd4-b3af-ffaec8a44636\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-wbmcv" Jun 15 02:42:19.877273 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:19.877237 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1be8db44-3fb9-4cd4-b3af-ffaec8a44636-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-wbmcv\" (UID: \"1be8db44-3fb9-4cd4-b3af-ffaec8a44636\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-wbmcv" Jun 15 02:42:19.877410 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:19.877305 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1be8db44-3fb9-4cd4-b3af-ffaec8a44636-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-wbmcv\" (UID: \"1be8db44-3fb9-4cd4-b3af-ffaec8a44636\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-wbmcv" Jun 15 02:42:19.877410 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:19.877329 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1be8db44-3fb9-4cd4-b3af-ffaec8a44636-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-wbmcv\" (UID: \"1be8db44-3fb9-4cd4-b3af-ffaec8a44636\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-wbmcv" Jun 15 02:42:19.978342 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:19.978315 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1be8db44-3fb9-4cd4-b3af-ffaec8a44636-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-wbmcv\" (UID: \"1be8db44-3fb9-4cd4-b3af-ffaec8a44636\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-wbmcv" Jun 15 02:42:19.978486 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:19.978346 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1be8db44-3fb9-4cd4-b3af-ffaec8a44636-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-wbmcv\" (UID: \"1be8db44-3fb9-4cd4-b3af-ffaec8a44636\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-wbmcv" Jun 15 02:42:19.978486 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:19.978369 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1be8db44-3fb9-4cd4-b3af-ffaec8a44636-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-wbmcv\" (UID: \"1be8db44-3fb9-4cd4-b3af-ffaec8a44636\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-wbmcv" Jun 15 02:42:19.978486 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:19.978385 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1be8db44-3fb9-4cd4-b3af-ffaec8a44636-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-wbmcv\" (UID: \"1be8db44-3fb9-4cd4-b3af-ffaec8a44636\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-wbmcv" Jun 15 02:42:19.978486 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:19.978403 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtv5p\" (UniqueName: \"kubernetes.io/projected/1be8db44-3fb9-4cd4-b3af-ffaec8a44636-kube-api-access-vtv5p\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-wbmcv\" (UID: \"1be8db44-3fb9-4cd4-b3af-ffaec8a44636\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-wbmcv" Jun 15 02:42:19.978486 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:19.978435 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1be8db44-3fb9-4cd4-b3af-ffaec8a44636-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-wbmcv\" (UID: \"1be8db44-3fb9-4cd4-b3af-ffaec8a44636\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-wbmcv" Jun 15 02:42:19.978796 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:19.978770 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1be8db44-3fb9-4cd4-b3af-ffaec8a44636-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-wbmcv\" (UID: \"1be8db44-3fb9-4cd4-b3af-ffaec8a44636\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-wbmcv" Jun 15 02:42:19.978908 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:19.978840 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1be8db44-3fb9-4cd4-b3af-ffaec8a44636-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-wbmcv\" (UID: \"1be8db44-3fb9-4cd4-b3af-ffaec8a44636\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-wbmcv" Jun 15 02:42:19.978908 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:19.978901 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1be8db44-3fb9-4cd4-b3af-ffaec8a44636-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-wbmcv\" (UID: \"1be8db44-3fb9-4cd4-b3af-ffaec8a44636\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-wbmcv" Jun 15 02:42:19.980707 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:19.980687 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1be8db44-3fb9-4cd4-b3af-ffaec8a44636-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-wbmcv\" (UID: \"1be8db44-3fb9-4cd4-b3af-ffaec8a44636\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-wbmcv" Jun 15 02:42:19.980872 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:19.980856 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1be8db44-3fb9-4cd4-b3af-ffaec8a44636-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-wbmcv\" (UID: \"1be8db44-3fb9-4cd4-b3af-ffaec8a44636\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-wbmcv" Jun 15 02:42:19.986816 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:19.986794 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtv5p\" (UniqueName: \"kubernetes.io/projected/1be8db44-3fb9-4cd4-b3af-ffaec8a44636-kube-api-access-vtv5p\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-wbmcv\" (UID: \"1be8db44-3fb9-4cd4-b3af-ffaec8a44636\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-wbmcv" Jun 15 02:42:20.111457 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:20.111389 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-wbmcv" Jun 15 02:42:20.254994 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:20.254966 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-wbmcv"] Jun 15 02:42:20.256354 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:42:20.256282 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1be8db44_3fb9_4cd4_b3af_ffaec8a44636.slice/crio-3c0b0d490d15d759fc2638f79df11a6bb7949b940637eb76523fc8bd410a28cb WatchSource:0}: Error finding container 3c0b0d490d15d759fc2638f79df11a6bb7949b940637eb76523fc8bd410a28cb: Status 404 returned error can't find the container with id 3c0b0d490d15d759fc2638f79df11a6bb7949b940637eb76523fc8bd410a28cb Jun 15 02:42:20.273797 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:20.273772 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-wbmcv" event={"ID":"1be8db44-3fb9-4cd4-b3af-ffaec8a44636","Type":"ContainerStarted","Data":"3c0b0d490d15d759fc2638f79df11a6bb7949b940637eb76523fc8bd410a28cb"} Jun 15 02:42:24.298730 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:24.298690 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-wbmcv" event={"ID":"1be8db44-3fb9-4cd4-b3af-ffaec8a44636","Type":"ContainerStarted","Data":"ea7c09deda5da7f0fe3ae39bfe9bf3216cdd2676e8697959b3d8653427bdff48"} Jun 15 02:42:26.312468 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:26.312434 2579 generic.go:356] "Generic (PLEG): container finished" podID="1be8db44-3fb9-4cd4-b3af-ffaec8a44636" containerID="ea7c09deda5da7f0fe3ae39bfe9bf3216cdd2676e8697959b3d8653427bdff48" exitCode=0 Jun 15 02:42:26.312847 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:26.312506 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-wbmcv" event={"ID":"1be8db44-3fb9-4cd4-b3af-ffaec8a44636","Type":"ContainerDied","Data":"ea7c09deda5da7f0fe3ae39bfe9bf3216cdd2676e8697959b3d8653427bdff48"} Jun 15 02:42:28.327284 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:28.327248 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-wbmcv" event={"ID":"1be8db44-3fb9-4cd4-b3af-ffaec8a44636","Type":"ContainerStarted","Data":"441ade875d7ab47937622dfa714769487dc270eca356711923444ffa3c4f5e86"} Jun 15 02:42:28.346599 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:28.346552 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-wbmcv" podStartSLOduration=3.06061513 podStartE2EDuration="9.346538116s" totalImagesPullingTime="4.986116971s" totalInitContainerRuntime="1.299806015s" isStatefulPod=false podCreationTimestamp="2026-06-15 02:42:19 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 02:42:28.343682677 +0000 UTC m=+841.211467419" watchObservedRunningTime="2026-06-15 02:42:28.346538116 +0000 UTC m=+841.214322839" Jun 15 02:42:30.112423 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:30.112386 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-wbmcv" Jun 15 02:42:30.112837 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:30.112462 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-wbmcv" Jun 15 02:42:30.125146 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:30.125120 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-wbmcv" Jun 15 02:42:30.349756 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:30.349730 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-wbmcv" Jun 15 02:42:41.671976 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:41.671942 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-k8kjc"] Jun 15 02:42:41.726739 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:41.726706 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-k8kjc"] Jun 15 02:42:41.726898 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:41.726824 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-k8kjc" Jun 15 02:42:41.729007 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:41.728979 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jun 15 02:42:41.859466 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:41.859440 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0bb11ae-dbd7-4174-84d6-a387a5842a1e-dshm\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-k8kjc\" (UID: \"e0bb11ae-dbd7-4174-84d6-a387a5842a1e\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-k8kjc" Jun 15 02:42:41.859594 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:41.859470 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mg6q\" (UniqueName: \"kubernetes.io/projected/e0bb11ae-dbd7-4174-84d6-a387a5842a1e-kube-api-access-4mg6q\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-k8kjc\" (UID: \"e0bb11ae-dbd7-4174-84d6-a387a5842a1e\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-k8kjc" Jun 15 02:42:41.859594 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:41.859492 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0bb11ae-dbd7-4174-84d6-a387a5842a1e-home\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-k8kjc\" (UID: \"e0bb11ae-dbd7-4174-84d6-a387a5842a1e\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-k8kjc" Jun 15 02:42:41.859594 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:41.859576 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0bb11ae-dbd7-4174-84d6-a387a5842a1e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-k8kjc\" (UID: \"e0bb11ae-dbd7-4174-84d6-a387a5842a1e\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-k8kjc" Jun 15 02:42:41.859728 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:41.859654 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0bb11ae-dbd7-4174-84d6-a387a5842a1e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-k8kjc\" (UID: \"e0bb11ae-dbd7-4174-84d6-a387a5842a1e\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-k8kjc" Jun 15 02:42:41.859728 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:41.859693 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e0bb11ae-dbd7-4174-84d6-a387a5842a1e-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-k8kjc\" (UID: \"e0bb11ae-dbd7-4174-84d6-a387a5842a1e\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-k8kjc" Jun 15 02:42:41.960802 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:41.960739 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0bb11ae-dbd7-4174-84d6-a387a5842a1e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-k8kjc\" (UID: \"e0bb11ae-dbd7-4174-84d6-a387a5842a1e\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-k8kjc" Jun 15 02:42:41.960802 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:41.960778 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e0bb11ae-dbd7-4174-84d6-a387a5842a1e-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-k8kjc\" (UID: \"e0bb11ae-dbd7-4174-84d6-a387a5842a1e\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-k8kjc" Jun 15 02:42:41.960802 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:41.960799 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0bb11ae-dbd7-4174-84d6-a387a5842a1e-dshm\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-k8kjc\" (UID: \"e0bb11ae-dbd7-4174-84d6-a387a5842a1e\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-k8kjc" Jun 15 02:42:41.961012 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:41.960817 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mg6q\" (UniqueName: \"kubernetes.io/projected/e0bb11ae-dbd7-4174-84d6-a387a5842a1e-kube-api-access-4mg6q\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-k8kjc\" (UID: \"e0bb11ae-dbd7-4174-84d6-a387a5842a1e\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-k8kjc" Jun 15 02:42:41.961012 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:41.960844 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0bb11ae-dbd7-4174-84d6-a387a5842a1e-home\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-k8kjc\" (UID: \"e0bb11ae-dbd7-4174-84d6-a387a5842a1e\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-k8kjc" Jun 15 02:42:41.961012 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:41.960891 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0bb11ae-dbd7-4174-84d6-a387a5842a1e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-k8kjc\" (UID: \"e0bb11ae-dbd7-4174-84d6-a387a5842a1e\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-k8kjc" Jun 15 02:42:41.961158 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:41.961113 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0bb11ae-dbd7-4174-84d6-a387a5842a1e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-k8kjc\" (UID: \"e0bb11ae-dbd7-4174-84d6-a387a5842a1e\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-k8kjc" Jun 15 02:42:41.961218 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:41.961165 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e0bb11ae-dbd7-4174-84d6-a387a5842a1e-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-k8kjc\" (UID: \"e0bb11ae-dbd7-4174-84d6-a387a5842a1e\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-k8kjc" Jun 15 02:42:41.961309 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:41.961286 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0bb11ae-dbd7-4174-84d6-a387a5842a1e-home\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-k8kjc\" (UID: \"e0bb11ae-dbd7-4174-84d6-a387a5842a1e\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-k8kjc" Jun 15 02:42:41.963128 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:41.963103 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0bb11ae-dbd7-4174-84d6-a387a5842a1e-dshm\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-k8kjc\" (UID: \"e0bb11ae-dbd7-4174-84d6-a387a5842a1e\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-k8kjc" Jun 15 02:42:41.963360 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:41.963343 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0bb11ae-dbd7-4174-84d6-a387a5842a1e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-k8kjc\" (UID: \"e0bb11ae-dbd7-4174-84d6-a387a5842a1e\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-k8kjc" Jun 15 02:42:41.968504 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:41.968483 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mg6q\" (UniqueName: \"kubernetes.io/projected/e0bb11ae-dbd7-4174-84d6-a387a5842a1e-kube-api-access-4mg6q\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-k8kjc\" (UID: \"e0bb11ae-dbd7-4174-84d6-a387a5842a1e\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-k8kjc" Jun 15 02:42:42.037401 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:42.037377 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-k8kjc" Jun 15 02:42:42.172315 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:42.172292 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-k8kjc"] Jun 15 02:42:42.174472 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:42:42.174440 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0bb11ae_dbd7_4174_84d6_a387a5842a1e.slice/crio-d63b209f90fc074f17ad65b129241e44a4d19e82d1e64b1d95456dff82d7b4b7 WatchSource:0}: Error finding container d63b209f90fc074f17ad65b129241e44a4d19e82d1e64b1d95456dff82d7b4b7: Status 404 returned error can't find the container with id d63b209f90fc074f17ad65b129241e44a4d19e82d1e64b1d95456dff82d7b4b7 Jun 15 02:42:42.406172 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:42.406133 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-k8kjc" event={"ID":"e0bb11ae-dbd7-4174-84d6-a387a5842a1e","Type":"ContainerStarted","Data":"e6c03180774b81c4ae83d2f9ea2fbc236766aa1c6729e6f9d4aa19ded316c795"} Jun 15 02:42:42.406334 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:42.406199 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-k8kjc" event={"ID":"e0bb11ae-dbd7-4174-84d6-a387a5842a1e","Type":"ContainerStarted","Data":"d63b209f90fc074f17ad65b129241e44a4d19e82d1e64b1d95456dff82d7b4b7"} Jun 15 02:42:44.418387 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:44.418356 2579 generic.go:356] "Generic (PLEG): container finished" podID="e0bb11ae-dbd7-4174-84d6-a387a5842a1e" containerID="e6c03180774b81c4ae83d2f9ea2fbc236766aa1c6729e6f9d4aa19ded316c795" exitCode=0 Jun 15 02:42:44.418881 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:44.418428 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-k8kjc" event={"ID":"e0bb11ae-dbd7-4174-84d6-a387a5842a1e","Type":"ContainerDied","Data":"e6c03180774b81c4ae83d2f9ea2fbc236766aa1c6729e6f9d4aa19ded316c795"} Jun 15 02:42:45.425501 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:45.425464 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-k8kjc" event={"ID":"e0bb11ae-dbd7-4174-84d6-a387a5842a1e","Type":"ContainerStarted","Data":"9eff99aaa8574ff125ca30eec3f966b1ccd08d10fe6ea9766bb4be0980017521"} Jun 15 02:42:45.444024 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:45.443976 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-k8kjc" podStartSLOduration=2.915831672 podStartE2EDuration="4.443961727s" totalImagesPullingTime="163.351003ms" totalInitContainerRuntime="1.364779052s" isStatefulPod=false podCreationTimestamp="2026-06-15 02:42:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 02:42:45.442034591 +0000 UTC m=+858.309819355" watchObservedRunningTime="2026-06-15 02:42:45.443961727 +0000 UTC m=+858.311746483" Jun 15 02:42:51.380847 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:51.380815 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cz5wzr"] Jun 15 02:42:51.385872 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:51.385852 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cz5wzr" Jun 15 02:42:51.388528 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:51.388496 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jun 15 02:42:51.397226 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:51.397203 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cz5wzr"] Jun 15 02:42:51.442217 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:51.442186 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69415e80-2a0f-4877-9ca7-ef5e11abf1da-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cz5wzr\" (UID: \"69415e80-2a0f-4877-9ca7-ef5e11abf1da\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cz5wzr" Jun 15 02:42:51.442332 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:51.442223 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69415e80-2a0f-4877-9ca7-ef5e11abf1da-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cz5wzr\" (UID: \"69415e80-2a0f-4877-9ca7-ef5e11abf1da\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cz5wzr" Jun 15 02:42:51.442332 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:51.442240 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/69415e80-2a0f-4877-9ca7-ef5e11abf1da-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cz5wzr\" (UID: \"69415e80-2a0f-4877-9ca7-ef5e11abf1da\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cz5wzr" Jun 15 02:42:51.442332 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:51.442260 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggs4b\" (UniqueName: \"kubernetes.io/projected/69415e80-2a0f-4877-9ca7-ef5e11abf1da-kube-api-access-ggs4b\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cz5wzr\" (UID: \"69415e80-2a0f-4877-9ca7-ef5e11abf1da\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cz5wzr" Jun 15 02:42:51.442500 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:51.442378 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69415e80-2a0f-4877-9ca7-ef5e11abf1da-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cz5wzr\" (UID: \"69415e80-2a0f-4877-9ca7-ef5e11abf1da\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cz5wzr" Jun 15 02:42:51.442500 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:51.442454 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69415e80-2a0f-4877-9ca7-ef5e11abf1da-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cz5wzr\" (UID: \"69415e80-2a0f-4877-9ca7-ef5e11abf1da\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cz5wzr" Jun 15 02:42:51.542849 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:51.542813 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69415e80-2a0f-4877-9ca7-ef5e11abf1da-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cz5wzr\" (UID: \"69415e80-2a0f-4877-9ca7-ef5e11abf1da\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cz5wzr" Jun 15 02:42:51.542983 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:51.542918 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69415e80-2a0f-4877-9ca7-ef5e11abf1da-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cz5wzr\" (UID: \"69415e80-2a0f-4877-9ca7-ef5e11abf1da\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cz5wzr" Jun 15 02:42:51.542983 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:51.542968 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69415e80-2a0f-4877-9ca7-ef5e11abf1da-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cz5wzr\" (UID: \"69415e80-2a0f-4877-9ca7-ef5e11abf1da\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cz5wzr" Jun 15 02:42:51.543142 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:51.543100 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/69415e80-2a0f-4877-9ca7-ef5e11abf1da-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cz5wzr\" (UID: \"69415e80-2a0f-4877-9ca7-ef5e11abf1da\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cz5wzr" Jun 15 02:42:51.543224 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:51.543145 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggs4b\" (UniqueName: \"kubernetes.io/projected/69415e80-2a0f-4877-9ca7-ef5e11abf1da-kube-api-access-ggs4b\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cz5wzr\" (UID: \"69415e80-2a0f-4877-9ca7-ef5e11abf1da\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cz5wzr" Jun 15 02:42:51.543224 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:51.543187 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69415e80-2a0f-4877-9ca7-ef5e11abf1da-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cz5wzr\" (UID: \"69415e80-2a0f-4877-9ca7-ef5e11abf1da\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cz5wzr" Jun 15 02:42:51.543224 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:51.543210 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69415e80-2a0f-4877-9ca7-ef5e11abf1da-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cz5wzr\" (UID: \"69415e80-2a0f-4877-9ca7-ef5e11abf1da\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cz5wzr" Jun 15 02:42:51.543379 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:51.543256 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69415e80-2a0f-4877-9ca7-ef5e11abf1da-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cz5wzr\" (UID: \"69415e80-2a0f-4877-9ca7-ef5e11abf1da\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cz5wzr" Jun 15 02:42:51.543457 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:51.543436 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/69415e80-2a0f-4877-9ca7-ef5e11abf1da-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cz5wzr\" (UID: \"69415e80-2a0f-4877-9ca7-ef5e11abf1da\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cz5wzr" Jun 15 02:42:51.545310 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:51.545292 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69415e80-2a0f-4877-9ca7-ef5e11abf1da-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cz5wzr\" (UID: \"69415e80-2a0f-4877-9ca7-ef5e11abf1da\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cz5wzr" Jun 15 02:42:51.545511 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:51.545492 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69415e80-2a0f-4877-9ca7-ef5e11abf1da-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cz5wzr\" (UID: \"69415e80-2a0f-4877-9ca7-ef5e11abf1da\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cz5wzr" Jun 15 02:42:51.550798 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:51.550775 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggs4b\" (UniqueName: \"kubernetes.io/projected/69415e80-2a0f-4877-9ca7-ef5e11abf1da-kube-api-access-ggs4b\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cz5wzr\" (UID: \"69415e80-2a0f-4877-9ca7-ef5e11abf1da\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cz5wzr" Jun 15 02:42:51.699303 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:51.699234 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cz5wzr" Jun 15 02:42:51.836898 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:51.836874 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cz5wzr"] Jun 15 02:42:51.838917 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:42:51.838885 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69415e80_2a0f_4877_9ca7_ef5e11abf1da.slice/crio-6c2142e181d62ef6bf3b3abd6d64d9290aed9e710e3d0caeda63a9faf14edb22 WatchSource:0}: Error finding container 6c2142e181d62ef6bf3b3abd6d64d9290aed9e710e3d0caeda63a9faf14edb22: Status 404 returned error can't find the container with id 6c2142e181d62ef6bf3b3abd6d64d9290aed9e710e3d0caeda63a9faf14edb22 Jun 15 02:42:52.037927 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:52.037891 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-k8kjc" Jun 15 02:42:52.038108 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:52.037988 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-k8kjc" Jun 15 02:42:52.051712 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:52.051684 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-k8kjc" Jun 15 02:42:52.463278 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:52.463197 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cz5wzr" event={"ID":"69415e80-2a0f-4877-9ca7-ef5e11abf1da","Type":"ContainerStarted","Data":"9a42def4dd7d2542bd01d304d2c8d89810fc0fd1ef5f884f62fdf6eb39db8a67"} Jun 15 02:42:52.463278 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:52.463260 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cz5wzr" event={"ID":"69415e80-2a0f-4877-9ca7-ef5e11abf1da","Type":"ContainerStarted","Data":"6c2142e181d62ef6bf3b3abd6d64d9290aed9e710e3d0caeda63a9faf14edb22"} Jun 15 02:42:52.478241 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:52.478213 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-k8kjc" Jun 15 02:42:53.883144 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:53.883117 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-6f969c879b-bsfmg"] Jun 15 02:42:53.888838 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:53.888820 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-bsfmg" Jun 15 02:42:53.891106 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:53.891076 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jun 15 02:42:53.898618 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:53.898593 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-6f969c879b-bsfmg"] Jun 15 02:42:53.967068 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:53.967041 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a1ef072-2776-45cc-adea-2d9bfeaa6e3c-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-bsfmg\" (UID: \"8a1ef072-2776-45cc-adea-2d9bfeaa6e3c\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-bsfmg" Jun 15 02:42:53.967214 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:53.967081 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2c6m\" (UniqueName: \"kubernetes.io/projected/8a1ef072-2776-45cc-adea-2d9bfeaa6e3c-kube-api-access-w2c6m\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-bsfmg\" (UID: \"8a1ef072-2776-45cc-adea-2d9bfeaa6e3c\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-bsfmg" Jun 15 02:42:53.967214 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:53.967112 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a1ef072-2776-45cc-adea-2d9bfeaa6e3c-model-cache\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-bsfmg\" (UID: \"8a1ef072-2776-45cc-adea-2d9bfeaa6e3c\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-bsfmg" Jun 15 02:42:53.967214 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:53.967130 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8a1ef072-2776-45cc-adea-2d9bfeaa6e3c-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-bsfmg\" (UID: \"8a1ef072-2776-45cc-adea-2d9bfeaa6e3c\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-bsfmg" Jun 15 02:42:53.967390 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:53.967220 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a1ef072-2776-45cc-adea-2d9bfeaa6e3c-dshm\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-bsfmg\" (UID: \"8a1ef072-2776-45cc-adea-2d9bfeaa6e3c\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-bsfmg" Jun 15 02:42:53.967390 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:53.967285 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a1ef072-2776-45cc-adea-2d9bfeaa6e3c-home\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-bsfmg\" (UID: \"8a1ef072-2776-45cc-adea-2d9bfeaa6e3c\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-bsfmg" Jun 15 02:42:54.068191 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:54.068163 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a1ef072-2776-45cc-adea-2d9bfeaa6e3c-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-bsfmg\" (UID: \"8a1ef072-2776-45cc-adea-2d9bfeaa6e3c\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-bsfmg" Jun 15 02:42:54.068308 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:54.068196 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2c6m\" (UniqueName: \"kubernetes.io/projected/8a1ef072-2776-45cc-adea-2d9bfeaa6e3c-kube-api-access-w2c6m\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-bsfmg\" (UID: \"8a1ef072-2776-45cc-adea-2d9bfeaa6e3c\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-bsfmg" Jun 15 02:42:54.068308 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:54.068232 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a1ef072-2776-45cc-adea-2d9bfeaa6e3c-model-cache\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-bsfmg\" (UID: \"8a1ef072-2776-45cc-adea-2d9bfeaa6e3c\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-bsfmg" Jun 15 02:42:54.068308 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:54.068259 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8a1ef072-2776-45cc-adea-2d9bfeaa6e3c-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-bsfmg\" (UID: \"8a1ef072-2776-45cc-adea-2d9bfeaa6e3c\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-bsfmg" Jun 15 02:42:54.068425 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:54.068308 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a1ef072-2776-45cc-adea-2d9bfeaa6e3c-dshm\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-bsfmg\" (UID: \"8a1ef072-2776-45cc-adea-2d9bfeaa6e3c\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-bsfmg" Jun 15 02:42:54.068425 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:54.068355 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a1ef072-2776-45cc-adea-2d9bfeaa6e3c-home\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-bsfmg\" (UID: \"8a1ef072-2776-45cc-adea-2d9bfeaa6e3c\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-bsfmg" Jun 15 02:42:54.068791 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:54.068765 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8a1ef072-2776-45cc-adea-2d9bfeaa6e3c-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-bsfmg\" (UID: \"8a1ef072-2776-45cc-adea-2d9bfeaa6e3c\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-bsfmg" Jun 15 02:42:54.068868 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:54.068796 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a1ef072-2776-45cc-adea-2d9bfeaa6e3c-model-cache\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-bsfmg\" (UID: \"8a1ef072-2776-45cc-adea-2d9bfeaa6e3c\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-bsfmg" Jun 15 02:42:54.068868 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:54.068834 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a1ef072-2776-45cc-adea-2d9bfeaa6e3c-home\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-bsfmg\" (UID: \"8a1ef072-2776-45cc-adea-2d9bfeaa6e3c\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-bsfmg" Jun 15 02:42:54.070575 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:54.070555 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a1ef072-2776-45cc-adea-2d9bfeaa6e3c-dshm\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-bsfmg\" (UID: \"8a1ef072-2776-45cc-adea-2d9bfeaa6e3c\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-bsfmg" Jun 15 02:42:54.070849 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:54.070833 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a1ef072-2776-45cc-adea-2d9bfeaa6e3c-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-bsfmg\" (UID: \"8a1ef072-2776-45cc-adea-2d9bfeaa6e3c\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-bsfmg" Jun 15 02:42:54.075064 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:54.075045 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2c6m\" (UniqueName: \"kubernetes.io/projected/8a1ef072-2776-45cc-adea-2d9bfeaa6e3c-kube-api-access-w2c6m\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-bsfmg\" (UID: \"8a1ef072-2776-45cc-adea-2d9bfeaa6e3c\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-bsfmg" Jun 15 02:42:54.201276 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:54.201215 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-bsfmg" Jun 15 02:42:54.331655 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:54.331618 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-6f969c879b-bsfmg"] Jun 15 02:42:54.332694 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:42:54.332672 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a1ef072_2776_45cc_adea_2d9bfeaa6e3c.slice/crio-b359285f471d772a11321351b89313a36c4ecd2955b047ecc96db02b60318d07 WatchSource:0}: Error finding container b359285f471d772a11321351b89313a36c4ecd2955b047ecc96db02b60318d07: Status 404 returned error can't find the container with id b359285f471d772a11321351b89313a36c4ecd2955b047ecc96db02b60318d07 Jun 15 02:42:54.478279 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:54.478244 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-bsfmg" event={"ID":"8a1ef072-2776-45cc-adea-2d9bfeaa6e3c","Type":"ContainerStarted","Data":"5fd98457cf99e3a46b71a269c3385af2c80256bd83e14486e2c3ca7df9fcd23c"} Jun 15 02:42:54.478433 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:54.478314 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-bsfmg" event={"ID":"8a1ef072-2776-45cc-adea-2d9bfeaa6e3c","Type":"ContainerStarted","Data":"b359285f471d772a11321351b89313a36c4ecd2955b047ecc96db02b60318d07"} Jun 15 02:42:56.491256 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:56.491222 2579 generic.go:356] "Generic (PLEG): container finished" podID="8a1ef072-2776-45cc-adea-2d9bfeaa6e3c" containerID="5fd98457cf99e3a46b71a269c3385af2c80256bd83e14486e2c3ca7df9fcd23c" exitCode=0 Jun 15 02:42:56.491660 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:56.491289 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-bsfmg" event={"ID":"8a1ef072-2776-45cc-adea-2d9bfeaa6e3c","Type":"ContainerDied","Data":"5fd98457cf99e3a46b71a269c3385af2c80256bd83e14486e2c3ca7df9fcd23c"} Jun 15 02:42:57.498193 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:57.498161 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-bsfmg" event={"ID":"8a1ef072-2776-45cc-adea-2d9bfeaa6e3c","Type":"ContainerStarted","Data":"e8f2a38c11f99bf7e9f346749950baf9bca1ed880e6fa6065a46fb94a2af21ac"} Jun 15 02:42:57.519774 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:42:57.519732 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-bsfmg" podStartSLOduration=3.055506335 podStartE2EDuration="4.519719618s" totalImagesPullingTime="158.96208ms" totalInitContainerRuntime="1.305251203s" isStatefulPod=false podCreationTimestamp="2026-06-15 02:42:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 02:42:57.516935118 +0000 UTC m=+870.384719864" watchObservedRunningTime="2026-06-15 02:42:57.519719618 +0000 UTC m=+870.387504363" Jun 15 02:43:04.202159 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:43:04.202122 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-bsfmg" Jun 15 02:43:04.202677 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:43:04.202190 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-bsfmg" Jun 15 02:43:04.214575 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:43:04.214552 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-bsfmg" Jun 15 02:43:04.548041 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:43:04.548016 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-bsfmg" Jun 15 02:43:41.782301 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:43:41.782271 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-w4m86_fb6051d8-9291-4309-9047-9e8d0b5d2e48/console-operator/1.log" Jun 15 02:43:41.785729 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:43:41.785703 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-w4m86_fb6051d8-9291-4309-9047-9e8d0b5d2e48/console-operator/1.log" Jun 15 02:43:45.761080 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:43:45.761042 2579 generic.go:356] "Generic (PLEG): container finished" podID="69415e80-2a0f-4877-9ca7-ef5e11abf1da" containerID="9a42def4dd7d2542bd01d304d2c8d89810fc0fd1ef5f884f62fdf6eb39db8a67" exitCode=0 Jun 15 02:43:45.761437 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:43:45.761114 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cz5wzr" event={"ID":"69415e80-2a0f-4877-9ca7-ef5e11abf1da","Type":"ContainerDied","Data":"9a42def4dd7d2542bd01d304d2c8d89810fc0fd1ef5f884f62fdf6eb39db8a67"} Jun 15 02:43:46.768252 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:43:46.768219 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cz5wzr" event={"ID":"69415e80-2a0f-4877-9ca7-ef5e11abf1da","Type":"ContainerStarted","Data":"2d8e6c56d2acebf128aedf5d5bb5a3fb44c929981b5d9f2a3a75daaf3d63ca39"} Jun 15 02:43:46.788476 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:43:46.788431 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cz5wzr" podStartSLOduration=1.9703516159999999 podStartE2EDuration="55.788415832s" totalImagesPullingTime="188.31174ms" totalInitContainerRuntime="53.629752476s" isStatefulPod=false podCreationTimestamp="2026-06-15 02:42:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 02:43:46.786889932 +0000 UTC m=+919.654674679" watchObservedRunningTime="2026-06-15 02:43:46.788415832 +0000 UTC m=+919.656200576" Jun 15 02:43:51.699663 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:43:51.699616 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cz5wzr" Jun 15 02:43:51.700050 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:43:51.699685 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cz5wzr" Jun 15 02:43:51.711841 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:43:51.711815 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cz5wzr" Jun 15 02:43:51.808357 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:43:51.808331 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cz5wzr" Jun 15 02:44:16.575449 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:44:16.575416 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-77c94b6cf4-kbdc2"] Jun 15 02:44:16.579135 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:44:16.579112 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-77c94b6cf4-kbdc2" Jun 15 02:44:16.586431 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:44:16.586347 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-77c94b6cf4-kbdc2"] Jun 15 02:44:16.653599 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:44:16.653571 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/253e9425-183b-4f15-8891-34028ae2fea5-oidc-ca\") pod \"authorino-77c94b6cf4-kbdc2\" (UID: \"253e9425-183b-4f15-8891-34028ae2fea5\") " pod="kuadrant-system/authorino-77c94b6cf4-kbdc2" Jun 15 02:44:16.653794 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:44:16.653769 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/253e9425-183b-4f15-8891-34028ae2fea5-tls-cert\") pod \"authorino-77c94b6cf4-kbdc2\" (UID: \"253e9425-183b-4f15-8891-34028ae2fea5\") " pod="kuadrant-system/authorino-77c94b6cf4-kbdc2" Jun 15 02:44:16.653926 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:44:16.653833 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkdfq\" (UniqueName: \"kubernetes.io/projected/253e9425-183b-4f15-8891-34028ae2fea5-kube-api-access-zkdfq\") pod \"authorino-77c94b6cf4-kbdc2\" (UID: \"253e9425-183b-4f15-8891-34028ae2fea5\") " pod="kuadrant-system/authorino-77c94b6cf4-kbdc2" Jun 15 02:44:16.755035 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:44:16.755010 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkdfq\" (UniqueName: \"kubernetes.io/projected/253e9425-183b-4f15-8891-34028ae2fea5-kube-api-access-zkdfq\") pod \"authorino-77c94b6cf4-kbdc2\" (UID: \"253e9425-183b-4f15-8891-34028ae2fea5\") " pod="kuadrant-system/authorino-77c94b6cf4-kbdc2" Jun 15 02:44:16.755149 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:44:16.755049 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/253e9425-183b-4f15-8891-34028ae2fea5-oidc-ca\") pod \"authorino-77c94b6cf4-kbdc2\" (UID: \"253e9425-183b-4f15-8891-34028ae2fea5\") " pod="kuadrant-system/authorino-77c94b6cf4-kbdc2" Jun 15 02:44:16.755149 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:44:16.755097 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/253e9425-183b-4f15-8891-34028ae2fea5-tls-cert\") pod \"authorino-77c94b6cf4-kbdc2\" (UID: \"253e9425-183b-4f15-8891-34028ae2fea5\") " pod="kuadrant-system/authorino-77c94b6cf4-kbdc2" Jun 15 02:44:16.755684 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:44:16.755663 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/253e9425-183b-4f15-8891-34028ae2fea5-oidc-ca\") pod \"authorino-77c94b6cf4-kbdc2\" (UID: \"253e9425-183b-4f15-8891-34028ae2fea5\") " pod="kuadrant-system/authorino-77c94b6cf4-kbdc2" Jun 15 02:44:16.757298 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:44:16.757272 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/253e9425-183b-4f15-8891-34028ae2fea5-tls-cert\") pod \"authorino-77c94b6cf4-kbdc2\" (UID: \"253e9425-183b-4f15-8891-34028ae2fea5\") " pod="kuadrant-system/authorino-77c94b6cf4-kbdc2" Jun 15 02:44:16.762402 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:44:16.762379 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkdfq\" (UniqueName: \"kubernetes.io/projected/253e9425-183b-4f15-8891-34028ae2fea5-kube-api-access-zkdfq\") pod \"authorino-77c94b6cf4-kbdc2\" (UID: \"253e9425-183b-4f15-8891-34028ae2fea5\") " pod="kuadrant-system/authorino-77c94b6cf4-kbdc2" Jun 15 02:44:16.888658 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:44:16.888574 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-77c94b6cf4-kbdc2" Jun 15 02:44:17.022991 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:44:17.022966 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-77c94b6cf4-kbdc2"] Jun 15 02:44:17.024850 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:44:17.024821 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod253e9425_183b_4f15_8891_34028ae2fea5.slice/crio-815a620f3210ac65a210f54b541e0968455987ff3724f70c07a6356496228e4e WatchSource:0}: Error finding container 815a620f3210ac65a210f54b541e0968455987ff3724f70c07a6356496228e4e: Status 404 returned error can't find the container with id 815a620f3210ac65a210f54b541e0968455987ff3724f70c07a6356496228e4e Jun 15 02:44:17.943694 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:44:17.943587 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-77c94b6cf4-kbdc2" event={"ID":"253e9425-183b-4f15-8891-34028ae2fea5","Type":"ContainerStarted","Data":"de4627d1e7385a63e9e4ad272ef8a334541b4d1b4ffd3d322f958a043ea0de3e"} Jun 15 02:44:17.943694 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:44:17.943652 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-77c94b6cf4-kbdc2" event={"ID":"253e9425-183b-4f15-8891-34028ae2fea5","Type":"ContainerStarted","Data":"815a620f3210ac65a210f54b541e0968455987ff3724f70c07a6356496228e4e"} Jun 15 02:44:17.959069 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:44:17.959012 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-77c94b6cf4-kbdc2" podStartSLOduration=1.5024512410000002 podStartE2EDuration="1.958996038s" totalImagesPullingTime="456.544797ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 02:44:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 02:44:17.957391081 +0000 UTC m=+950.825175827" watchObservedRunningTime="2026-06-15 02:44:17.958996038 +0000 UTC m=+950.826780782" Jun 15 02:44:17.991665 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:44:17.991490 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d8fcddbc5-4v9h5"] Jun 15 02:44:17.991930 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:44:17.991724 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-6d8fcddbc5-4v9h5" podUID="20d349ab-b828-46f7-bf08-2152b6dfa9c1" containerName="authorino" containerID="cri-o://fb704c36e27aba2fb5cedad739d2fd141a106eec8da48d53e8134e48707629b1" gracePeriod=30 Jun 15 02:44:18.266738 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:44:18.266704 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d8fcddbc5-4v9h5" Jun 15 02:44:18.369367 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:44:18.369338 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-skqxg\" (UniqueName: \"kubernetes.io/projected/20d349ab-b828-46f7-bf08-2152b6dfa9c1-kube-api-access-skqxg\") pod \"20d349ab-b828-46f7-bf08-2152b6dfa9c1\" (UID: \"20d349ab-b828-46f7-bf08-2152b6dfa9c1\") " Jun 15 02:44:18.369560 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:44:18.369406 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/20d349ab-b828-46f7-bf08-2152b6dfa9c1-oidc-ca\") pod \"20d349ab-b828-46f7-bf08-2152b6dfa9c1\" (UID: \"20d349ab-b828-46f7-bf08-2152b6dfa9c1\") " Jun 15 02:44:18.369560 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:44:18.369430 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/20d349ab-b828-46f7-bf08-2152b6dfa9c1-tls-cert\") pod \"20d349ab-b828-46f7-bf08-2152b6dfa9c1\" (UID: \"20d349ab-b828-46f7-bf08-2152b6dfa9c1\") " Jun 15 02:44:18.371610 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:44:18.371564 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20d349ab-b828-46f7-bf08-2152b6dfa9c1-kube-api-access-skqxg" (OuterVolumeSpecName: "kube-api-access-skqxg") pod "20d349ab-b828-46f7-bf08-2152b6dfa9c1" (UID: "20d349ab-b828-46f7-bf08-2152b6dfa9c1"). InnerVolumeSpecName "kube-api-access-skqxg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 02:44:18.374496 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:44:18.374468 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20d349ab-b828-46f7-bf08-2152b6dfa9c1-oidc-ca" (OuterVolumeSpecName: "oidc-ca") pod "20d349ab-b828-46f7-bf08-2152b6dfa9c1" (UID: "20d349ab-b828-46f7-bf08-2152b6dfa9c1"). InnerVolumeSpecName "oidc-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 02:44:18.379750 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:44:18.379725 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20d349ab-b828-46f7-bf08-2152b6dfa9c1-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "20d349ab-b828-46f7-bf08-2152b6dfa9c1" (UID: "20d349ab-b828-46f7-bf08-2152b6dfa9c1"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 15 02:44:18.470230 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:44:18.470184 2579 reconciler_common.go:299] "Volume detached for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/20d349ab-b828-46f7-bf08-2152b6dfa9c1-oidc-ca\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:44:18.470230 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:44:18.470215 2579 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/20d349ab-b828-46f7-bf08-2152b6dfa9c1-tls-cert\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:44:18.470230 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:44:18.470224 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-skqxg\" (UniqueName: \"kubernetes.io/projected/20d349ab-b828-46f7-bf08-2152b6dfa9c1-kube-api-access-skqxg\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:44:18.950823 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:44:18.950779 2579 generic.go:356] "Generic (PLEG): container finished" podID="20d349ab-b828-46f7-bf08-2152b6dfa9c1" containerID="fb704c36e27aba2fb5cedad739d2fd141a106eec8da48d53e8134e48707629b1" exitCode=0 Jun 15 02:44:18.950823 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:44:18.950822 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d8fcddbc5-4v9h5" Jun 15 02:44:18.951262 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:44:18.950862 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6d8fcddbc5-4v9h5" event={"ID":"20d349ab-b828-46f7-bf08-2152b6dfa9c1","Type":"ContainerDied","Data":"fb704c36e27aba2fb5cedad739d2fd141a106eec8da48d53e8134e48707629b1"} Jun 15 02:44:18.951262 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:44:18.950905 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6d8fcddbc5-4v9h5" event={"ID":"20d349ab-b828-46f7-bf08-2152b6dfa9c1","Type":"ContainerDied","Data":"e3453b0f26f7eb7993f4407171aba56e0a2b309bd20ce48440e002efd1637988"} Jun 15 02:44:18.951262 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:44:18.950922 2579 scope.go:117] "RemoveContainer" containerID="fb704c36e27aba2fb5cedad739d2fd141a106eec8da48d53e8134e48707629b1" Jun 15 02:44:18.960474 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:44:18.960457 2579 scope.go:117] "RemoveContainer" containerID="fb704c36e27aba2fb5cedad739d2fd141a106eec8da48d53e8134e48707629b1" Jun 15 02:44:18.960744 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:44:18.960725 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb704c36e27aba2fb5cedad739d2fd141a106eec8da48d53e8134e48707629b1\": container with ID starting with fb704c36e27aba2fb5cedad739d2fd141a106eec8da48d53e8134e48707629b1 not found: ID does not exist" containerID="fb704c36e27aba2fb5cedad739d2fd141a106eec8da48d53e8134e48707629b1" Jun 15 02:44:18.960808 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:44:18.960751 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb704c36e27aba2fb5cedad739d2fd141a106eec8da48d53e8134e48707629b1"} err="failed to get container status \"fb704c36e27aba2fb5cedad739d2fd141a106eec8da48d53e8134e48707629b1\": rpc error: code = NotFound desc = could not find container \"fb704c36e27aba2fb5cedad739d2fd141a106eec8da48d53e8134e48707629b1\": container with ID starting with fb704c36e27aba2fb5cedad739d2fd141a106eec8da48d53e8134e48707629b1 not found: ID does not exist" Jun 15 02:44:18.974505 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:44:18.974479 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d8fcddbc5-4v9h5"] Jun 15 02:44:18.978383 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:44:18.978361 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6d8fcddbc5-4v9h5"] Jun 15 02:44:19.492795 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:44:19.492762 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20d349ab-b828-46f7-bf08-2152b6dfa9c1" path="/var/lib/kubelet/pods/20d349ab-b828-46f7-bf08-2152b6dfa9c1/volumes" Jun 15 02:45:46.145973 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:45:46.145940 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-ddf7649c7-jxwp6"] Jun 15 02:45:46.146485 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:45:46.146157 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-ddf7649c7-jxwp6" podUID="30e800c7-b85f-4404-be63-e9ad90b8dee1" containerName="manager" containerID="cri-o://409de71dd6dc305b25d4d5dadf6160ac76275c8b56869259f21112af00ad8edc" gracePeriod=30 Jun 15 02:45:46.399895 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:45:46.399840 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-ddf7649c7-jxwp6" Jun 15 02:45:46.458383 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:45:46.458356 2579 generic.go:356] "Generic (PLEG): container finished" podID="30e800c7-b85f-4404-be63-e9ad90b8dee1" containerID="409de71dd6dc305b25d4d5dadf6160ac76275c8b56869259f21112af00ad8edc" exitCode=0 Jun 15 02:45:46.458540 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:45:46.458410 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-ddf7649c7-jxwp6" Jun 15 02:45:46.458540 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:45:46.458440 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-ddf7649c7-jxwp6" event={"ID":"30e800c7-b85f-4404-be63-e9ad90b8dee1","Type":"ContainerDied","Data":"409de71dd6dc305b25d4d5dadf6160ac76275c8b56869259f21112af00ad8edc"} Jun 15 02:45:46.458540 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:45:46.458498 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-ddf7649c7-jxwp6" event={"ID":"30e800c7-b85f-4404-be63-e9ad90b8dee1","Type":"ContainerDied","Data":"a2ad9b8d88fd3a55bd96386b8df14a78942424eba74808c387e1ea3561af8ccc"} Jun 15 02:45:46.458669 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:45:46.458557 2579 scope.go:117] "RemoveContainer" containerID="409de71dd6dc305b25d4d5dadf6160ac76275c8b56869259f21112af00ad8edc" Jun 15 02:45:46.467346 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:45:46.467331 2579 scope.go:117] "RemoveContainer" containerID="409de71dd6dc305b25d4d5dadf6160ac76275c8b56869259f21112af00ad8edc" Jun 15 02:45:46.467596 ip-10-0-143-51 kubenswrapper[2579]: E0615 02:45:46.467576 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"409de71dd6dc305b25d4d5dadf6160ac76275c8b56869259f21112af00ad8edc\": container with ID starting with 409de71dd6dc305b25d4d5dadf6160ac76275c8b56869259f21112af00ad8edc not found: ID does not exist" containerID="409de71dd6dc305b25d4d5dadf6160ac76275c8b56869259f21112af00ad8edc" Jun 15 02:45:46.467717 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:45:46.467602 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"409de71dd6dc305b25d4d5dadf6160ac76275c8b56869259f21112af00ad8edc"} err="failed to get container status \"409de71dd6dc305b25d4d5dadf6160ac76275c8b56869259f21112af00ad8edc\": rpc error: code = NotFound desc = could not find container \"409de71dd6dc305b25d4d5dadf6160ac76275c8b56869259f21112af00ad8edc\": container with ID starting with 409de71dd6dc305b25d4d5dadf6160ac76275c8b56869259f21112af00ad8edc not found: ID does not exist" Jun 15 02:45:46.475988 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:45:46.475972 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/30e800c7-b85f-4404-be63-e9ad90b8dee1-webhook-cert\") pod \"30e800c7-b85f-4404-be63-e9ad90b8dee1\" (UID: \"30e800c7-b85f-4404-be63-e9ad90b8dee1\") " Jun 15 02:45:46.476048 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:45:46.476021 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mrhzc\" (UniqueName: \"kubernetes.io/projected/30e800c7-b85f-4404-be63-e9ad90b8dee1-kube-api-access-mrhzc\") pod \"30e800c7-b85f-4404-be63-e9ad90b8dee1\" (UID: \"30e800c7-b85f-4404-be63-e9ad90b8dee1\") " Jun 15 02:45:46.477827 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:45:46.477804 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30e800c7-b85f-4404-be63-e9ad90b8dee1-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "30e800c7-b85f-4404-be63-e9ad90b8dee1" (UID: "30e800c7-b85f-4404-be63-e9ad90b8dee1"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 15 02:45:46.478012 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:45:46.477977 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30e800c7-b85f-4404-be63-e9ad90b8dee1-kube-api-access-mrhzc" (OuterVolumeSpecName: "kube-api-access-mrhzc") pod "30e800c7-b85f-4404-be63-e9ad90b8dee1" (UID: "30e800c7-b85f-4404-be63-e9ad90b8dee1"). InnerVolumeSpecName "kube-api-access-mrhzc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 02:45:46.577169 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:45:46.577149 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mrhzc\" (UniqueName: \"kubernetes.io/projected/30e800c7-b85f-4404-be63-e9ad90b8dee1-kube-api-access-mrhzc\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:45:46.577169 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:45:46.577170 2579 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/30e800c7-b85f-4404-be63-e9ad90b8dee1-webhook-cert\") on node \"ip-10-0-143-51.ec2.internal\" DevicePath \"\"" Jun 15 02:45:46.780157 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:45:46.780130 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-ddf7649c7-jxwp6"] Jun 15 02:45:46.783762 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:45:46.783742 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-ddf7649c7-jxwp6"] Jun 15 02:45:47.493848 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:45:47.493813 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30e800c7-b85f-4404-be63-e9ad90b8dee1" path="/var/lib/kubelet/pods/30e800c7-b85f-4404-be63-e9ad90b8dee1/volumes" Jun 15 02:45:47.551378 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:45:47.551348 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-ddf7649c7-b4wrs"] Jun 15 02:45:47.551711 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:45:47.551698 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20d349ab-b828-46f7-bf08-2152b6dfa9c1" containerName="authorino" Jun 15 02:45:47.551761 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:45:47.551712 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="20d349ab-b828-46f7-bf08-2152b6dfa9c1" containerName="authorino" Jun 15 02:45:47.551761 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:45:47.551738 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30e800c7-b85f-4404-be63-e9ad90b8dee1" containerName="manager" Jun 15 02:45:47.551761 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:45:47.551744 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="30e800c7-b85f-4404-be63-e9ad90b8dee1" containerName="manager" Jun 15 02:45:47.551859 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:45:47.551794 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="20d349ab-b828-46f7-bf08-2152b6dfa9c1" containerName="authorino" Jun 15 02:45:47.551859 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:45:47.551803 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="30e800c7-b85f-4404-be63-e9ad90b8dee1" containerName="manager" Jun 15 02:45:47.556121 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:45:47.556104 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-ddf7649c7-b4wrs" Jun 15 02:45:47.560056 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:45:47.560028 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 15 02:45:47.560203 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:45:47.560068 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-mqtvz\"" Jun 15 02:45:47.560203 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:45:47.560170 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jun 15 02:45:47.567841 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:45:47.567814 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-ddf7649c7-b4wrs"] Jun 15 02:45:47.585460 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:45:47.585433 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3a8bbc51-fb13-44bb-8156-acfb444405fe-webhook-cert\") pod \"maas-controller-ddf7649c7-b4wrs\" (UID: \"3a8bbc51-fb13-44bb-8156-acfb444405fe\") " pod="opendatahub/maas-controller-ddf7649c7-b4wrs" Jun 15 02:45:47.585581 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:45:47.585474 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ftcs\" (UniqueName: \"kubernetes.io/projected/3a8bbc51-fb13-44bb-8156-acfb444405fe-kube-api-access-5ftcs\") pod \"maas-controller-ddf7649c7-b4wrs\" (UID: \"3a8bbc51-fb13-44bb-8156-acfb444405fe\") " pod="opendatahub/maas-controller-ddf7649c7-b4wrs" Jun 15 02:45:47.686356 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:45:47.686330 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3a8bbc51-fb13-44bb-8156-acfb444405fe-webhook-cert\") pod \"maas-controller-ddf7649c7-b4wrs\" (UID: \"3a8bbc51-fb13-44bb-8156-acfb444405fe\") " pod="opendatahub/maas-controller-ddf7649c7-b4wrs" Jun 15 02:45:47.686498 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:45:47.686370 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ftcs\" (UniqueName: \"kubernetes.io/projected/3a8bbc51-fb13-44bb-8156-acfb444405fe-kube-api-access-5ftcs\") pod \"maas-controller-ddf7649c7-b4wrs\" (UID: \"3a8bbc51-fb13-44bb-8156-acfb444405fe\") " pod="opendatahub/maas-controller-ddf7649c7-b4wrs" Jun 15 02:45:47.688668 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:45:47.688627 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3a8bbc51-fb13-44bb-8156-acfb444405fe-webhook-cert\") pod \"maas-controller-ddf7649c7-b4wrs\" (UID: \"3a8bbc51-fb13-44bb-8156-acfb444405fe\") " pod="opendatahub/maas-controller-ddf7649c7-b4wrs" Jun 15 02:45:47.693121 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:45:47.693102 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ftcs\" (UniqueName: \"kubernetes.io/projected/3a8bbc51-fb13-44bb-8156-acfb444405fe-kube-api-access-5ftcs\") pod \"maas-controller-ddf7649c7-b4wrs\" (UID: \"3a8bbc51-fb13-44bb-8156-acfb444405fe\") " pod="opendatahub/maas-controller-ddf7649c7-b4wrs" Jun 15 02:45:47.869172 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:45:47.869104 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-ddf7649c7-b4wrs" Jun 15 02:45:47.997827 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:45:47.997804 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-ddf7649c7-b4wrs"] Jun 15 02:45:47.999334 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:45:47.999292 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a8bbc51_fb13_44bb_8156_acfb444405fe.slice/crio-f1e28c2f03d3a08473418c32e6024e68bffd346d631a1ea4aa37ef3882b5bade WatchSource:0}: Error finding container f1e28c2f03d3a08473418c32e6024e68bffd346d631a1ea4aa37ef3882b5bade: Status 404 returned error can't find the container with id f1e28c2f03d3a08473418c32e6024e68bffd346d631a1ea4aa37ef3882b5bade Jun 15 02:45:48.000768 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:45:48.000748 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 15 02:45:48.472374 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:45:48.472346 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-ddf7649c7-b4wrs" event={"ID":"3a8bbc51-fb13-44bb-8156-acfb444405fe","Type":"ContainerStarted","Data":"7c714d961c393372b728c87e4fa49b11e599090bb1391d19d949e568088c29c9"} Jun 15 02:45:48.472511 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:45:48.472411 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-ddf7649c7-b4wrs" Jun 15 02:45:48.472511 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:45:48.472435 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-ddf7649c7-b4wrs" event={"ID":"3a8bbc51-fb13-44bb-8156-acfb444405fe","Type":"ContainerStarted","Data":"f1e28c2f03d3a08473418c32e6024e68bffd346d631a1ea4aa37ef3882b5bade"} Jun 15 02:45:48.490398 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:45:48.490348 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-ddf7649c7-b4wrs" podStartSLOduration=1.101062635 podStartE2EDuration="1.490329616s" totalImagesPullingTime="389.266981ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 02:45:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 02:45:48.486897004 +0000 UTC m=+1041.354681748" watchObservedRunningTime="2026-06-15 02:45:48.490329616 +0000 UTC m=+1041.358114360" Jun 15 02:45:59.483831 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:45:59.483800 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-ddf7649c7-b4wrs" Jun 15 02:48:41.826366 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:48:41.826336 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-w4m86_fb6051d8-9291-4309-9047-9e8d0b5d2e48/console-operator/1.log" Jun 15 02:48:41.830681 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:48:41.830660 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-w4m86_fb6051d8-9291-4309-9047-9e8d0b5d2e48/console-operator/1.log" Jun 15 02:49:02.061995 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:02.061963 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-77c94b6cf4-kbdc2_253e9425-183b-4f15-8891-34028ae2fea5/authorino/0.log" Jun 15 02:49:07.685674 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:07.685622 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-ddf7649c7-b4wrs_3a8bbc51-fb13-44bb-8156-acfb444405fe/manager/0.log" Jun 15 02:49:08.173475 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:08.173442 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-785bc74785-wd7m6_7e1b26b3-7e13-45d8-94d5-9cc30ef55807/copy-manifests/0.log" Jun 15 02:49:08.186105 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:08.186076 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-785bc74785-wd7m6_7e1b26b3-7e13-45d8-94d5-9cc30ef55807/manager/0.log" Jun 15 02:49:09.315331 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:09.315306 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gq984_5f0a46ca-b767-486a-b97e-f3b0f34098c4/extract/0.log" Jun 15 02:49:09.321788 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:09.321769 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gq984_5f0a46ca-b767-486a-b97e-f3b0f34098c4/util/0.log" Jun 15 02:49:09.327932 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:09.327912 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gq984_5f0a46ca-b767-486a-b97e-f3b0f34098c4/pull/0.log" Jun 15 02:49:09.451005 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:09.450984 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mbjjv_9fa08f07-32c5-4a19-a836-0f0bc4a33280/extract/0.log" Jun 15 02:49:09.456307 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:09.456287 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mbjjv_9fa08f07-32c5-4a19-a836-0f0bc4a33280/util/0.log" Jun 15 02:49:09.461752 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:09.461734 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mbjjv_9fa08f07-32c5-4a19-a836-0f0bc4a33280/pull/0.log" Jun 15 02:49:09.588075 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:09.588015 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjmwq_7e76d04b-a546-422a-8a7b-98936020ed85/util/0.log" Jun 15 02:49:09.594146 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:09.594117 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjmwq_7e76d04b-a546-422a-8a7b-98936020ed85/pull/0.log" Jun 15 02:49:09.600223 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:09.600207 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjmwq_7e76d04b-a546-422a-8a7b-98936020ed85/extract/0.log" Jun 15 02:49:09.722167 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:09.722151 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t6zwb_48c71614-5ca4-40b8-8ac2-fed4e8989831/extract/0.log" Jun 15 02:49:09.728485 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:09.728467 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t6zwb_48c71614-5ca4-40b8-8ac2-fed4e8989831/util/0.log" Jun 15 02:49:09.734270 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:09.734256 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t6zwb_48c71614-5ca4-40b8-8ac2-fed4e8989831/pull/0.log" Jun 15 02:49:09.867780 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:09.867732 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-77c94b6cf4-kbdc2_253e9425-183b-4f15-8891-34028ae2fea5/authorino/0.log" Jun 15 02:49:10.247020 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:10.247002 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-m4kcm_c09566f4-206a-4394-934d-6acb05947718/kuadrant-console-plugin/0.log" Jun 15 02:49:10.521651 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:10.521573 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-8tzrh_ef684a3a-f3a0-47ba-bf1c-0b2249816ef4/manager/0.log" Jun 15 02:49:11.570142 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:11.570105 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-59c549f7b4-gpxhv_bd72927c-a9ed-46a5-879a-b86ddfce6a3e/kube-auth-proxy/0.log" Jun 15 02:49:11.812395 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:11.812369 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-579785bc64-rngxp_007497bc-5b0b-4320-bda5-4e08f8228820/payload-pre-processing/0.log" Jun 15 02:49:12.466893 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:12.466860 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-f5c55bdc-wbmcv_1be8db44-3fb9-4cd4-b3af-ffaec8a44636/main/0.log" Jun 15 02:49:12.474030 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:12.474009 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-f5c55bdc-wbmcv_1be8db44-3fb9-4cd4-b3af-ffaec8a44636/storage-initializer/0.log" Jun 15 02:49:12.744310 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:12.744282 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-68bb6c58d9-k8kjc_e0bb11ae-dbd7-4174-84d6-a387a5842a1e/storage-initializer/0.log" Jun 15 02:49:12.751021 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:12.750995 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-68bb6c58d9-k8kjc_e0bb11ae-dbd7-4174-84d6-a387a5842a1e/main/0.log" Jun 15 02:49:12.876428 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:12.876390 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cz5wzr_69415e80-2a0f-4877-9ca7-ef5e11abf1da/storage-initializer/0.log" Jun 15 02:49:12.882337 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:12.882319 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cz5wzr_69415e80-2a0f-4877-9ca7-ef5e11abf1da/main/0.log" Jun 15 02:49:13.007716 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:13.007661 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-6f969c879b-bsfmg_8a1ef072-2776-45cc-adea-2d9bfeaa6e3c/storage-initializer/0.log" Jun 15 02:49:13.014664 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:13.014647 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-6f969c879b-bsfmg_8a1ef072-2776-45cc-adea-2d9bfeaa6e3c/main/0.log" Jun 15 02:49:20.209154 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:20.209126 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-qnxs7_d436483f-cbb4-4ce9-b4d5-a21564cf6e97/global-pull-secret-syncer/0.log" Jun 15 02:49:20.299557 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:20.299526 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-tsd5w_fc66bce4-865f-4b30-aca2-93cedd05ffe3/konnectivity-agent/0.log" Jun 15 02:49:20.355988 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:20.355961 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-51.ec2.internal_66c462553606b0900fb46b9deb100df6/haproxy/0.log" Jun 15 02:49:24.422187 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:24.422158 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gq984_5f0a46ca-b767-486a-b97e-f3b0f34098c4/extract/0.log" Jun 15 02:49:24.444233 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:24.444206 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gq984_5f0a46ca-b767-486a-b97e-f3b0f34098c4/util/0.log" Jun 15 02:49:24.464059 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:24.464039 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gq984_5f0a46ca-b767-486a-b97e-f3b0f34098c4/pull/0.log" Jun 15 02:49:24.489641 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:24.489618 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mbjjv_9fa08f07-32c5-4a19-a836-0f0bc4a33280/extract/0.log" Jun 15 02:49:24.509947 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:24.509925 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mbjjv_9fa08f07-32c5-4a19-a836-0f0bc4a33280/util/0.log" Jun 15 02:49:24.528407 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:24.528390 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mbjjv_9fa08f07-32c5-4a19-a836-0f0bc4a33280/pull/0.log" Jun 15 02:49:24.560384 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:24.560368 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjmwq_7e76d04b-a546-422a-8a7b-98936020ed85/extract/0.log" Jun 15 02:49:24.579234 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:24.579215 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjmwq_7e76d04b-a546-422a-8a7b-98936020ed85/util/0.log" Jun 15 02:49:24.599191 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:24.599171 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjmwq_7e76d04b-a546-422a-8a7b-98936020ed85/pull/0.log" Jun 15 02:49:24.623114 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:24.623093 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t6zwb_48c71614-5ca4-40b8-8ac2-fed4e8989831/extract/0.log" Jun 15 02:49:24.640246 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:24.640219 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t6zwb_48c71614-5ca4-40b8-8ac2-fed4e8989831/util/0.log" Jun 15 02:49:24.658809 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:24.658790 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t6zwb_48c71614-5ca4-40b8-8ac2-fed4e8989831/pull/0.log" Jun 15 02:49:24.735720 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:24.735699 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-77c94b6cf4-kbdc2_253e9425-183b-4f15-8891-34028ae2fea5/authorino/0.log" Jun 15 02:49:24.806897 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:24.806876 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-m4kcm_c09566f4-206a-4394-934d-6acb05947718/kuadrant-console-plugin/0.log" Jun 15 02:49:24.869658 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:24.869625 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-8tzrh_ef684a3a-f3a0-47ba-bf1c-0b2249816ef4/manager/0.log" Jun 15 02:49:26.727466 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:26.727422 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_eb38ec6a-eee4-43fe-b3e1-51aee1ca310b/alertmanager/0.log" Jun 15 02:49:26.750610 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:26.750587 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_eb38ec6a-eee4-43fe-b3e1-51aee1ca310b/config-reloader/0.log" Jun 15 02:49:26.768200 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:26.768182 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_eb38ec6a-eee4-43fe-b3e1-51aee1ca310b/kube-rbac-proxy-web/0.log" Jun 15 02:49:26.786893 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:26.786878 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_eb38ec6a-eee4-43fe-b3e1-51aee1ca310b/kube-rbac-proxy/0.log" Jun 15 02:49:26.806916 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:26.806900 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_eb38ec6a-eee4-43fe-b3e1-51aee1ca310b/kube-rbac-proxy-metric/0.log" Jun 15 02:49:26.824206 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:26.824185 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_eb38ec6a-eee4-43fe-b3e1-51aee1ca310b/prom-label-proxy/0.log" Jun 15 02:49:26.844385 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:26.844371 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_eb38ec6a-eee4-43fe-b3e1-51aee1ca310b/init-config-reloader/0.log" Jun 15 02:49:27.150719 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:27.150657 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-l6v8k_4bf0467b-72a9-4ab5-a525-2385561f9da4/node-exporter/0.log" Jun 15 02:49:27.167617 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:27.167596 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-l6v8k_4bf0467b-72a9-4ab5-a525-2385561f9da4/kube-rbac-proxy/0.log" Jun 15 02:49:27.184340 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:27.184325 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-l6v8k_4bf0467b-72a9-4ab5-a525-2385561f9da4/init-textfile/0.log" Jun 15 02:49:27.213044 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:27.213028 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-6b8d965dd5-4m5dz_cfbbc525-96a9-4c20-b125-742b83a9db41/kube-rbac-proxy-main/0.log" Jun 15 02:49:27.234145 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:27.234128 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-6b8d965dd5-4m5dz_cfbbc525-96a9-4c20-b125-742b83a9db41/kube-rbac-proxy-self/0.log" Jun 15 02:49:27.253997 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:27.253980 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-6b8d965dd5-4m5dz_cfbbc525-96a9-4c20-b125-742b83a9db41/openshift-state-metrics/0.log" Jun 15 02:49:27.431588 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:27.431513 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-864fcfc6f9-z7lj5_09014b39-704e-4344-a80e-535a5a9a0e81/prometheus-operator/0.log" Jun 15 02:49:27.467607 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:27.467589 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-864fcfc6f9-z7lj5_09014b39-704e-4344-a80e-535a5a9a0e81/kube-rbac-proxy/0.log" Jun 15 02:49:27.491317 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:27.491292 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5cf667749c-bml6h_44e88673-e402-4647-81e6-52be405164ef/prometheus-operator-admission-webhook/0.log" Jun 15 02:49:27.528991 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:27.528974 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-646b5685b-q2pwl_fcb07ac5-8ec3-45d0-abbc-76db097c3ecb/telemeter-client/0.log" Jun 15 02:49:27.548070 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:27.548053 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-646b5685b-q2pwl_fcb07ac5-8ec3-45d0-abbc-76db097c3ecb/reload/0.log" Jun 15 02:49:27.569318 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:27.569298 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-646b5685b-q2pwl_fcb07ac5-8ec3-45d0-abbc-76db097c3ecb/kube-rbac-proxy/0.log" Jun 15 02:49:29.940088 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:29.940026 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-w4m86_fb6051d8-9291-4309-9047-9e8d0b5d2e48/console-operator/1.log" Jun 15 02:49:29.945144 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:29.945120 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5dbbb5d744-w4m86_fb6051d8-9291-4309-9047-9e8d0b5d2e48/console-operator/2.log" Jun 15 02:49:30.515006 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:30.514961 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-b5d5ff784-9m5g7_57191038-ba31-4483-9d09-dc0a07f99dbd/console/0.log" Jun 15 02:49:31.551014 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:31.550980 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-7twvb/perf-node-gather-daemonset-6bbgb"] Jun 15 02:49:31.554517 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:31.554498 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7twvb/perf-node-gather-daemonset-6bbgb" Jun 15 02:49:31.556616 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:31.556588 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7twvb\"/\"openshift-service-ca.crt\"" Jun 15 02:49:31.556936 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:31.556918 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7twvb\"/\"kube-root-ca.crt\"" Jun 15 02:49:31.557370 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:31.557354 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-7twvb\"/\"default-dockercfg-qxpq8\"" Jun 15 02:49:31.562497 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:31.562478 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7twvb/perf-node-gather-daemonset-6bbgb"] Jun 15 02:49:31.706010 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:31.705980 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/531388aa-7df6-4fe3-b3bf-972ea37db329-proc\") pod \"perf-node-gather-daemonset-6bbgb\" (UID: \"531388aa-7df6-4fe3-b3bf-972ea37db329\") " pod="openshift-must-gather-7twvb/perf-node-gather-daemonset-6bbgb" Jun 15 02:49:31.706164 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:31.706018 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/531388aa-7df6-4fe3-b3bf-972ea37db329-lib-modules\") pod \"perf-node-gather-daemonset-6bbgb\" (UID: \"531388aa-7df6-4fe3-b3bf-972ea37db329\") " pod="openshift-must-gather-7twvb/perf-node-gather-daemonset-6bbgb" Jun 15 02:49:31.706164 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:31.706038 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/531388aa-7df6-4fe3-b3bf-972ea37db329-sys\") pod \"perf-node-gather-daemonset-6bbgb\" (UID: \"531388aa-7df6-4fe3-b3bf-972ea37db329\") " pod="openshift-must-gather-7twvb/perf-node-gather-daemonset-6bbgb" Jun 15 02:49:31.706164 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:31.706058 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/531388aa-7df6-4fe3-b3bf-972ea37db329-podres\") pod \"perf-node-gather-daemonset-6bbgb\" (UID: \"531388aa-7df6-4fe3-b3bf-972ea37db329\") " pod="openshift-must-gather-7twvb/perf-node-gather-daemonset-6bbgb" Jun 15 02:49:31.706164 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:31.706131 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9772h\" (UniqueName: \"kubernetes.io/projected/531388aa-7df6-4fe3-b3bf-972ea37db329-kube-api-access-9772h\") pod \"perf-node-gather-daemonset-6bbgb\" (UID: \"531388aa-7df6-4fe3-b3bf-972ea37db329\") " pod="openshift-must-gather-7twvb/perf-node-gather-daemonset-6bbgb" Jun 15 02:49:31.806714 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:31.806645 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/531388aa-7df6-4fe3-b3bf-972ea37db329-proc\") pod \"perf-node-gather-daemonset-6bbgb\" (UID: \"531388aa-7df6-4fe3-b3bf-972ea37db329\") " pod="openshift-must-gather-7twvb/perf-node-gather-daemonset-6bbgb" Jun 15 02:49:31.806714 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:31.806679 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/531388aa-7df6-4fe3-b3bf-972ea37db329-lib-modules\") pod \"perf-node-gather-daemonset-6bbgb\" (UID: \"531388aa-7df6-4fe3-b3bf-972ea37db329\") " pod="openshift-must-gather-7twvb/perf-node-gather-daemonset-6bbgb" Jun 15 02:49:31.806714 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:31.806699 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/531388aa-7df6-4fe3-b3bf-972ea37db329-sys\") pod \"perf-node-gather-daemonset-6bbgb\" (UID: \"531388aa-7df6-4fe3-b3bf-972ea37db329\") " pod="openshift-must-gather-7twvb/perf-node-gather-daemonset-6bbgb" Jun 15 02:49:31.806961 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:31.806720 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/531388aa-7df6-4fe3-b3bf-972ea37db329-podres\") pod \"perf-node-gather-daemonset-6bbgb\" (UID: \"531388aa-7df6-4fe3-b3bf-972ea37db329\") " pod="openshift-must-gather-7twvb/perf-node-gather-daemonset-6bbgb" Jun 15 02:49:31.806961 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:31.806780 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9772h\" (UniqueName: \"kubernetes.io/projected/531388aa-7df6-4fe3-b3bf-972ea37db329-kube-api-access-9772h\") pod \"perf-node-gather-daemonset-6bbgb\" (UID: \"531388aa-7df6-4fe3-b3bf-972ea37db329\") " pod="openshift-must-gather-7twvb/perf-node-gather-daemonset-6bbgb" Jun 15 02:49:31.806961 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:31.806791 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/531388aa-7df6-4fe3-b3bf-972ea37db329-sys\") pod \"perf-node-gather-daemonset-6bbgb\" (UID: \"531388aa-7df6-4fe3-b3bf-972ea37db329\") " pod="openshift-must-gather-7twvb/perf-node-gather-daemonset-6bbgb" Jun 15 02:49:31.806961 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:31.806782 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/531388aa-7df6-4fe3-b3bf-972ea37db329-proc\") pod \"perf-node-gather-daemonset-6bbgb\" (UID: \"531388aa-7df6-4fe3-b3bf-972ea37db329\") " pod="openshift-must-gather-7twvb/perf-node-gather-daemonset-6bbgb" Jun 15 02:49:31.806961 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:31.806834 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/531388aa-7df6-4fe3-b3bf-972ea37db329-podres\") pod \"perf-node-gather-daemonset-6bbgb\" (UID: \"531388aa-7df6-4fe3-b3bf-972ea37db329\") " pod="openshift-must-gather-7twvb/perf-node-gather-daemonset-6bbgb" Jun 15 02:49:31.806961 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:31.806843 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/531388aa-7df6-4fe3-b3bf-972ea37db329-lib-modules\") pod \"perf-node-gather-daemonset-6bbgb\" (UID: \"531388aa-7df6-4fe3-b3bf-972ea37db329\") " pod="openshift-must-gather-7twvb/perf-node-gather-daemonset-6bbgb" Jun 15 02:49:31.813847 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:31.813823 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9772h\" (UniqueName: \"kubernetes.io/projected/531388aa-7df6-4fe3-b3bf-972ea37db329-kube-api-access-9772h\") pod \"perf-node-gather-daemonset-6bbgb\" (UID: \"531388aa-7df6-4fe3-b3bf-972ea37db329\") " pod="openshift-must-gather-7twvb/perf-node-gather-daemonset-6bbgb" Jun 15 02:49:31.865920 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:31.865881 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7twvb/perf-node-gather-daemonset-6bbgb" Jun 15 02:49:31.994091 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:31.994067 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7twvb/perf-node-gather-daemonset-6bbgb"] Jun 15 02:49:31.996131 ip-10-0-143-51 kubenswrapper[2579]: W0615 02:49:31.996098 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod531388aa_7df6_4fe3_b3bf_972ea37db329.slice/crio-f40c4cd406cbb9ab6876773045dc911fadee00f590c02c628bad488a91bc4669 WatchSource:0}: Error finding container f40c4cd406cbb9ab6876773045dc911fadee00f590c02c628bad488a91bc4669: Status 404 returned error can't find the container with id f40c4cd406cbb9ab6876773045dc911fadee00f590c02c628bad488a91bc4669 Jun 15 02:49:32.420781 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:32.420687 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8wpj7_1c70b33b-a4a3-4e1e-9693-41d77a5217fd/dns/0.log" Jun 15 02:49:32.438515 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:32.438496 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8wpj7_1c70b33b-a4a3-4e1e-9693-41d77a5217fd/kube-rbac-proxy/0.log" Jun 15 02:49:32.579363 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:32.579342 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wthd7_f96eb18e-3121-4b4e-a6fb-a0c8bd807285/dns-node-resolver/0.log" Jun 15 02:49:32.725152 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:32.725083 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7twvb/perf-node-gather-daemonset-6bbgb" event={"ID":"531388aa-7df6-4fe3-b3bf-972ea37db329","Type":"ContainerStarted","Data":"b4ae33d431c16d9eb3661aa6f395fefb15539c98679092c8e82e06cac76bfe49"} Jun 15 02:49:32.725152 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:32.725133 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7twvb/perf-node-gather-daemonset-6bbgb" event={"ID":"531388aa-7df6-4fe3-b3bf-972ea37db329","Type":"ContainerStarted","Data":"f40c4cd406cbb9ab6876773045dc911fadee00f590c02c628bad488a91bc4669"} Jun 15 02:49:32.725375 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:32.725254 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-7twvb/perf-node-gather-daemonset-6bbgb" Jun 15 02:49:32.739725 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:32.739683 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-7twvb/perf-node-gather-daemonset-6bbgb" podStartSLOduration=1.739624007 podStartE2EDuration="1.739624007s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 02:49:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 02:49:32.739042899 +0000 UTC m=+1265.606827635" watchObservedRunningTime="2026-06-15 02:49:32.739624007 +0000 UTC m=+1265.607408751" Jun 15 02:49:33.143508 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:33.143483 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9zrhr_6798a601-4421-44c1-8ef7-3b2527cad953/node-ca/0.log" Jun 15 02:49:34.183714 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:34.183686 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-59c549f7b4-gpxhv_bd72927c-a9ed-46a5-879a-b86ddfce6a3e/kube-auth-proxy/0.log" Jun 15 02:49:34.237724 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:34.237700 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-579785bc64-rngxp_007497bc-5b0b-4320-bda5-4e08f8228820/payload-pre-processing/0.log" Jun 15 02:49:34.865574 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:34.865546 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hz7mx_c8c3e57a-ae17-4f87-ba38-acc0e14583c4/serve-healthcheck-canary/0.log" Jun 15 02:49:35.438532 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:35.438489 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lfc67_a4c0951e-c8e4-496e-8761-b6856b0fefb3/kube-rbac-proxy/0.log" Jun 15 02:49:35.455408 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:35.455385 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lfc67_a4c0951e-c8e4-496e-8761-b6856b0fefb3/exporter/0.log" Jun 15 02:49:35.472519 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:35.472502 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lfc67_a4c0951e-c8e4-496e-8761-b6856b0fefb3/extractor/0.log" Jun 15 02:49:37.792575 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:37.792549 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-ddf7649c7-b4wrs_3a8bbc51-fb13-44bb-8156-acfb444405fe/manager/0.log" Jun 15 02:49:37.961886 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:37.961849 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-785bc74785-wd7m6_7e1b26b3-7e13-45d8-94d5-9cc30ef55807/manager/0.log" Jun 15 02:49:37.977017 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:37.976998 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-785bc74785-wd7m6_7e1b26b3-7e13-45d8-94d5-9cc30ef55807/copy-manifests/0.log" Jun 15 02:49:38.739492 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:38.739466 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-7twvb/perf-node-gather-daemonset-6bbgb" Jun 15 02:49:39.500997 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:39.500968 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7d59cb4d96-gj9zl_bea9d430-60c3-4aca-a77b-8caf8160f58a/manager/0.log" Jun 15 02:49:45.246365 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:45.246337 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-59c549f7b4-gpxhv_bd72927c-a9ed-46a5-879a-b86ddfce6a3e/kube-auth-proxy/0.log" Jun 15 02:49:45.283373 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:45.283349 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-579785bc64-rngxp_007497bc-5b0b-4320-bda5-4e08f8228820/payload-pre-processing/0.log" Jun 15 02:49:46.380540 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:46.380509 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5hrsv_54db0e1c-0d25-46d2-a3e3-aebf9339e272/kube-multus-additional-cni-plugins/0.log" Jun 15 02:49:46.399319 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:46.399290 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5hrsv_54db0e1c-0d25-46d2-a3e3-aebf9339e272/egress-router-binary-copy/0.log" Jun 15 02:49:46.418801 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:46.418767 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5hrsv_54db0e1c-0d25-46d2-a3e3-aebf9339e272/cni-plugins/0.log" Jun 15 02:49:46.442794 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:46.442775 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5hrsv_54db0e1c-0d25-46d2-a3e3-aebf9339e272/bond-cni-plugin/0.log" Jun 15 02:49:46.453569 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:46.453547 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-59c549f7b4-gpxhv_bd72927c-a9ed-46a5-879a-b86ddfce6a3e/kube-auth-proxy/0.log" Jun 15 02:49:46.461152 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:46.461136 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5hrsv_54db0e1c-0d25-46d2-a3e3-aebf9339e272/routeoverride-cni/0.log" Jun 15 02:49:46.479071 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:46.479052 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5hrsv_54db0e1c-0d25-46d2-a3e3-aebf9339e272/whereabouts-cni-bincopy/0.log" Jun 15 02:49:46.494373 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:46.494360 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-579785bc64-rngxp_007497bc-5b0b-4320-bda5-4e08f8228820/payload-pre-processing/0.log" Jun 15 02:49:46.496967 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:46.496947 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5hrsv_54db0e1c-0d25-46d2-a3e3-aebf9339e272/whereabouts-cni/0.log" Jun 15 02:49:46.811907 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:46.811873 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-lnjtl_e9ec7a6c-a0b6-44b8-aff6-ee7c930408b4/kube-multus/0.log" Jun 15 02:49:46.891009 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:46.890987 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-954mx_0c770be9-5ffa-4da5-bc85-58fd4c82b027/network-metrics-daemon/0.log" Jun 15 02:49:46.906865 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:46.906828 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-954mx_0c770be9-5ffa-4da5-bc85-58fd4c82b027/kube-rbac-proxy/0.log" Jun 15 02:49:48.026070 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:48.026037 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qg4vc_0b0dd3a3-9aef-4f34-8499-c293d4d81a93/ovn-controller/0.log" Jun 15 02:49:48.042655 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:48.042615 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qg4vc_0b0dd3a3-9aef-4f34-8499-c293d4d81a93/ovn-acl-logging/0.log" Jun 15 02:49:48.066093 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:48.066071 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qg4vc_0b0dd3a3-9aef-4f34-8499-c293d4d81a93/kube-rbac-proxy-node/0.log" Jun 15 02:49:48.083084 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:48.083063 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qg4vc_0b0dd3a3-9aef-4f34-8499-c293d4d81a93/kube-rbac-proxy-ovn-metrics/0.log" Jun 15 02:49:48.098407 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:48.098392 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qg4vc_0b0dd3a3-9aef-4f34-8499-c293d4d81a93/northd/0.log" Jun 15 02:49:48.115867 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:48.115849 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qg4vc_0b0dd3a3-9aef-4f34-8499-c293d4d81a93/nbdb/0.log" Jun 15 02:49:48.133119 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:48.133098 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qg4vc_0b0dd3a3-9aef-4f34-8499-c293d4d81a93/sbdb/0.log" Jun 15 02:49:48.162933 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:48.162914 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-59c549f7b4-gpxhv_bd72927c-a9ed-46a5-879a-b86ddfce6a3e/kube-auth-proxy/0.log" Jun 15 02:49:48.172727 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:48.172708 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qg4vc_0b0dd3a3-9aef-4f34-8499-c293d4d81a93/ovnkube-controller/0.log" Jun 15 02:49:48.215199 ip-10-0-143-51 kubenswrapper[2579]: I0615 02:49:48.215180 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-579785bc64-rngxp_007497bc-5b0b-4320-bda5-4e08f8228820/payload-pre-processing/0.log"